OpenWrt failed to load after extroot USB drive is attached

I followed the instructions https://openwrt.org/docs/guide-user/additional-software/extroot_configuration to make my USB drive extroot.
By the way, I met a mount with the USB drive:
When I mount it without parameters, I get the next error message:

root@OpenWrt:~# mount /dev/sda1 /mnt
NTFS signature is missing.
Failed to mount '/dev/sda1': Invalid argument
The device '/dev/sda1' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

If I mount it with -t parameter, it is mounted without an error (mount /dev/sda1 /mnt -t ext4).

After I completed steps 1-6, the device stuck on loading after the first reboot.
If I remove the USB drive and restart the device, it loads successfully.

Here are the diagnostic output

root@OpenWrt:~# block info; uci show fstab; logread | sed -n -e "/- preinit -/,/- init -/p"
/dev/ubiblock0_0: UUID="a9962274-5781b041-710e6e96-15a1febb" VERSION="4.0" MOUNT="/rom" TYPE="squashfs"
/dev/ubi0_1: UUID="43ccb23c-6047-4a18-aa93-668a9c35b8c1" VERSION="w5r0" MOUNT="/overlay" TYPE="ubifs"
/dev/sda1: UUID="a6ea7c54-53de-4b73-b494-cb85aa775a55" LABEL="extroot" VERSION="1.0" TYPE="ext4"
fstab.@global[0]=global
fstab.@global[0].anon_swap='0'
fstab.@global[0].anon_mount='0'
fstab.@global[0].auto_swap='1'
fstab.@global[0].auto_mount='1'
fstab.@global[0].delay_root='5'
fstab.@global[0].check_fs='0'
fstab.@mount[0]=mount
fstab.@mount[0].target='/mnt/sda1'
fstab.@mount[0].uuid='F808FA8E08FA4B60'
fstab.@mount[0].enabled='0'
fstab.extroot=mount
fstab.extroot.uuid='a6ea7c54-53de-4b73-b494-cb85aa775a55'
fstab.extroot.target='/overlay'
fstab.extroot.fstype='ext4'
fstab.rwm=mount
fstab.rwm.device='/dev/ubi0_1'
fstab.rwm.target='/rwm'
Sun Dec  8 18:42:59 2024 user.info kernel: [    7.402092] init: - preinit -
Sun Dec  8 18:42:59 2024 kern.notice kernel: [    8.197005] random: jshn: uninitialized urandom read (4 bytes read)
Sun Dec  8 18:42:59 2024 kern.notice kernel: [    8.317740] random: jshn: uninitialized urandom read (4 bytes read)
Sun Dec  8 18:42:59 2024 kern.notice kernel: [    8.368063] random: jshn: uninitialized urandom read (4 bytes read)
Sun Dec  8 18:42:59 2024 kern.info kernel: [    8.714259] mtk_soc_eth 1e100000.ethernet eth0: configuring for fixed/rgmii link mode
Sun Dec  8 18:42:59 2024 kern.info kernel: [    8.726954] mtk_soc_eth 1e100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Sun Dec  8 18:42:59 2024 kern.info kernel: [    8.727443] mt7530-mdio mdio-bus:1f lan1: configuring for phy/gmii link mode
Sun Dec  8 18:42:59 2024 kern.info kernel: [    8.743043] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Sun Dec  8 18:42:59 2024 kern.info kernel: [   10.171588] usb 1-2: new high-speed USB device number 2 using xhci-mtk
Sun Dec  8 18:42:59 2024 kern.info kernel: [   10.355375] hub 1-2:1.0: USB hub found
Sun Dec  8 18:42:59 2024 kern.info kernel: [   10.359562] hub 1-2:1.0: 4 ports detected
Sun Dec  8 18:42:59 2024 kern.err kernel: [   10.363714] xhci-mtk 1e1c0000.xhci: drop_ep_quirk high-speed ep1in intr, mpkt:1, interval:12/256ms
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.113495] UBIFS (ubi0:1): Mounting in unauthenticated mode
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.119464] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 473
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.317334] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.325225] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.335118] UBIFS (ubi0:1): FS size: 38219776 bytes (36 MiB, 301 LEBs), max 311 LEBs, journal size 1904640 bytes (1 MiB, 15 LEBs)
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.346761] UBIFS (ubi0:1): reserved for root: 1805213 bytes (1762 KiB)
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   11.353369] UBIFS (ubi0:1): media format: w5/r0 (latest is w5/r0), UUID 43CCB23C-6047-4A18-AA93-668A9C35B8C1, small LPT model
Sun Dec  8 18:42:59 2024 user.info kernel: [   11.378943] mount_root: loading kmods from internal overlay
Sun Dec  8 18:42:59 2024 user.info kernel: [   11.587577] kmodloader: loading kernel modules from /tmp/overlay/upper/etc/modules-boot.d/*
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   12.030612] SCSI subsystem initialized
Sun Dec  8 18:42:59 2024 kern.info kernel: [   12.062708] usbcore: registered new interface driver usb-storage
Sun Dec  8 18:42:59 2024 kern.info kernel: [   12.074191] usbcore: registered new interface driver uas
Sun Dec  8 18:42:59 2024 user.info kernel: [   12.080122] kmodloader: done loading kernel modules from /tmp/overlay/upper/etc/modules-boot.d/*
Sun Dec  8 18:42:59 2024 user.info kernel: [   12.215345] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
Sun Dec  8 18:42:59 2024 user.info kernel: [   12.230714] block: extroot: device not present, retrying in 5 seconds
Sun Dec  8 18:42:59 2024 kern.info kernel: [   12.255745] mt7530-mdio mdio-bus:1f lan1: Link is Up - 1Gbps/Full - flow control rx/tx
Sun Dec  8 18:42:59 2024 kern.info kernel: [   12.255795] IPv6: ADDRCONF(NETDEV_CHANGE): lan1: link becomes ready
Sun Dec  8 18:42:59 2024 kern.info kernel: [   16.861531] usb 1-2.1: new high-speed USB device number 3 using xhci-mtk
Sun Dec  8 18:42:59 2024 kern.info kernel: [   16.994613] usb-storage 1-2.1:1.0: USB Mass Storage device detected
Sun Dec  8 18:42:59 2024 kern.info kernel: [   17.002001] scsi host0: usb-storage 1-2.1:1.0
Sun Dec  8 18:42:59 2024 user.err kernel: [   17.263385] block: extroot: cannot find device with UUID a6ea7c54-53de-4b73-b494-cb85aa775a55
Sun Dec  8 18:42:59 2024 user.info kernel: [   17.276485] mount_root: switching to ubifs overlay
Sun Dec  8 18:42:59 2024 user.warn kernel: [   17.295427] urandom-seed: Seeding with /etc/urandom.seed
Sun Dec  8 18:42:59 2024 kern.info kernel: [   17.385653] mt7530-mdio mdio-bus:1f lan1: Link is Down
Sun Dec  8 18:42:59 2024 user.info kernel: [   17.406241] procd: - early -
Sun Dec  8 18:42:59 2024 user.info kernel: [   17.409359] procd: - watchdog -
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.032980] scsi 0:0:0:0: Direct-Access     USB 2.0  Flash Disk       0.00 PQ: 0 ANSI: 2
Sun Dec  8 18:42:59 2024 user.info kernel: [   18.042238] procd: - watchdog -
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.045671] sd 0:0:0:0: [sda] 2015231 512-byte logical blocks: (1.03 GB/984 MiB)
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.054184] sd 0:0:0:0: [sda] Write Protect is off
Sun Dec  8 18:42:59 2024 user.info kernel: [   18.057993] procd: - ubus -
Sun Dec  8 18:42:59 2024 kern.debug kernel: [   18.059101] sd 0:0:0:0: [sda] Mode Sense: 00 00 00 00
Sun Dec  8 18:42:59 2024 kern.err kernel: [   18.062838] sd 0:0:0:0: [sda] Asking for cache data failed
Sun Dec  8 18:42:59 2024 kern.err kernel: [   18.068424] sd 0:0:0:0: [sda] Assuming drive cache: write through
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.163992] random: ubusd: uninitialized urandom read (4 bytes read)
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.175349] random: ubusd: uninitialized urandom read (4 bytes read)
Sun Dec  8 18:42:59 2024 kern.info kernel: [   18.220224]  sda: sda1
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.225571] sd 0:0:0:0: [sda] Attached SCSI removable disk
Sun Dec  8 18:42:59 2024 kern.notice kernel: [   18.250587] random: ubusd: uninitialized urandom read (4 bytes read)
Sun Dec  8 18:42:59 2024 user.info kernel: [   18.267851] procd: - init -

Could you please give me a hint or idea or steps how to fix the issue and make my USB drive working as extroot.

  1. backup important data on the partition

  2. delete the current partition and the MBR and if possible recreate everything from zero, choose an ext4 file system try to mount it with the command:

mount /dev/sda1 /mnt

there must be no errors ...

show:
https://www.google.com/search?q=linux++delete+MBR

https://openwrt.org/docs/guide-user/additional-software/extroot_configuration#partitioning_and_formatting

as there are some problems on this partition for a ext4 partition:

1 Like

Hello ncompact,
Thank you for your steps.
I recently tried them.

I deleted ext4 partition and mbr by
dd if=/dev/zero of=/dev/sda bs=512 count=1

parted shows me no label after that:

(parted) print free
Error: /dev/sda: unrecognised disk label
Model: USB 2.0 Flash Disk (scsi)
Disk /dev/sda: 1032MB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

Then I started from point 2 of the instruction:

2. Partitioning and formatting

DISK="/dev/sda"
parted -s ${DISK} -- mklabel gpt mkpart extroot 2048s -2048s
DEVICE="${DISK}1"
mkfs.ext4 -L extroot ${DEVICE}
root@OpenWrt:~# parted -s ${DISK} -- mklabel gpt mkpart extroot 2048s -2048s
Warning: Could not determine physical sector size for /dev/sda.
Using the logical sector size (512).
root@OpenWrt:~# DEVICE="${DISK}1"
root@OpenWrt:~# mkfs.ext4 -L extroot ${DEVICE}
mke2fs 1.47.0 (5-Feb-2023)
/dev/sda1 contains a ext4 file system labelled 'extroot'
        last mounted on /tmp/extroot/overlay on Thu Jan  1 04:00:17 1970
Proceed anyway? (y,N) y
Creating filesystem with 251392 4k blocks and 62848 inodes
Filesystem UUID: b2470cb0-8f44-4d20-aeba-a3de59ad7e51
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

Finally I hit the same issue again

root@OpenWrt:~# mount ${DEVICE} /mnt
NTFS signature is missing.
Failed to mount '/dev/sda1': Invalid argument
The device '/dev/sda1' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

You can see below how disc partitions looked before and looks now

(parted) print
Model: USB 2.0 Flash Disk (scsi)
Disk /dev/sda: 1032MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  1031MB  1030MB  ext4         extroot

Any ideas?

so let's take a step back ...

what model of router do you have (brand/model)

what model of hard disk do you have (brand/model)

how is the hard disk connected to the router (I assume direct USB without hub)

you have the possibility to redo the whole process again without choosing "gpt" as the structure but the old "mbr" structure

a document that is not inherent to openwrt but linux in general:

"what model of router do you have (brand/model)" - ASUS RT-AX53U

"what model of hard disk do you have (brand/model)" - It is a noname (I assume AData ) flash drive of 1 Gb size, more than 14 years old.

"how is the hard disk connected to the router (I assume direct USB without hub)" - Currently, It is connected directly. I also tried a USB hub previously. The result was the same.

ok

can you erase all sectors of the hard disk?

dd if=/dev/zero of=/dev/sda

and if you can change the structure from "gpt" to "mbr"

I erased all sectors on the disc and created mbr with ext4 primary partition.

root@OpenWrt:~# dd if=/dev/zero of=/dev/sda
dd: error writing '/dev/sda': No space left on device
2015232+0 records in
2015231+0 records out

parted /dev/sda
Warning: Could not determine physical sector size for /dev/sda.
Using the logical sector size (512).
GNU Parted 3.6
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
root@OpenWrt:~# parted
(parted) mklabel msdos
(parted) mkpart
Partition type?  primary/extended? primary
File system type?  [ext2]? ext4
Start? 0%
End? 100%
(parted) quit
Information: You may need to update /etc/fstab.

root@OpenWrt:~#  mkfs.ext4 /dev/sda1
mke2fs 1.47.0 (5-Feb-2023)
Creating filesystem with 251392 4k blocks and 62848 inodes
Filesystem UUID: 0ca84636-e513-4de8-bca7-8111849dab69
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

root@OpenWrt:~# mount /dev/sda1 /mnt
NTFS signature is missing.
Failed to mount '/dev/sda1': Invalid argument
The device '/dev/sda1' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

Unfortunately, the solution doesn't work for the drive.

I ran gdisk for the flash drive and found interesting info:

Expert command (? for help): p
Disk /dev/sda: 2015231 sectors, 984.0 MiB
Model: Flash Disk
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 489FFBC0-277C-4E89-A9A7-FF632F370495
Partition table holds up to 128 entries
**Main partition table begins at sector 2 and ends at sector 33**
**First usable sector is 34**, last usable sector is 2015197
Partitions will be aligned on 128-sector boundaries
Total free space is 4028 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1             128         2011263   982.0 MiB   0700  Microsoft basic data

Looks like the disk has a protected area (2-33 sectors) which keeps a link to NTFS partition.

Finally, I connected the flash drive to windows computer and recreated the NTFS partition instead of ext4

and ... almost no issues on mount

root@OpenWrt:~# mount /dev/sda1 /mnt
The disk contains an unclean file system (0, 0).
The file system wasn't safely closed on Windows. Fixing.
root@OpenWrt:~# umount /mnt
root@OpenWrt:~# mount /dev/sda1 /mnt
root@OpenWrt:~# umount /mnt
root@OpenWrt:~# mount /dev/sda1 /mnt

I'm going to try another flash drive(as no way to explicitly point the ext4 partition to mount during router boot)

I tried 2 additional flash drives ( Transcend JetFlash 110 TS256MJF110 USB 256Mb and another noname 16gb ).
Both devices have the same issue on mount after changing the partition to ext4.

Additionally, I broke /mnt mounting point during experiments:

root@OpenWrt:~# mount /dev/sda1 /mnt
NTFS signature is missing.
Failed to mount '/dev/sda1': Invalid argument
The device '/dev/sda1' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
mount: mounting /dev/sda1 on /mnt failed: No such file or directory
root@OpenWrt:~# mount /dev/sda1 /mnt
The disk contains an unclean file system (0, 0).
The file system wasn't safely closed on Windows. Fixing.
ntfs-3g-mount: failed to access mountpoint /mnt: No such file or directory
mount: mounting /dev/sda1 on /mnt failed: No such file or directory

I don't know how I can help you more, :yum:

I can only advise you to buy a USB key and proceed with all the checks on the router without going through Windows, in the hope that by deleting the usually pre-formatted FAT32 partition and creating the configuration from scratch you won't have these problems. :grinning:

Unfortunately I have no other advice to give you ... I'm sorry :yum:

here you just need to recreate the directory

mkdir /mnt
1 Like

You have to finish all the 6 steps in the extroot configuration guide. The last step (step 6) reboots the router and the /dev/sda1 would be mounted automatically.

Can you explain why you need extroot vs normal mount? Default 32MB overlay should be plenty for kmods and critical libraries permitting to store "data" on simple mount.

1 Like

Hello rwl408, Unfortunately, the router can't load back if the USB stick is in the USB port.

Is there a way to set up a permanent log to track what is going on reboot?

Hello brada4,
Good question.
How on the device with 128 MB flash I have only 33 MB available for packages (as OpenWRT may be installed to routers with even 16 MB flash).

root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 4.3M      4.3M         0 100% /rom
tmpfs                   122.2M      3.7M    118.6M   3% /tmp
/dev/ubi0_1              33.0M     30.0M      1.4M  96% /overlay
overlayfs:/overlay       33.0M     30.0M      1.4M  96% /
tmpfs                   512.0K         0    512.0K   0% /dev

From the perspective of the package:
XRay VPN solution is vital for me and requires about 16 MB. I installed it initially but can't update it now because the current amount of free space isn't enough.

Also, I want DLNA but the set of packages additionally needs 8+ MB

By the way, just hit the limit during the mass update Luci's packages

Collected errors:
 * resolve_conffiles: Existing conffile /etc/config/luci is different from the conffile in the new package. The new conffile will be placed at /etc/config/luci-opkg.
 * verify_pkg_installable: Only have 0kb available on filesystem /overlay, pkg luci-proto-ipv6 needs 3
 * verify_pkg_installable: Only have 0kb available on filesystem /overlay, pkg luci needs 1
 * verify_pkg_installable: Only have 0kb available on filesystem /overlay, pkg luci-light needs 1
 * verify_pkg_installable: Only have 0kb available on filesystem /overlay, pkg luci-mod-network needs 54

Thank you! I can mount to /mnt again

OK, got the idea, are you ready to reset and start a new era?
You will not very far with upgrading packages with opkg which takes space for both old and new versions.

You are upgrading wrong, best way to upgrade to not waste more disk space or risk soft bricks is to either compose images with imagebuilder/firmware-selector (xray core would be compressed back to 16MB once integrated into squash fs) or use attendedsysupgrade web service.
At this point you really do not need USB right away but here we go.

  • check /etc/opkg.conf - you can use your USB drive just like opkg install -d ram , you have to add new place say -d usb pointing to /mnt/sda/packages to $PATH , like export PATH=$PATH:/mnt/sda/packages/bin:....:.../usr/sbin (in every start script having commands in USB) and add respective lib/ directories to LD_LIBRARY_PATH (thus recommended to install libraries to normal overlay.

  • Openwrt 24 enables safexcel cryptographic processor which accelerates (some) cryptograph for cryptsetup and ipsec (not WG or openvpn) so you can keep your USB stick out in open public room. This will not get sysupgraded, it is the place you manually upgrade packages (finally :slight_smile: )

  • If you have flaky electricity xfs is better than ext4 as it does not need fsck at boot.

  • Any problems with auc/opkg/owut/usb you cannot figure out in wiki - we are here to help.

2 Likes

I have a few ideas to check:

  1. On the flash drives with predefined NTFS filesystem try to create one more partition in ext4 and check how it works
  2. Try flashcards 1-16 gb through USB-card reader like a flash drives
  3. There is an old 250 gb SATA HDD 2.5' . I can plug in it via SATA - USB adapter and try to follow the extroot guide again.
  4. I still don't have a clear picture of what exact errors appear when the boot is stuck with extroot USB flash drive plugged in (with all these ext4-NTFS mounting issues). As I can mount a built-in NTFS flash drive with ext4 partition if I explicitly provide mount -t ext4, I believe I can also configure it somehow for the correct mount on startup.
  5. Custom or updated firmware image with extended /overlay partition? (just an idea or a dream :slight_smile: )

1 format drive, ntfs does not have unix permissions
2 that adds more problems
3 you need powered USB hub
4. since you do not need it anymore - why bother?
5. nope, custom image with xray-core in squashfs /rom :wink: and you have 16MB /overlay left.

According to the extroot instruction

The extroot can be anything that block can mount. Currently block creates some restrictions on what extroot can be. It must a filesystem of type: ext2/3/4, f2fs, btrfs, **ntfs**, or ubifs 

Therefor I decided to go with NTFS partition on flash drive.
Unfortunately when I execute command

root@OpenWrt:~# block info ${DEVICE}
/dev/sda1: UUID="D250B84250B82F5B" LABEL="DemoNtfs" TYPE="ntfs"
root@OpenWrt:~# uci set fstab.extroot.uuid="D250B84250B82F5B"
uci: Invalid argument

Looks like extrood UUID can't be NTFS partition UUID.
How should I configure extroot in for the NTFS partition?

change to ext4 filesystem