I am having trouble keeping a USB drive installed/mounted on my OpenWrt router (Archer C7, OpenWrt 19.07.0, r10860-a3ffeb413b - a snapshot release). I am aware of an earlier topic (that I also started), but these symptoms are a bit different and more clear, so I'm creating a new topic.
I have a 1 GByte USB memory stick mounted as /opt on my router. I formatted/mounted the drive using the Quick Start for Adding a USB drive wiki page.
I installed files and software onto the drive (YAMon4, see Github issue), and it was working successfully until I had a power failure on Tuesday. At that point:
I ssh'd into the router after the power failure, ls -al /dev/sd* did not show any entries. ls -al /opt showed no files or directory
Unplugging, waiting 5 seconds and re-plugging the USB stick didn't change - ls ... was empty
Rebooting the router didn't change anything - ls ... remained empty
Plugging the USB stick into a separate Unix box showed the expected directory structure which seemed OK. Files in YAMon4/data were last modified this Tuesday (power failure day)
After re-inserting it into the OpenWrt router, ls -al /dev/sd* and ls /opt/YAMon4 showed files as expected
Other symptoms:
This has happened on at least two OpenWrt routers with different firmware versions.
This has happened many times across the last 18-24 months, with multiple installs/versions of YAMon.
I could believe, but don't have enough evidence to prove, that each of those failures was related to a power failure. (Our power goes out frequently - I don't know if every failure caused a loss of the USB stick.)
So, am I "doing something wrong" with this USB Drive? What could cause the USB drive to get unmounted and not be re-mounted?
PS I also wrote the USB Quick Start wiki page, even though I don't understand all the in's and out's of that stuff. Maybe the Quick Start needs to be improved... Many thanks!
Thanks for this information. It's working OK now, but next time it happens...
I think you're saying I should stop the YAMon service then umount and fsck? True? (That's the only process that's writing to /opt...)
Should it be fsck.ext4 -f /dev/sdaorfsck.ext4 -f /dev/sda1?
Does the Quick Start touch all the bases? (I'm happy to revise if it could be better...)
For the record, here's the output when things are working:
root:~# ls -al /dev/sd*
brw------- 1 root root 8, 0 Dec 31 1969 /dev/sda
brw------- 1 root root 8, 1 Dec 31 1969 /dev/sda1
root:~# ls -l /sys/block
lrwxrwxrwx 1 root root 0 Aug 7 11:16 mtdblock0 -> ../devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/mtd0/mtdblock0
lrwxrwxrwx 1 root root 0 Aug 7 11:16 mtdblock1 -> ../devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/mtd1/mtdblock1
lrwxrwxrwx 1 root root 0 Aug 7 11:16 mtdblock2 -> ../devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/mtd2/mtdblock2
lrwxrwxrwx 1 root root 0 Aug 7 11:16 mtdblock3 -> ../devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/mtd3/mtdblock3
lrwxrwxrwx 1 root root 0 Aug 7 11:16 mtdblock4 -> ../devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/mtd4/mtdblock4
lrwxrwxrwx 1 root root 0 Aug 7 11:16 mtdblock5 -> ../devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/mtd5/mtdblock5
lrwxrwxrwx 1 root root 0 Dec 31 1969 sda -> ../devices/platform/ehci-platform.1/usb2/2-1/2-1:1.0/host0/target0:0:0/0:0:0:0/block/sda
Thanks again.
PS re: using a UPS... I'm too cheap, and if it's a failure on loss of power, it only delays/makes it less frequent. (What if my UPS runs out of power...)
Yes, assuming this is the only process, otherwise umount fails until you search open files and terminate all the related processes:
lsof | grep -e /opt
It depends on the partitioning scheme if any.
It should be fine, although I'd move the option anon_mount somewhere to extras.
In addition, a brand new device may have no partitions, i.e. just /dev/sda.
You can skip partitioning and format the whole drive to the desired file system if you don't plan to use it elsewhere.
I am so glad to get help from someone who knows so much more about this than I do. There's a gold mine of information in what you said, if I can only suss it out...
I feel a need to add external storage to OpenWrt to hold stats (YAMon, collectd, etc.) That data can go into a single partition on external storage so that it doesn't burn out the flash of the router itself. I don't envision that people will ever want to remove the flash drive - it'll just sit there saving data 'til the router dies.
Here's my goal: I want a simple procedure with a small number of "no option" steps that Just Work. It would be best if it had this outline:
Insert the USB drive
Type a couple commands into the router's command line
Voilà! Your USB drive is ready to hold data [1]
What commands/steps could we take out from the Quick Start and still leave it bullet-proof? (You mentioned removing the "anon_mount" stuff; formatting the whole drive; not worrying about partitions; what else?) Thank you!
It's easier if you prepare (partition and format) the USB stick externally, on a desktop linux system; yes all the necessary packages are also available for OpenWrt (but not normally preinstalled).
nuke the USB stick:
# wipefs -a /dev/sdX
partition:
# fdisk /dev/sdX
Welcome to fdisk (util-linux 2.36).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x3d3c513a.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-262143, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-262143, default 262143):
Created a new partition 1 of type 'Linux' and of size 127 MiB.
Command (m for help): t
Selected partition 1
Hex code or alias (type L to list all): 83
Changed type of partition 'Linux' to 'Linux'.
Command (m for help): w
The partition table has been altered.
Syncing disks.
format:
# mkfs -t ext4 /dev/sdX1
Now it's ready to configure via blockd, using /etc/config/fstab.
I realize I am probably asking for mutually contradictory tasks in my "quest for simplicity". But I'm gonna ask anyway For example:
The opkg install ... command in the Quick Start already loads a bunch of packages, specifically, opkg install block-mount e2fsprogs kmod-fs-ext4 kmod-usb-storage kmod-usb2 kmod-usb3
I would be happy to add new packages, but I'm also aware that each additional package consumes space.
It looks as if neither wipefs and fdisk are in the default image. So we need to balance the memory consumption against the value for future use (Is it needed just once? maybe less useful).
"All we need to do..." is blow away everything on the USB drive to create a known partition format. It would be great to find a package with a one-liner that does this.
The current procedure says pre-format the USB on a laptop, then uses mkfs - is there anything simpler?
I use gparted on a laptop to partition and format the drive, then e2fs-progs is not absolutely needed on the router, though it is good to have fsck available.
kmod-usb-storage and kmod-fs- are always needed. The base usb host kmod (usb2, usb3, dwc-otg etc) should be in the build already.
With block-mount and a proper configuration in /etc/config/fstab (not the old direct way /etc/fstab), mounting should be automatic either on bootup or hotplugging the drive.
wipefs is kind of optional, but it's the easiest/ fastest method to clean up old cruft (partitioning, filesystem signatures, etc.) from a disk (if your disk's contents are sane, you can also use fdisk to delete- or just change the partition type of the existing partitions, but this requires assessing the existing status of the disk first - not good for a 'simple' guide; cat /dev/zero >/dev/sdX would basically achieve the same, but especially with today's slow, but large, USB sticks this can take half and hour or more, while wipefs is instant). fdisk is basically the easiest method (yes, using GPT/ gdisk might also be an option, but…) to set up the partitions on the device in question.
This feels like progress: I'm aiming toward a minimal set of steps that newcomers can do reliably, using information they already know. (That is, they read the step, and say, "Oh, I know how to do that.")
For example, tell the reader they must format the USB stick on their laptop/desktop. (There are a zillion descriptions of how to format a USB stick on the intertubes.) They then avoid installing or horsing around with wipefs, fdisk, gparted, or gdisk.) Does it matter what format/partition scheme they use?
Winnow down the set of packages to install, removing those already in the base system. (I don't mind giving them a few commands they can paste into the command line, even if they don't understand them.) For example, opkg install block-mount e2fsprogs kmod-fs-ext4 kmod-usb-storage
So now they have a blank USB stick in the OpenWrt computer, and a bunch of packages that are ready to do their bidding.
What are the next steps for formatting/mounting the drive? (What changes - additions or removals - would you recommend for the Quick Start?) Thanks
Almost every drive less than 64 GB or so comes from the factory formatted with its entire space one partition holding a (mostly) empty FAT32 filesystem. Theoretically you can take the drive out of the package and mount it with kmod-fs-vfat. Of course that filesystem doesn't offer Linux features like file owner and permission and symbolic links.
The f2fs system should also be considered, it is designed for less wear on the flash than ext4.
The fastest way is just to enable anon_mount which should work together with auto_mount already enabled by default to mount automatically any drive we connect to the router with no need for additional configuration.
However, this way does not allow using specific mount points and the mount point name is not really persistent, i.e. it depends on the partitioning as well as the connected device order if we have more than one device.
Also note that a good how-to should solve a single task and do not mix different tasks.
Partitioning and formatting are separate tasks which should be either moved to extras or split into another how-to.
There is an option in the fstab uci config to enable fs checking of specified mount point on mount time
check_fs boolean no 0 run e2fsck on device prior to a mount
It can also be set from System --> Mount Points
in Luci web interface.
This should probably be enabled to avoid the issues you are facing. You obviously need e2fsck for this to work on ext4
using a UPS... I'm too cheap, and if it's a failure on loss of power, it only delays/makes it less frequent. (What if my UPS runs out of power...)
While an UPS is not a solution to this issue, in general, having an UPS in places where electricity is spotty like that will protect them from voltage spikes and other disturbances in the power lines.
There are also plenty of cheaper devices called "12V UPS", that are basically a slightly modified power bank with lithium power cells that is "charging" all the time and it just feeds 5/9/12/15/21 volts DC to a device, there is no inverter for DC-AC conversion.
They are great for smaller stuff like network devices and home automation.
tell the reader they must format the USB stick on their laptop/desktop.
The simplest way is to have the users format their drive from a PC with whatever filesystem they want, then have them connect the drive to OpenWrt device and doing the format to ext4.
There is really no point in partitioning and wiping filesystems from OpenWrt with command line.
