USB drive claims "read-only file system" via SSH/LUCI

I did a Force Reload on LUCI NLBWMon and received this error notification in LUCI.

Failed to commit database: Error while processing command: Read-only file system 

I have it configured to write to USB (/mnt/sda1/nlbwmon). If I SSH into the router and do a "touch test" I get the same error. I pulled the drive and tested it. It's fine. When I put it back in and reset the router it can be written to again. What can I do to sort this out?

Check the permissions?

I'm logged in as root and all permissions are root/root.

Does mount show it is mounted ro? If it is check the log to find out why. Are you using block-mount to control mounting of the external device?

2 Likes

Here's the output of mount. I have LUCI configuring the mount, so not sure about "block-mount". LUCI is also set to check the filesystem when mounted. It worked when it was first mounted, so I don't think it's the filesystem type, and the drive isn't locked when I pull and test it in Windows.

/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
/dev/ubi0_1 on /overlay type ubifs (rw,noatime,assert=read-only,ubi=0,vol=1)
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
tmpfs on /dev type tmpfs (rw,nosuid,noexec,noatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,noatime,mode=600,ptmxmode=000)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)
none on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,noatime,mode=700)
pstore on /sys/fs/pstore type pstore (rw,noatime)
/dev/sda1 on /mnt/sda1 type vfat (ro,relatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)

I had a similar problem in the past. Isn't your flash space full?

If by flash space you mean the USB, no, it's got 1.7GB free.

This problem occurs during boot?
Are you sure that the original flash space from the router has space to load all installed packages?
I will look for, if I wrote details about and answer when I find it.

Yeah, the routers flash is 22MB used out of 74MB. As stated, I pulled the USB and tested it. It R/W just fine, then put it back in and reboot the router. It mounts R/W, but after a while the error appears and you cannot write to it anymore.

From my experience, this happens when "disk" is full.
Execute df -k to see if overlay isn't 100% used.

Have you looked at dmesg or logread?

Here's the output. Not full.

/dev/root                 8704      8704         0 100% /rom
tmpfs                   237076      1440    235636   1% /tmp
/dev/ubi0_1              75692     22372     49416  31% /overlay
overlayfs:/overlay       75692     22372     49416  31% /
tmpfs                      512         0       512   0% /dev
/dev/sda1              1953788    172736   1781052   9% /mnt/sda1
/dev/sdb1              3788896    177856   3611040   5% /mnt/sdb1

Output of dmesg. Appears to be a file system issue on the drive. I've tried to run tools to do a check/fix, but none of them appear to be installed or can be found by opkg.

[260912.244139] FAT-fs (sda1): error, fat_get_cluster: invalid cluster chain (i_pos 131734)

This thread cites a possibel FAT bug. Could OpenWRT have one? Or possibly the HNYMAN build I'm running?

Sorry, I'm not an expert with storage at all. However, I highly suggest you make backups of whatever is on that device you don't want to lose.

Of course. Already done. Whatever caused the corruption is anyone's guess, but I'll just reformat to a different file system and see how that goes.

When adding USB storage I always use ext4 format. Never had problems.

FAT32 should be okay, but exFAT might be even better. Ext{3,4} for cheap flash storage is a bad idea. For future reference: a simple fsck might fix the errors you're seeing, before resorting to a reformat.

I had to replace microSD card recently, I formatted it with F2FS, which is the de facto standard for flash on Android nowadays AFAIK.

I'm running the latest build from @HNYMAN on my R7800. It doesn't appear to have any of these file system tools installed, so I need to add those packages. I tried from the command line with opkg, but it couldn't seem to find them. I'll try from the UI.

In the meantime, I added an additional exFat USB drive about and pointed all my data there. So far, so good.

Installed packages and checked both drives. It appears that NLBWMon has issues on both drives. Here's a short summary. The exfat drive was still usable. Posting this in case there's an error in NLBWMon causing this.

FAT32 USB (mounted R/O due to these errors)

(root@OpenWrt)-(08:31)
([~])-(0 files): fsck.fat /dev/sda1
/nlbwmon/20220701.db.gz
  Contains a free cluster (303450). Assuming EOF.
/nlbwmon/20220701.db.gz
  File size is 6804 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
Reclaimed 3 unused clusters (12288 bytes).
Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
1) Remove dirty bit
2) No action
[12?q]? 1
Free cluster summary wrong (445272 vs. really 445266)

ExFat USB (Still R/W)

(root@OpenWrt)-(08:34)
([/mnt/sda1])-(10 files): fsck.exfat /dev/sdb1
exfatprogs version : 1.1.3
ERROR: /nlbwmon/20220801.db.gz: first cluster is wrong. Truncate (y/N)? n
/dev/sdb1: checking stopped. directories 50, files 144
/dev/sdb1: files corrupted 1, files fixed 0