OpenWrt Forum Archive

Topic: Extroot Pivot Overlay problem: Read-only file system

The content of this topic has been archived on 30 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Post #1
arneboon
30 Jun 2015, 10:43

Hi there,

Thanks for the awesome work on OpenWrt.

I finally succeeded in creating an extroot pivot overlay configuration with a usbstick following the instructions (http://wiki.openwrt.org/doc/howto/extroot, section Chaos Calmer). Read all the docs and posts I could find on extroot.

Then I ran into the problem of a Read-only file system in the extroot configuration. This while it was explicitly set to 'rw,sync' for the overlay 'mount' in the fstab configuration.

No files can be created on /root or any other folder (except the /tmp folder). As a result no packages can be installed, no applications can be upload, can not connect to the internet since no configuration files can be written. Extroot cannot be used like this.

I found one post on stack-overflow about someone meeting this issue (https://stackoverflow.com/questions/310 … s-readonly).

Here you find a gist containing (https://gist.github.com/arneboon/0f1230180d23dcad0b78):
a) the exact steps i took for creating the pivot overlay
b) logs with verious commands while inside the pivor overlay through ssh

One thing that pops out from the dmesg log:

[    0.520000] 0x0000000ff358-0x0000003f0000 : "rootfs"
[    0.530000] mtd: partition "rootfs" must either start or end on erase block boundary or be smaller than an erase block -- forcing read-only
[    0.540000] mtd: device 2 (rootfs) set to be root filesystem

Very much hoping anyone has a solution or suggestion about how to move forward. Thanks in advance!

Best regards from the Netherlands,
Arne.

(Last edited by arneboon on 30 Jun 2015, 11:52)

Post #2
zloop
30 Jun 2015, 19:37

Your USB stick has a defective filesystem. This forces a read only mount.

form your gist:

EXT4-fs (sda1): Remounting filesystem read-only

Post #3
arneboon
1 Jul 2015, 12:12

hi zloop,

thanks for your response and pointing out the filesystem defect.

i have reformatted the stick and run a filesystem check on gparted, which claims the partition is correct. see gist here:
https://gist.github.com/arneboon/0f1230 … stem-check

i made a clean openwrt install and redid all the steps to create a pivot overlay with fstab. at boot the output from dmesg is this:
https://gist.github.com/arneboon/0f1230 … ep-ext4-fs

root@wifibox:~# dmesg | grep EXT4-fs
[    8.890000] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts:
[    9.470000] EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts:
[   30.490000] EXT4-fs error (device sda1): __ext4_journal_start_sb:62: Detected aborted journal
[   30.500000] EXT4-fs (sda1): Remounting filesystem read-only
[   31.110000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[   32.100000] EXT4-fs (sdb3): recovery complete
[   32.100000] EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts:
[   38.020000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   38.030000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   70.690000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm dropbear: reading directory lblock 0
[   70.860000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #12: comm ash: reading directory lblock 0
[   83.960000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[   83.970000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm ls: reading directory lblock 0
[  157.320000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[  162.270000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[  163.750000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[  199.150000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[  207.780000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm touch: reading directory lblock 0

it shows that again the sda1 filesystem is rejected and mounted as read-only. no files can be written (obviously, but tested).
the sda3 'data-usb' filesystem is mounted correctly however. and files can be created (tested).

i will try to use the correct sda3 partition for overlay, lets see how that works out.

and i will try reformatting the partitions and checking them again. and try another usb stick.

any other suggestions? thanks.

Post #4
arneboon
1 Jul 2015, 12:32

using the correctly mounted sda3 partition does not make a difference.
now the sda1 partition is mounted correctly and the sda3 partition is rejected.

root@wifibox:~# dmesg | grep EXT4-fs
[    8.900000] EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts:
[    9.370000] EXT4-fs (sda1): recovery complete
[    9.370000] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts:
[   30.020000] EXT4-fs error (device sda3): __ext4_journal_start_sb:62: Detected aborted journal
[   30.030000] EXT4-fs (sda3): Remounting filesystem read-only
[   30.250000] EXT4-fs (sdb1): recovery complete
[   30.250000] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts:
[   30.820000] EXT4-fs warning (device sda3): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[   41.110000] EXT4-fs error (device sda3): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   41.120000] EXT4-fs error (device sda3): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   41.460000] EXT4-fs error (device sda3): ext4_find_entry:1309: inode #2: comm dropbear: reading directory lblock 0
[   41.730000] EXT4-fs error (device sda3): ext4_find_entry:1309: inode #12: comm ash: reading directory lblock 0
Post #5
arneboon
1 Jul 2015, 12:52

also no luck with partitions using ext3

root@wifibox:/data-usb# dmesg | grep read
[    0.510000] mtd: partition "kernel" must either start or end on erase block boundary or be smaller than an erase block -- forcing read-only
[    0.530000] mtd: partition "rootfs" must either start or end on erase block boundary or be smaller than an erase block -- forcing read-only
[    1.180000] VFS: Mounted root (squashfs filesystem) readonly on device 31:2.
[    5.850000] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   30.940000] sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   31.080000] EXT4-fs (sda1): Remounting filesystem read-only
[   31.980000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[   40.820000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   40.830000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   59.640000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm dropbear: reading directory lblock 0
[   59.810000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #8193: comm ash: reading directory lblock 0
root@wifibox:/data-usb# dmesg | grep ext
[    8.900000] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem
[    9.500000] EXT4-fs (sda3): mounting ext3 file system using the ext4 subsystem
[   31.070000] EXT4-fs error (device sda1): __ext4_journal_start_sb:62: Detected aborted journal
[   31.500000] EXT4-fs (sdb3): mounting ext3 file system using the ext4 subsystem
[   31.980000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[   40.820000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   40.830000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   59.640000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm dropbear: reading directory lblock 0
[   59.810000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #8193: comm ash: reading directory lblock 0
Post #6
arneboon
1 Jul 2015, 13:35

no luck with ext2 either:

root@wifibox:~# touch test.js
touch: test.js: Input/output error
root@wifibox:~# dmesg | grep sda1
[    5.960000]  sda: sda1 sda2 sda3
[    8.900000] EXT4-fs (sda1): mounting ext2 file system using the ext4 subsystem
[    8.920000] EXT4-fs (sda1): mounted filesystem without journal. Opts:
[   32.560000] EXT4-fs warning (device sda1): __ext4_read_dirblock:908: error reading directory block (ino 2, block 0)
[   33.240000] EXT4-fs error (device sda1): __ext4_get_inode_loc:4028: inode #49175: block 196611: comm block: unable to read itable block
[   33.250000] EXT4-fs error (device sda1) in ext4_reserve_inode_write:4957: IO failure
[   36.030000] EXT4-fs error (device sda1): __ext4_get_inode_loc:4028: inode #49184: block 196611: comm dnsmasq: unable to read itable block
[   36.040000] EXT4-fs error (device sda1) in ext4_reserve_inode_write:4957: IO failure
[   36.980000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   36.990000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm uhttpd: reading directory lblock 0
[   50.970000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #2: comm dropbear: reading directory lblock 0
[   51.130000] EXT4-fs error (device sda1): ext4_find_entry:1309: inode #49153: comm ash: reading directory lblock 0
[   61.240000] EXT4-fs error (device sda1): ext4_read_inode_bitmap:175: comm touch: Cannot read inode bitmap - block_group = 2, inode_bitmap = 65537
[   65.920000] EXT4-fs error (device sda1): __ext4_get_inode_loc:4028: inode #49175: block 196611: comm kworker/u2:1: unable to read itable block
[   70.940000] EXT4-fs error (device sda1): __ext4_get_inode_loc:4028: inode #49184: block 196611: comm kworker/u2:1: unable to read itable block
Post #7
tmo26
1 Jul 2015, 18:32

Have you tried increasing "option    delay_root    '5'"? (10..30sec)
What happens if you set your fstab to "enabled_fsck    '1'" for the /overlay mount?
What happens if you omitt "option    options    'rw,sync'" in your fstab?
Have you tried partitioning and formatting with fdisk and mkfs.ext4?
Have you tried partitioning and formatting **on your MR3020** i/o in a VM?

Post #8
tmo26
1 Jul 2015, 18:43

grep your dmesg for "USB disconnect"

The discussion might have continued from here.