Suddenly my LEDE turned into a read-only filesystem.
I googled and looked into OpenWrt forums cause there wasn't enough information about this situation, so I tried to
mount -o remount,rw /
or
mtd unlock rootfs_data
but nothing changed.
Is there anyone who has gone through the same or similar problem?
what does it show ?
df -Th
What device do you have ? I dont know if you have ext4 rootfs or have squashfs rom with OverlayFS, if is rootfs may be you had your ext4 corrupt, if you have overlayfs you may have failure of space or something on the overlay
sample of OverlayFS:
root@LEDE:~# df -Th
Filesystem Type Size Used Available Use% Mounted on
rootfs rootfs 12.3M 3.4M 8.9M 28% /
/dev/root squashfs 2.5M 2.5M 0 100% /rom
overlayfs:/overlay overlay 12.3M 3.4M 8.9M 28%
Sample of ext4 rootfs:
root@LEDE:~# df -Th
Filesystem Type Size Used Available Use% Mounted on
/dev/root ext4 252.0M 15.2M 231.7M 6% /
/dev/sda1 ext4 8.0M 2.5M 5.1M 32% /bootIt says :
root@LEDE:~# df -Th
Filesystem Type Size Used Available Use% Mounted on
/dev/root squashfs 8.3M 8.3M 0 100% /rom
tmpfs tmpfs 233.8M 460.0K 233.4M 0% /tmp
/dev/ubi0_1 ubifs 76.0M 3.7M 68.4M 5% /overlay
overlayfs:/overlay overlay 76.0M 3.7M 68.4M 5% /
tmpfs tmpfs 512.0K 0 512.0K 0% /dev
Does it say I'm running out of space? Then will it be solved if I erase some files on it?
I think the 100% usage is not the problem. It was already 100% before the problem occurred.
/dev/root squashfs 8.3M 8.3M 0 100% /rom
That is normal when using overlayfs.
try
touch ~/testfile
touch /overlay/upper/root/testfile2
To see if there is some error. Maybe its has to do with ubifs, sorry but I don't know much about it. http://www.linux-mtd.infradead.org/faq/ubifs.html#L_sudden_ro
Please try reboot and post here dmesg and logread outputs.
Thank you for your information.
I found there was a problem with ubifs,
[ 19.651002] UBIFS warning (ubi0:1 pid 223): ubifs_ro_mode: switched to re ad-only mode, error -2
and before that, there were also problems,
[ 19.643751] UBIFS error (ubi0:1 pid 223): ubifs_iget: failed to read inod e 1988, error -2
[ 19.643795] UBIFS error (ubi0:1 pid 223): ubifs_lookup: dead directory en try 'firewall', error -2
[ 19.651002] UBIFS warning (ubi0:1 pid 223): ubifs_ro_mode: switched to re ad-only mode, error -2
[ 19.772424] UBIFS error (ubi0:1 pid 223): ubifs_iget: failed to read inod e 1988, error -2
[ 19.773568] UBIFS error (ubi0:1 pid 223): ubifs_lookup: dead directory en try 'firewall', error -2
I'm not sure this will be enough information for you. Anyway thanks to you at least I know the problem was due to ubifs.
you were playing with uci firewall or /etc/config/firewall or something?
Yes I did.
Give me a minute, Im investigating if there is some steps to solve this problem.
What device is ? Please if you can try to open it and see if you see something damaged on the board or if you see a lot of dust/dirt or inflated capacitors...
Check your kernel version too, there is two known issues with ubi/ubifs with some specific kernel versions. uname -a
Im still investigating..
NETGEAR R7800. I bought it recently so I don't think the board was damaged...
Kernel version :
root@LEDE:~# uname -a
Linux LEDE 4.4.71 #0 SMP Sat Jun 10 07:01:27 2017 armv7l GNU/Linux
Thank you.
OK, your kernel seems to be newer..
in: http://lists.infradead.org/pipermail/linux-mtd/2010-February/029017.html
Theorically you could fix your problem doing:
find /etc/ -type f -print0 | xargs -0 head \; > /dev/null
This commands finds all files in /etc and tries to read first portions of each files, then redirect all unwanted output to null device, then do a sync a couple of times, reboot and check again your logs.
try it and see if its worked, if not you should probably will have to make a list of installed packages, do a system backup and then reflash, re install packages an restore configuration backup. There is some user somewhere that was asking for a script to backup/restore only files actually different from ROM to not fill with unnecessary files the /overlay/upper on overlayfs and save some space. Another alternative if that you restore the needed configuration manually.
Let me know if worked.
Thank you for your help. Unfortunately, it is still read-only though.
I should backup the files and factory reset it.
Anyway, thank you for your help. You taught me a lot!
Glad to help, let me know if you need more help.