How to force fsck at boot?

You may have to unmount manually...

umount -l partition name

e2fsck -y -v -f partition name

It is the root partition. I doubt i can unmount it. And even if i could, i would not have a working system anymore...

Put it in read-only mode.

1 Like

Alright, remounting in read-only mode did the trick; thank you!
Somehow i thought of ro-mode as not useful and write-protected also for fsck. But fsck and tune2fs can modify the filesystem in ro-mode.

Now i will combine it with another tip to put the check before rw-remount in:
/lib/preinit/80_mount_root

or i will wait for boot-completion and do the check with a few remounts.

So while this is possible to have the running system fsck'ed i wonder why the process of checking and fixing is not supported by LEDE/OpenWRT on its own? Sudden powerloss accidents do happen from time to time ...

2 Likes

Hi connor2,

I am looking a way to check rootfs before it is mounted, could you give me some clues or give me some instructions to do that.

Thanks

Tuan Nguyen

I'm not sure which exact command @conner2 used but I had success on the Raspberry Pi (running 17.01 based system) by modifying the /lib/preinit/80_mount_root file adding this new line (you need to have the e2fsprogs package installed):
/usr/sbin/e2fsck -y -v -f /dev/mmcblk0p2
before this existing line:
[ "$INITRAMFS" = "1" ] || boot_hook_add preinit_main do_mount_root

Also I tested the in-built "check filesystem" option under the menu System->Mount Points but it didn't appear to work and in most cases gave a 'Bad Gateway' error after which the only recovery path was a reboot/power cycle.

Anyway it would be great if there was working support for filesystem checks in OpenWRT/LEDE as some storage systems and set ups aren't quite so reliable.

1 Like

EXT4 is a journaling filesystem, after a power outage the journal will be replayed during the next mount.

On desktop systems, it behaves like this...on OpenWRT/LEDE it does NOT!

As stated in the initial post, no method whatsoever will enable this at boot for the root partition.

My solution is to drive the system in read-only mode for all filesystems that reside on the sd-card. I additionally mount an USB-stick with ext4 to write stuff. There it behaves like you would expect and properly recovers from power failures and such incidents.

When a file-system has gone to read-only state, you can fsck and repair it from within a running running system...even if it is the root partition. And then you can force it to stay read-only as i have explained above. Or you need to take out the sd-card, check and repair it on a desktop system and then prevent it from going to write-mode ever again if you want to protect it.

1 Like

add fsck in /lib/preinit/80_mount_init to check the disk every time it boots
the modified file looks like

#!/bin/sh
# Copyright (C) 2006 OpenWrt.org
# Copyright (C) 2010 Vertical Communications

do_mount_root() {
#code for checking disks
        if [ -x '/usr/sbin/fsck.ext4' ]; then
                if [ -e '/dev/sda1' ]; then
                        echo "Checking disk /dev/sda1...."
                        /usr/sbin/fsck.ext4 -y /dev/sda1
                fi
                if [ -e '/dev/sda2' ]; then
                        echo "Checking disk /dev/sds2..."
                        /usr/sbin/fsck.ext4 -y /dev/sda2
                fi
        fi

        mount_root
        boot_run_hook preinit_mount_root
        [ -f /sysupgrade.tgz ] && {
                echo "- config restore -"
                cd /
                tar xzf /sysupgrade.tgz
        }
}

[ "$INITRAMFS" = "1" ] || boot_hook_add preinit_main do_mount_root

2 Likes

Just a side note here:

According to https://openwrt.org/docs/techref/block_mount, through configuring

option 'enabled_fsck' '1'

shall work.

1 Like

can you translate that to simpler terms?. The reference link is way over my head. I have a USB3 external drive and would like to have the fsck run on boot. Where exactly is this line placed (in what config file)? Thanks.

You can also configure these things on the web GUI, under the "Mount Points" menu.

Oh, wow, that was easy. Under the "System", "Mount Points" following down the list of topics there is a section "Mount Points" with the USB drive is in the list and is checked if it is enabled. If you click the "Edit" option, then on the "Advanced settings" is a check box for "Run system file check" (on boot). Excellent! Thanks!!

1 Like

Warning:

After I did that, my router is in readonly mode. And I have no way to put it back.

How to know it's getting executed?
I added prints with "logger "Checking disk /dev/sda1...."; I don't see these in the logread output.

Try dmesg | grep sda

1 Like
root@~# dmesg | grep sda
[    1.557888] sd 0:0:0:0: [sda] 31277232 512-byte logical blocks: (16.0 GB/14.9 GiB)
[    1.558485] sd 0:0:0:0: [sda] Write Protect is off
[    1.558744] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    1.558999] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    1.562788]  sda: sda1 sda2
[    1.563935] sd 0:0:0:0: [sda] Attached SCSI disk
[    1.568100] EXT4-fs (sda2): mounted filesystem without journal. Opts: (null)
[    4.958188] EXT4-fs (sda2): re-mounted. Opts: (null)
[    4.987387] EXT4-fs (sda1): mounted filesystem without journal. Opts: (null)

I don't see the prints I have added in the /lib/preinit/80_mount_root file.

However, to check I installed tune2fs, using that I see that:

tune2fs -l /dev/sda1 | grep Last
Last mounted on:
Last mount time: Tue Jul 25 23:54:51 2023
Last write time: Tue Jul 25 23:54:51 2023
Last checked: Tue Jul 25 22:08:39 2023

For some reason, Last checked is not updating on every reboot.

After I installed tune2fs, just to get the "Last checked" stats, I notice that "tune2fs -c 1" is required to execute the block of code given above in /lib/preinit/80_mount_root file.

In summary, below worked for me.

  1. Have the check using /usr/sbin/fsck.ext4 -y /dev/sda1 in /lib/preinit/80_mount_root as suggested.
  2. tune2fs -c 1 /dev/sda1 ; tune2fs -c 1 /dev/sda2
  3. Confirm disck check happening at boot using command tune2fs -l /dev/sda1 | grep Last

I appreciate if some expert vets my steps are correct.
I am a complete noob on filesystems.

Preinit is run before any mounts and before the system logging is enabled. Thus "logger" likely fails. (kernel log is already active).

As it is run so early, no change from overlay can be used, and the script change needs to be in the original rootfs in the compiled/flashed firmware. (at least with a normal NAND flash with overlay. Not sure about SD cards, eMMC etc,)

2 Likes

Thank you for the comment. I got the part that logger is a bit too late, didn't understand other parts of your post.

So anyway, the below output tells me filesystem check is happening at every reboot, and I am all set?

reboot time is matching with Last checked time.

~# tune2fs -l /dev/sda1 | grep Last
Last mounted on:          <not available>
Last mount time:          Wed Jul 26 00:25:44 2023
Last write time:          Wed Jul 26 00:25:44 2023
Last checked:             Wed Jul 26 00:25:44 2023