Attended Sysupgrade: keep grub.cfg edits to prevent failed boots & physical keyboard commands every upgrade

[ikjadoon]

Hello! I was curious if Attended Sysupgrade could have a toggle to keep edits to grub.cfg. Currently, I believe it only keeps config, but not boot/grub/grub.cfg.

The Problem (with examples)

Some x86 systems get stuck at Grub & fail to boot OpenWRT. Why? Its default grub.cfg has two problematic lines in grub.cfg on serial & terminal output that block boot. Thus Attended Sysupgrade requires a physical keyboard to even boot OpenWRT (otherwise, no ssh, no IP, nothing).

Example 1 (yours truly)

Example 2

Example 3

Example 4

Example 5

Example 6 (from 2019!)

Example 7

Example 8

Example 9

Example 10 (from 2018!)

What Affected Users Do Today

1. Perform Attended Sysupgrade with “keep config” on certain x86 router boxes.
2. The router fails to boot into OpenWRT. Nothing is accessible, not even ssh.
3. Workaround Step One (required every upgrade): Connect a physical USB keyboard and press enter to boot OpenWRT to move past the Grub bootloader screen (stuck due to serial).
4. Workaround Step Two (required every upgrade): Do the physical workaround first to get into OpenWRT→ use nano or a Live USB to comment out the serial lines in grub.cfg.
5. Then, your x86 router can boot / reboot directly into OpenWRT without a physical keyboard.
6. Until you run another Attended Sysupgrade: where OpenWRT resets grub.cfg to failing-boot-defaults and you need a physical keyboard to boot first → edit grub.cfg on each upgrade.

Possible Solution:

1. When users select “keep config” in Attended Sysupgrade, it ought to keep grub.cfg’s edits made by the user. “Keep config” ought not to reset grub.cfg on each Attended Sysupgrade.

The Utopian Future:

1. Perform Attended Sysupgrade with “keep config” on certain x86 router boxes.
2. Refresh your browser, voila. Done. There is no step 3.

Now, I know very little about OpenWRT. I’m not sure if this is feasible or may have adverse knock-on effects for other users. Perhaps there is a very good reason why grub.cfg is overwritten even in Attended Sysupgrade.

But, if this could be helpful to others (it took five days of troubleshooting to figure this out on my post), I hope this solution could be a good enough workaround: you edit grub.cfg once and your grub.cfg is maintained between upgrades if you tick “keep config”.

[frollic]

Tried white listing grub.cfg in /etc/sysupgrade.conf ?

instead of ASU, tried OWUT too ?

[efahl]

It is the sysupgrade program doing this, not Attended Sysupgrade. The ASU clients just invoke sysupgrade at the end of their processing at which point backups, image writing, restores and all that are out of their hands.

To further diagnose things, try poking around in the platform-specific part of sysupgrade:

$ grep '/boot' /lib/upgrade/platform.sh
                mkdir -p /tmp/boot
                mount -o rw,noatime "/dev/$bootpart" /tmp/boot
                        -d "/tmp/boot/boot/grub" \
                umount /tmp/boot
                sed -i "s/\(PARTUUID=\)[a-f0-9-]\+/\1$4$3$2$1-$6$5-$8$7-$9/ig" /mnt/boot/grub/grub.cfg

[ikjadoon]

I believe the whitelisting has worked! I added this line:

/boot/grub/grub.cfg

After upgrading to 24.10.3, I checked via nano and the edits were maintained. It took quite a while to upgrade from 23.05.6, I think definitely over 5 minutes / 300 seconds.

Not the best test, as I unfortunately still had my keyboard connected, so I hit “enter” anyways (no HDMI output ever came) and it booted after that, but I can’t tell if it was still upgrading (but I suspect so because grub bootloader does have an HDMI output) or it was because I hit enter.

But, overall, I think this is the fix! Thank you very kindly; I finally understand what this menu does.

@efahl, thank you kindly for this helpful clarification. I also now understand ASU vs sysupgrade. I will look into this, too.