X86 Booting two instances of OpenWrt

I'm running OpenWRT combined ext4-EFI on a Dell 9020M and a 256 GB ssd. Obviously this is far more space than needed and my OpenWRT install is only using ~5GB. To facilitate upgrades and test configurations, I was going to simply create a second instance of OpenWRT and modify the grub.cfg file to boot either instance. This is where I run out of Linux knowledge though. I'm confident I can add partitions and apply the OpenWRT image to the new partitions and even how to modify the grub.cfg file to boot the 2nd kernel, but how would I tell this 2nd kernel to look at the 2nd config partition? Am I completely off base?

The "root=" parameter to the menu item should take care of pointing it at the right partition for all disk reads, including both kernel and config.

(Edit: also, examine the "set root=" line in the main section of the file. You can override this with a different stanza inside the braces of a new menu entry. In 21.02.0 it refers to a gpt partition number.)

Was thinking of doing this myself, for similar reasons: upgrading an x86[_64] installation in the conventional way results in overwriting the partition with a new ~100MB one, clobbering any resize you've done and of course wiping out your package additions. Whereas using the existing installation to create and modify as needed a new partition for the update, then using grub to activate it when done, gives you a chance to fall back.

Haven't got around to actually doing it, though. And of course you have to remember never to do a standard upgrade again, as it will clobber your partition table and send you back to square one.

I see, so I wouldn't even need a 2nd EFI partition (makes sense), just the second data partition?

Number  Start (sector)    End (sector)  Size       Code  Name
   1             512          131583   64.0 MiB    EF00  EFI System Partition
   2          132096        10617855   5.0 GiB     0700
   3        10617856        21103615   5.0 GiB     0700  Microsoft basic data
 128              34             511   239.0 KiB   EF02

where the GUID of partition 3 = 3941692B-EA2A-432C-998F-E9882AA6C908
grub.cfg changes from

set default="0"
set timeout="1"

menuentry "OpenWrt" {
        linux /boot/vmlinuz root=PARTUUID=d0f061a8-e9cc-9f5b-24fd-9b04a3962702 rootwait   console=tty0 console=ttyS0,115200n8 noinitrd
menuentry "OpenWrt (failsafe)" {
        linux /boot/vmlinuz failsafe=true root=PARTUUID=d0f061a8-e9cc-9f5b-24fd-9b04a3962702 rootwait   console=tty0 console=ttyS0,115200n8 noinitrd


set default="0"
set timeout="1"

menuentry "OpenWrt" {
        linux /boot/vmlinuz root=PARTUUID=d0f061a8-e9cc-9f5b-24fd-9b04a3962702 rootwait   console=tty0 console=ttyS0,115200n8 noinitrd
menuentry "OpenWrt" {
        linux /boot/vmlinuz root=PARTUUID=3941692B-EA2A-432C-998F-E9882AA6C908 rootwait   console=tty0 console=ttyS0,115200n8 noinitrd
menuentry "OpenWrt (failsafe)" {
        linux /boot/vmlinuz failsafe=true root=PARTUUID=d0f061a8-e9cc-9f5b-24fd-9b04a3962702 rootwait   console=tty0 console=ttyS0,115200n8 noinitrd

and switching instances should be a matter of changing the default from 0 to 1 and rebooting?

Before changing the default I would increase the timeout to something that gives you time to respond (sometimes monitors don't sync quickly enough) and use the menu first, until you're confident everything's working properly.

Also, change the label of your new entry. "OpenWrt" is not very descriptive

Give it a try -- if you don't change the default I'm pretty sure you're not going to break your system.

So you don't have this line at the head of the file?
set root='(hd0,gpt1)'
That's fine -- I got this one but I think it's new to 21.02. If you did have this line you would want to override it in the menuentry stanza, but looks like it's not needed here.

Haha, I wasn't even paying attention to the menuentry line. Probably would have changed it as you suggest. Not too worried about breaking things, in the process of setting it up initially I took the time to put together a USB stick and a shell script that automates the process of applying an OpenWRT image, resizing sda2, applying an existing config, and modifying the grub.cfg file to drop the timeout and remove the set root line. The whole process only take as long as it does to copy and apply the new image but still requires me to go touch the box. My goal was to try and switch back and forth between the two partitions via the grub.cfg file so I didn't have to go touch it unless things went really sideways.

The set root line was causing a no boot scenario if I left the USB stick plugged in. Complained it couldn't find hd0,gpt1. Removing it cleared that problem and didn't seem to cause any new ones yet.

Hmm... just trying it myself now. It looks like boot is a separate partition, so changing that menuentry is going to get you the old vmlinuz (i.e. the kernel) but the new filesystem. Not optimal. Going to duplicate the boot partition as well and see how that goes.

Edit: I'm guessing what I'll have to do is after downloading an openwrt image, unpack it on some scratch volume, then write the two resulting partitions to separate new partitions on the router's hard drive, then update grub to boot (the root= line) from the smaller "EFI" partition, and use the other larger partition as root in the linux command line. Let you know how it goes (if you haven't got there first by then).

1 Like

This was what I was concerned about and where my linux fu was lacking. What about storing a 2nd vmlinuz image with a different name on the EFI partition rather that two "EFI" partitions? I don't have free time to play at the moment, but usually have free time to theorize. Thanks for all the info so far!

For testing purposes I merely duplicated the contents of the first two partitions into two more, then modified grub.cfg as described, inserting a new menuentry:

menuentry "OpenWrt (lame hack)" {
        set root='(hd0,gpt3)'
        linux /boot/vmlinuz root=PARTUUID=<uuid of the new /dev/sda4> rootwait   console=tty0 console=ttyS0,115200n8 noinitrd

Temporarily mounted the new partition to /mnt and added "touch /mnt/etc/config/THIS_IS_THE_NEW_ROOT_PARTITION"

Rebooted, interrupted the boot and selected the new entry. Everything seems correct (and the indicator file is present) except /boot doesn't get mounted. I don't know if it's really needed, given that the entire contents of /boot consist of:


none of which need to be in the mounted filesystem once the system is up. Nevertheless in the interest of completeness I looked around for the reason. /boot seems to get mounted in the strangely-named /lib/preinit/79_move_config, but the way in which it determines what to mount there is buried in a few included common functions which resolve to a "magic" method for deciding which partition by examining the raw disk. I'm disinclined to try to adapt or replace this sort of pure empiricism approach to coding until I really need to. Leaving the matter there for now.

(Edit: tested your suggested approach of leaving only a single boot partition to put all the kernel images in under different names or directories. Accordingly left out the "set root" line in the menuentry stanza and named a different kernel file in the linux path. Works fine and the alternate kernel image is loaded, but it still doesn't automatically mount /boot.)

Good to hear it worked seemingly as expected. I will have to do some testing of my own. Of note on the mounting of /boot. I discovered that /boot wasn't mounted on my 21.02 install as well, nor could I mount it manually. Had to add kmod-fs-vfat before I could mount it. Certainly not needed unless you want to manipulate the grub.cfg or kernel(s).

Curious. It mounts by default on my 21.02 install, only failing when I use grub to boot to a different root as described. And even then I can mount it manually. (One curiosity, the /boot volume contains a /boot directory; to keep it from becoming /boot/boot they do a bind map to its parent in that preinit script I named above.

Finally got time to play with this and it mostly took me down a Grub rabbit hole. Things I've learned:

OpenWRT should be using search --label <filesystem label> ... to locate fat formatted partitions or search --fs-uuid <partition UUID> ... to locate ext4 partitions rather than (hdX,gptX) nomenclature. This will eliminate a no boot scenario if a USB drive is plugged in at boot as the USB drive gets assigned hd0 by Grub.

The Kernel(s) can live on the EFI partition, another FAT partition, or an ext4 partition so long as the partition and path are identified correctly in the grub.cfg file. Obviously the root partition is identified by the PARTUUID.

From there as expected one can boot any number of OpenWRT instances so long as they have enough storage space and a properly configured grub.cfg file.

Haven't taken a dive into the mounting of the EFI partition at /boot yet.