Sysupgrade help for x86_64

Is there more detail on how to do this two parallel partitions? It would be great to see it documented on the wiki.

it's no rocket science, create a 2nd partition, write the openwrt image, add a 2nd entry to grub.conf / syslinux.cfg.

you probably have to adapt the root= kernel param, so each openwrt uses the correct partition.

1 Like

I'm afraid that I don't know as much as you, so I have to guess. Please let us know if there is something I haven't thought of in the following outline:

  1. First time: Follow the instructions to install ext4-combined.img.gz
  2. When resizing the root partition, change the sizing to use half the disk, not the full remaining
  3. Create an additional partition after the root, and size that to the remaining space
  4. When preparing to upgrade, extract the latest ext4-rootfs.img.gz into the additional partition
  5. Copy the settings (/etc/config, others) from current partition to the alternate
  6. Set up grub to provide an alternate boot and change the default to the alternate
1 Like

4a) write the new versions combined to the new partition.
4b) resize the FS to fill the whole partition

Looks good in PowerPoint mode :wink:

Hi,

Probably it is not rocket science and most probably i misunderstand the process, so please advise:

  1. let's start with an empty disk (sda)
  2. dd if=openwrt-21.02.1-combined-ext4.img of=/dev/sda (not efi version for simplicity)
  3. this creates two partitions, sda1 and sda2. sda1 is the boot, sda2 is the root partition. grub.cfg and vmlinuz is in sda1 (mounted as /boot), the whole os in sda2 as /.
  4. this is bootable, so far we are good.
  5. create a new partition (sda3)
  6. dd if=openwrt-21.02.0-rootfs.img of=/dev/sda3
  7. now add the new partition as new menuentry into grub.cfg
  8. during boot i can switch back & forth between the two entries. looks nice ... but!

as sda1 is the /boot for both environments the linux kernel (vmlinuz) is the same for both which is not good.
if step 5-6 is wrong and would need to add two new partitions, i.e. a new boot and new root then dd the combined image somehow, then how to configure grub to use the 1st boot+root for environment1 and 2nd boot+root for env2? and which grub.cfg to configure: the one on the 1st boot or the other one on the 2nd boot partition?

Short message, I'm about to ZzZzzzz ....

You can have several kernels in /boot, you just have to use different file names for them,
then make sure grub.conf use those specific kernel files along with the correct root=
parameter value, pointing to the root FS matching the kernel.

Another option is to write the combined image to sda, then use sda1 only for grub.
You'd dd the combined image again, to sda2.
This will always keep the grub boot environment intact on sda1, no matter whan you do with
the actual OS
Afterwards adapt grub.conf on sda1 to boot from sda2, and if you have a 2nd
installation, from sda4.

Last option
Dd combined image to sda,
Create sda3, and dd the root FS image of the 2nd install to it. Copy the stand alone
kernel for the 2nd install to /boot on sda1, and change grub.conf as described earlier.
Next time you refresh any install, you'd use this method to install it.
You cannot dd a combined image, because it'd overwrite the grub.conf for you,
along with 2nd installs kernel.

thanks for the explanation.

As of now I'm writing this on

Firmware Version
OpenWrt 21.02.1 r16325-88151b8303 / LuCI openwrt-21.02 branch git-21.327.65561-7f37a58
Kernel Version 5.4.154

Okay so I plan on testing something on the current snapshot nightly builds.
Would this work with out breaking any libraries dependencies packages configs ?

say as of writing this the current snapshot is * so I would ssh this.

cd /tmp; wget https://downloads.openwrt.org/snapshots/targets/x86/64/openwrt-x86-64-generic-ext4-combined.img.gz
sysupgrade -v /tmp/generic-ext4-combined.img.gz

Upon then preform a reboot an run...

opkg update; opkg list-upgradable; opkg upgrade netifd; opkg list-upgradable | cut -f 1 -d ' ' | xargs -r opkg upgrade

Then I should double check to see if I have more than 32MB space (issue I had in the past) after
...and if not use said package from ** to extend yeah ?

There's no sysupgrade for devices that aren't "regular routers".

You just dd the image to wherever.

So, I was playing with my APU1, and thought I'd try it myself ...

The starting point is a SD card with a snapshot image written, and I added an additional 21.02.01 to the same SD card, then rebooted into 21.02.01.

This is all done from the already running snapshot version

note: with the default size of /dev/sda1, you will only be able to fit three kernels.

you'll need:

opkg install resize2fs fdisk blkid

create new partition for the new openwrt version, I used 250MB.

fdisk /dev/sda

Command (m for help): n
Partition type
   p   primary (2 primary, 0 extended, 2 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (3,4, default 3): 3
First sector (33280-15523839, default 247808):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (247808-15523839, default 15523839): +250M

Created a new partition 3 of type 'Linux' and of size 250 MiB.

Command (m for help): p
Disk /dev/sda: 7.4 GiB, 7948206080 bytes, 15523840 sectors
Disk model: Card  Reader
Geometry: 245 heads, 62 sectors/track, 1021 cylinders
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x4958be76

Device     Boot  Start    End Sectors  Size Id Type
/dev/sda1  *       512  33279   32768   16M 83 Linux
/dev/sda2        33792 246783  212992  104M 83 Linux
/dev/sda3       247808 759807  512000  250M 83 Linux

Command (m for help): w
The partition table has been altered.
Syncing disks.

wget and write the new openwrts ext4 root FS of the new partition, resize it to fit the partition size, and check it.

cd /tmp
wget https://downloads.openwrt.org/releases/21.02.1/targets/x86/64/openwrt-21.02.1-x86-64-generic-ext4-rootfs.img.gz
gzip -d -c openwrt-21.02.1-x86-64-generic-ext4-rootfs.img.gz | dd of=/dev/sda3
root@OpenWrt:~# resize2fs /dev/sda3
resize2fs 1.45.6 (20-Mar-2020)
Resizing the filesystem on /dev/sda3 to 64000 (4k) blocks.
The filesystem on /dev/sda3 is now 64000 (4k) blocks long.

root@OpenWrt:~# fsck.ext4 /dev/sda3
e2fsck 1.45.6 (20-Mar-2020)
rootfs: clean, 1347/13312 files, 4856/64000 blocks

DL the new version kernel, and store it in /boot

wget https://downloads.openwrt.org/releases/21.02.1/targets/x86/64/openwrt-21.02.1-x86-64-generic-kernel.bin -O /boot/vmlinuz-21.02.1

get the PARTUUID of the new partition

blkid /dev/sda3
/dev/sda3: LABEL="rootfs" UUID="ff313567-e9f1-5a5d-9895-3ba130b4a864" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="4958be76-03"

add a new menu entry to grub, by cloning the existing one

vi /boot/grub/grub.conf

replace the kernel file name (default vmlinuz) with the one you used when you wget it, and update the
PARTUUID with the one from the blkid command.

menuentry "OpenWrt 21.02.1" {
        linux /boot/vmlinuz-21.02.1 root=PARTUUID=4958be76-03 rootwait console=tty0 console=ttyS0,115200n8 noinitrd
}

in grub.conf the set default="0" controls which menu entry in the config file will be booted by default,
if noone interrupts the countdown. 0 is obviously the 1st entry. The timeout="2" is the count down
timer, in sec.

reboot, and keep fingers crossed :wink:

1 Like

Decided to update to OpenWrt 21.02.2 r16495-bf0c965af0 / LuCI openwrt-21.02 branch git-22.052.50801-31a27f3

Was not too bad here is the process on: How I did it.

scp firmware_image.bin root@openwrt.lan:/tmp
opkg list-installed | cut -f 1 -d ' ' > /etc/config/x86_64-pkg.list
sysupgrade -v /tmp/firmware_image.bin
reboot
scp $backup_files.7z && 'chmod 755'
opkg update; opkg install $(cat /etc/config/x86_64-pkg.list)
reboot
opkg update; opkg remove dnsmasq ip6tables; opkg --force-overwrite install dnsmasq-full tc-full bash
reboot
opkg update; opkg list-upgradable; opkg upgrade netifd; opkg list-upgradable | cut -f 1 -d ' ' | xargs -r opkg upgrade
~ Update "SSL" Certificate. && reboot 

Everything seems to be how it was before I updated except with now I see live changes from v x.1 - v x.2

1 Like

Well I take that back.
I see the the SSD ext4 defaulted...
Went from xxxGB back to stock 32MB

No Big deal everything is still working just says

Free space:
30% 32.6 MB

Now instead of xxxGB as when I did the DD format.

Filter:

I just wanted to chime in here and thank you all for the great ideas. I bought a Sophos SG-135r1 off eBay about six months ago, under $100 shipped and I was psyched to get started. Then I read: On x86 machines, on the other hand, upgrading is more complex than the first installation.. Yay.

Knowing I wanted to have larger than normal A/B partitions (or I guess technically 2/3 partitions) from beginning. I bought a fresh SSD, used the dd method to copy 21.02.2 to the drive, expanded the root partition/filesystem to 512MB+/-. Created a second root 512MB+/- partition and dd'd the contents of sda2 -> sda3 and updated the filesystem guid. That was about all I had the stomach for, but wanted to get this unit in place to support other network upgrades I had in progress, hoping I had planned enough that I didn't need to redo everything when I wanted to upgrade next.

I learned some about Image Builder, and it seemed a little too abstract at the time. I support about a half-dozen 'dumb APs' in my network, a mix of mostly TP-Link Archer A7, CPE210, and a few others. I'm running a mix of fast roaming and mesh, which requires the non-base version of wpad, and upgrading is a bear of flashing, SSHing in, replacing wpad, installing a few other packages, etc.. I learned how to run Image Builder as a Docker image on the Sophos, and roll my own images with all the packages I need out of the gate.

Fast forward to upgrading this time around, and at a high level this was my process:

create a backup file using sysupgrade (I save it to the /dev/sda4 partition I mount to /opt)
using Image Builder: to get both a kernel and a rootfs.tar.gz you need to compile twice, once with CONFIG_TARGET_ROOTFS_PARTSIZE=512 (or large enough for all your packages) in the .config file, and then once with CONFIG_TARGET_ROOTFS_PARTSIZE=104 (the default, but smaller than it can actually build an image file), this will leave the .tar.gz in the directory when it fails. This saves the effort of extracting the rootfs from the image.

I've got a few helper scripts I'm still working on, but I do things like create a /.sda2 or /.sda3 file to help me keep track of which partition is mounted as root (since OpenWrt does remount trickery and shows stuff like /dev/root on / type ext4 (rw,noatime) in mount so you can't really tell otherwise. The gist of the root filesystem upgrade is (assuming you're running on /dev/sda2):

mount /dev/sda3 to /mnt/sda3
cd /mnt/sda3
rm -rf *
tar -zxvf /opt/imagebuilder/bin/openwrt-imagebuilder-22.03.2-x86-64.Linux-x86_64/openwrt-22.03.2-x86-64-generic-rootfs.tar.gz
cp /opt/imagebuilder/bin/openwrt-imagebuilder-22.03.2-x86-64.Linux-x86_64/bin/targets/x86/64/openwrt-22.03.2-x86-64-generic-kernel.bin /boot

(you might need to remove a third kernel as you can only really support three with the default /boot volume size)

vi /boot/grub/grub.cfg

Duplicate the menuentry lines, I generally make them read like:

menuentry "OpenWrt sda3" {
        linux /boot/openwrt-22.03.2-x86-64-generic-kernel.bin root=PARTUUID=4ce52bd8-aafd-4253-9e39-38f47211cfa1 rootwait   cons
}
menuentry "OpenWrt sda3 (failsafe)" {
        linux /boot/openwrt-22.03.2-x86-64-generic-kernel.bin failsafe=true root=PARTUUID=4ce52bd8-aafd-4253-9e39-38f47211cfa1 r
}

Change the default line to match (set default="2") and hope for the best? Personally I put a USB to serial adapter in the Archer A7 sitting next to the Sophos device and used a (Cisco?) console cable to get a grub console (via screen of all things). I generally leave the default to the known working config, hand boot up by selecting the new menu option in the serial console via screen, and it it works editing the default entry in grub.cfg.

If it boots up fine I restore the backup.

An anomaly I've found: If you change the partition GUID, which I did, but I'm not sure it's necessary, it will not properly mount the /boot partition (like at all). This is generally a simple:

mkdir /boot
mount /dev/sda1 /boot
mount --bind /boot/boot /boot

I wrote a helper script that does this, and a few other things like determine which root partition is currently mounted, and mounts the other partition to /mnt/sda2, for example.

Hopefully some or all of this makes sense, and helps the next person thinking, "wow upgrading x86_64 is too hard" because it is a lot. I'd love to see some of these concepts integrated, somehow.

I prefer having 2 separate SD Cards for the alternating installation on my x86 OpenWRT. Currently using 2x 32GB (smaller cards practically aren‘t really cheaper) and just leaving 99% on each card empty, not bothering the effort to repartition OpenWRT after applying the image.

I have an additional SSD attached, used as data-only drive, used as OpenWRT-hosted SMB file share. This way, I do not need to touch the data SSD, when upgrading OpenWRT, as OpenWRT goes onto the SD Cards only. So no migration effort for the data and no extra risk of damaging data, if the update would go south.

2x USB sticks would also do it instead of SD. Yet I prefer SD Cards: Several vendors offer write-durability enhanced SD Cards (I guess they only bin the better SD cards and sell them for a slightly higher price). They are not reasonably more expensive, so I got those. I have seen enough cheap USB sticks malfunctioning after a few weeks and I dont want to waste extra lifetime with such issues on a 24x7 router.

The BIOS on my x86 OpenWRT automatically boots from any attached USB stick/SD Card in any of its USB slots (I have a regular PC mainboard in my x86 OpenWRT and use an thumbsized SD Card adapter in one of its external USB ports on the back, so I can easily switch SD cards without opening the device.

When updating OpenWRT, I go like this:

  1. Download and apply the new OpenWRT image via my work PC on the alternating secondary SD card (SD slot on PC, using Win32 disk imager freeware).
  2. LuCi: backup config from the current OpenWRT system
  3. putty: get a list of the user-installed packages, see
    https://forum.openwrt.org/t/script-to-list-installed-packages-for-simplifying-sysupgrade
  4. swap SD Cards on the router and reboot the router with the new, but yet vanilla default OpenWRT release
  5. putty: opkg install the list on the new OpenWRT, using the list of packages as extracted in step 2
  6. LuCi: restore the config backup from step 1

1-6 takes about 5min. And if something does not work, I have a fully working fall back ready on the other (untouched) SD Card.

That forum thread from step 3 is amazing by the way. I actually wish, that the regular LuCi backup script would just include that, to write that list of user-installed packages to the backup archive as a bare text file. A restore would not automatically need to autoapply it, but simply having the list part in the config backup archive would be amazing.

2 Likes

The Attended Sysupgrade package (https://openwrt.org/docs/guide-user/installation/attended.sysupgrade) has the same function. It creates a list of user-installed packages, sends a request to sysupgrade.openwrt.org for a newly-built image with those packages, and installs it into the router. If you also Keep Settings, no reconfiguration is necessary. it's been working since 19.07 days...

I suspect you could put in your older SD card, then run luci-app-sysupgrade to get the newest image. You'd still have the prior (working) SD card as a backup...

that's why i'd suggest to change grub to use simple partitions:

menuentry "OpenWrt-21.02.3-sda2" {
        linux /boot/vmlinuz root=/dev/sda2 rootwait   console=tty0 console=ttyS0,115200n8 noinitrd
}

PARTUUID in frequently changing hardware environment might make sense but it is a router, once it is up i don't want to add disks which might change disk order. so personally the very first thing i do is replacing PARTUUID. then can resize/add partitions as needed without jeopardizing hanging during boot because UUID i forgot to update (e.g. after partition resize).

But doesn't this then bork the mount of /boot with both menuentrys? I guess maybe it would still work as long as you leave the PARTUUID the default value? For both root volumes?

Once I'm comfortable running the scripts I'm working on I guess it doesn't much matter if it auto-mounts boot for me using the normal boot process. The current hard-coded detection of a fixed PARTUUID to then mount boot seems... a unique way to do it. Not that I'm offering a better solution at this time than an out-of-band script. :slight_smile:

this is my current grub.cfg which seems to work (for me), so i guess it does not bork anything as i am using this system to reply:

$ cat /boot/grub/grub.cfg
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 --rtscts=off
terminal_input console serial; terminal_output console serial

set default="2"
set timeout="5"
set root='(hd0,gpt1)'

menuentry "OpenWrt-21.02.3-sda2" {
        linux /boot/vmlinuz root=/dev/sda2 rootwait   console=tty0 console=ttyS0                                             ,115200n8 noinitrd
}
menuentry "OpenWrt-21.02.5-sda3" {
        linux /boot/openwrt-21.02.5-x86-64-generic-kernel.bin root=/dev/sda3 roo                                             twait   console=tty0 console=ttyS0,115200n8 noinitrd
}
menuentry "OpenWrt-22.03.2-sda4" {
        linux /boot/openwrt-22.03.2-x86-64-generic-kernel.bin root=/dev/sda4 roo                                             twait   console=tty0 console=ttyS0,115200n8 noinitrd
}
menuentry "OpenWrt (failsafe) on sda2" {
        linux /boot/vmlinuz failsafe=true root=/dev/sda2 rootwait   console=tty0                                              console=ttyS0,115200n8 noinitrd

I use partition names too, so much easier, if you need to do frequent changes to the OS or grub.

You both certainly can't be wrong. :slight_smile: Well sweet baby jesus, one less thing to worry about. I just adjusted my currently booting grub menuentry to be root=/dev/sda3 and yep, boots just fine like I expected, but also automatically mounts /dev/sda1 to boot with the proper mount --bind remount. Excellent!

I think @grrr2 is just trying to make us envious with three root partitions... Showoff.

1 Like