My setup is a Raspberry Pi 4B with the built in ethernet port for lan and a TP-Link UE300 for wan using a squashfs image which works great with CAKE and so far has been a very positive experience overall.
When I upgraded from 21.02.0 to 21.02.01 today my initial plan was to make a settings backup, download the new stock image and corresponding kmod-mii, kmod-usb-net, kmod-usb-net-rtl8152 (to get the internet connection via usb-lan adapter running), flash/install them, then install the rest of the custom packages via luci and finally restore the settings backup.
Since the "Keep settings" checkbox was selected by default during the sysupgrade, which from my understanding of the keep settings wiki should have been fine since this is a minor version increment and only kept the settings, I was tempted to try it.
The "Skip from backup" checkbox was not selected by default and since I was not familiar with it, I don't remember it from earlier upgrades (snapshot - RC4 - stable) and I didn't see it mentioned in the sysupgrade wiki, I kept it unselected.
After the upgrade was complete I was surprised to find that not only my settings but also the custom packages had been retained. If I remember correctly the upgrade from RC4 to stable deleted the custom packages.
Unfortunately the usb-lan adapter was not working so I tried uploading the new kmod-mii from my local host which gave me the following error message:
This was unexpected as the release notes said that the kernel was updated from 5.4.143 to 5.4.154 and wolfssl from 4.7.0 to 4.8.1.
From what I could understand there seemed to have been a version mismatch and if I remember correctly wolfssl displayed with version 4.7.0.
My workaround was to reset to defaults and then proceed with my initial plan which had the router back up running within a couple of minutes.
So here are my questions/user experience feedback:
Is it generally advised to unselect the "Keep settings" checkbox and if yes why is it selected by default?
What does the "Skip from backup" checkbox do? Would it have made sure that the correct wolfssl version was installed and if yes why isn't it also selected by default or get's automatically selected when the "Keep settings" checkbox is selected by the user?
Might it be possible to implement tooltips/links to the wiki in the sysupgrade form that explain the checkboxes functions in more depth to the user?
I never had the Skip from backup checkbox show when I upgraded, just Keep and Include, which I checked both.
All I needed was to reload all add-ons.
In order to access the Internet from the existing ethernet lan port to get my usb driver and add-ons, I use the following lines in /etc/config/network :
config interface 'lan'
option proto 'static'
option ipaddr '192.168.1.x'
option netmask '255.255.255.0'
option ip6assign '60'
option device 'br-lan'
# option gateway '192.168.1.y'
# list dns '192.168.1.y'
where x=static address for network lan port and y is static address to main working router/gateway (I always upgrade my OpenWrt devices inside a working network with internet access).
Just uncomment the last two lines to become a full network client with internet access from the ethernet lan port and re-comment when returning to router mode with usb dongle as wan port, changing x if needed.
This way you load everything from official repo without any versioning issues.
Of course if your RPI4 is your only router, this may not work out for you.
I see that forgot to mention that I use the RPi4 as my only router and the previous router as wifi access point only which would have explained why a go about it this way.
It still is a great tip though and I didn't know this could be as easy as un-/commenting two lines so I will keep this in mind.
The three packages I needed for the lan adapter I got from https://downloads.openwrt.org/releases/21.02.1/targets/bcm27xx/bcm2711/packages/
which to my understanding is where opkg also looks for them.
And as I mentioned after a reset which if I understand it correctly puts the router in the same state as it would have been had I unselected the "Keep settings" checkbox those packages got everything running.
It's just that I'm still to new to all of this that I'm not sure whether I remembered incorrectly and the checkbox was meant to behave in a way that not only kept the settings but also the old packages or the checkbox did something different than it was meant to do.
To be clear, it was in no way a big problem! As I said after a couple more minutes I had everything up and running, but it made the user experience less polished and a little more clunky than I think it had to be.
Either way a better description of the checkboxes intended function, ideally right in the sysupgrade form and maybe different default selections would alleviate this in my opinion.
Except the as of yet still empty /dev/mmcblk0p3 partition everything should be the way the initial image got flashed onto the SD card.
I'm not aware of doing anything as advanced as setting up an extroot.
When it comes to OpenWRT I'm pretty much an end user who relies on the wiki and uses the GUI for almost everything.
I do however work as a software tester and would be happy to try and reproduce the behaviour of the sysupgrade from 21.02.0 to 21.02.1 and test the checkboxes on the next weekend if that helps.
Though I'd probably need some pointers as to what to do (apart from recreating the initial state as closely as possible) and what to look for exactly.
In case I might have done something wrong during setup/update:
As promised I did some tests today:
In short unselecting the "Keep settings" checkbox did nothing, both my settings and custom packages were kept!
Neither did selecting both the "Keep settings" and the "Skip from backup" checkbox or flashing the factory image instead of the sysupgrade image make any difference.
Using sysupgrade in luci I could not get rid of the installed packages.
Am I just missing something and this is an intended feature of a squashfs installation?
The "Perform reset" button was the only way to restore the initial state/clean state after the sysupgrade (or downgrade).
Full version of what I did:
I used a spare/different SD card than the one with my main installation and instead of first flashing the snapshot version from march and then upgrading to RC4 and then to 21.02.0 as I did on my main installation, I directly flashed the 21.02.0 stable factory squashfs image with Balena Etcher (Raspberry Pi Imager had failed to validate the flashed image for some reason).
With fdisk and mkfs.f2fs I created the third partition (/dev/mmcblk0p3) with the empty space of the SD card as to recreate the initial state on my main installation as closely as possible.
For the first test I used the 21.02.1 sysupgrade sqashfs image and selected the "Keep settings" and the "Include in backup a list of...".
After the upgrade the version was displayed as 21.02.1 in the GUI but not only were all the custom packages I had installed in steps 2 and 3 still present, the kernel and libwolfssl versions were shown as 5.4.143-1, not 5.4.154-1 and 4.7.0-stable-2, not 4.8.1-stable-4 respectively.
The wan interface (usb lan adapter) was not working and the necessary 21.02.1 kmod packages could not be installed. I presume due to a version mismatch with the kernel and libwolfssl packages.
I then downgraded to 21.02.0 using the sysupgrade image I had used on my main installation to upgrade from RC4 and recreated the initial state (steps 2 - 4).
This time I unselected the "Keep settings" checkbox and flashed the 21.02.1 sysupgrade image again.
=> The result was the same as in step 7.
I downgraded to 21.02.0 using the sysupgrade image and found all my settings and custom packages still present.
This time I selected both the "Keep settings" and the "Skip from backup" checkbox when I upgraded with the 21.02.1 sysupgrade image.
=> Again the result was the same as in step 7.
I downgraded to 21.02.0 this time using the factory image from step 1.
=> Same result as in step 12.
I then upgraded using the 21.02.1 factory squashfs image, unselected the "Keep settings" checkbox and forced the upgrade.
=> Again, same result as in step 7.
=> The only way I could obtain a clean state after a sysupgrade was to "Perform reset" after the upgrade. Otherwise the settings and custom packages (with their old version numbers) were kept and the old kernel and libwolfssl version was still present.
=> Is this normal for sqashfs or minor upgrades and I just did not know this?
After further testing I now believe to have found a bug in the sysupgrade process that's specific to squashfs installations and only present when upgrading from 21.02.0 stable to 21.02.1.
However I don't know where and how to report it for the devs to get notified.
The methodology was kept mostly the same as in yesterdays post with the exception of when I flashed ext4 21.02.0 stable or squashfs 21.02.0 RC4 I reformatted the SD card first.
(As a reminder to myself: Balena Etcher could not flash/validate the ext4 image and RPi Imager could not flash/validate the squashfs image.)
The additional tests were as follows:
With the /dev/mmcblk0p3 f2fs partition still in place I recreated the initial state of 21.02.0 squashfs (for details please see yesterdays post steps 2-3) but instead of importing my settings backup I left the default settings in place. When I upgraded to 21.02.1 I unselected the "Keep settings" checkbox.
=> Same result as yesterday, user-installed packages and old kernel and libwolfssl versions were deleted/updated. My config files from the settings backup did not influence the outcome.
I recreated the initial state of 21.02.0 squashfs and the deleted the /dev/mmcblk0p3 f2fs partition as per your suggestion. Again unselected the "Keep settings" checkbox and upgraded to 21.02.1.
=> Same result, my /dev/mmcblk0p3 f2fs partition did not influence the outcome.
As per Bobcat's suggestion I flashed the ext4 factory image of 21.02.0 stable, set up the initial state and upgraded using the 21.02.1 ext4 sysupgrade image, unselecting the "Keep settings" checkbox.
=> User-installed packages were deleted and the correct versions of kernel and libwolfssl were present.
I flashed the 21.02.0 RC4 squashfs image, set up the initial state and then upgraded using the 21.02.0 stable squashfs sysupgrade image, unselecting the "Keep settings" checkbox.
=> User-installed packages were deleted and the correct versions of kernel and libwolfssl were present.
=> Only the sysupgrade process from squashfs 21.02.0 stable to 21.02.1 is affected, both squashfs 21.02.0 RC4 to 21.02.0 stable and ext4 21.02.0 stable to 21.02.1 work as expected.
=> As a side note the "Skip from backup" and "Include in backup a list..." checkboxes were only present when upgrading the squashfs installation from 21.02.0 stable to 21.02.1. They were not present when upgrading the squashfs installation from 21.02.0 RC4 to 21.02.0 stable, nor ext4 21.02.0 stable to 21.02.1. So maybe the unexpected behavior has something to do with a change that was made here?
to be clear... what I (meant to) suggest/ed is that you;
zero your sdcard
start with a clean official image ( imagebuilder squashfs would also work for this test... if you are able to discern built in packages vs post-installed ones)
do not restore... do not mess around with partitions...
change a thing or two (install a package or two) then upgrade...
this would be a reliable test...
(your test conditions had too many variables... and while you are likely seeing some unusual behavior... you need to go back to the very basics... test there... and add one thing at a time)
As one of my additional tests (please see above post) I tried the ext4 image as you suggested.
The unexpected behavior was indeed not present (it was also not present with the upgrade from squashfs RC4 to 21.02.0 stable).
I initially chose the squashfs image as this wiki page said:
ext4-sdcard.img.gz
not optimized for flash memory (journaling speed up flash wear) [emphasis mine]
sdcard can be easyly mounted externally for modification
desktop linux standard
squashfs-sdcard.img.gz
newer images include a hidden f2fs filesystem, which is optimized for flash memory [emphasis mine]
compressed
needs special mount procedure, to externally modify it
all changes are done in an overlay partition
Thanks to now having seen the ext4 image I found out it's also missing the "Perform reset" function and has approximately 28(?)MB less space for user-installed packages. Though if I understand it correctly the latter can be remedied by expanding the partition as you suggested.
So I think I still prefer squashfs due to the reset function.
Oh I see, I will redo test 2 and report back later.
Since I reformatted the SD card before I did tests 3 and 4 respectively EDIT: and did not change anything regarding the partitions this could indeed be a difference I failed to account for.
As for restoring the settings backup and installing all my usual packages to obtain a controlled initial state:
I think my tests showed that neither did influence the outcome.
In testing it is simply unrealistic to test every single possible combination of variables.
All the images I used are clean official ones by the way.
After the reboot log into Luci and under System>Software check the versions of the installed kernel and libwolfssl and whether or not kmod-mii is present or not.
=> Expected result: kernel 5.4.154-1, libwolfssl 4.8.1-stable-4, kmod-mii is not present.
=> Actual result: kernel 5.4.143-1, libwolfssl 4.7.0-stable-2, kmod-mii is present.
Workaround: After the update go to System>Backup/Flash Firmware and klick the "Perform reset" button.
EDIT: The sysupgrade process works as expected when upgrading from squashfs 21.02.0 RC4 to squashfs 21.02.0 stable and from ext4 21.02.0 stable to ext4 21.02.1.
EDIT2: Yes, the SD card was properly zeroed before flashing the 21.02.0 factory image.
EDIT3: added link to kmod-mii.
rootfs_data/overlay filesystem is retained leading to the above symptoms...;
note: kernel + mii reported info is purely metadata and does not reflect real packages on /rom / system
config files/settings are retained
additionally;
same behavior is present upgrading from 21.02.1>21.02.1 (unresolved in 21.02.1)
rootfs_data/overlay is also retained when keep settings is selected (in otherwords everyone on 21.02.1(upgraded-to) + squashfs + sdcard/rpi?1 has corrupted/stale opkg metadata)
1
thinking back over the changes on this target... you'll probably find this is a 'c' level issue i.e. block_mount
in which case... it would probably effect more than just the rpi4-squashfs
(if as you state rc3>rc4 or whatever was ok)
(but it seems not many people select 'do not keep settings' + use squashfs)
edit: bug is resolved in master branch
hmmm... they all seem to contain the same fs-tools version so maybe not 'c' after all...
late to the party but it's seems to confirm what i could no understand in my other post here.. this was with squashfs trying to no retain settings.. maybe this is useful otherwise sorry for the noise..
I encountered exactly the same under exactly the described circumstances, but I too didn't understand what was actually happening at the time. I backup up settings, flashed clean with 21.02.1 and restored the backup and all was well, but overlayfs was definitely in a wonky state after sysupgrade.
Thank you for validating my findings and taking a closer look than I could.
I don't know if it helps but I noticed that the bug only occurs after user-installing a package.
So if you sysupgrade a completely untouched (or only setting a password and changing nothing else) 21.02.0>21.02.1 Luci shows the expected versions of the kernel and libwolfssl (though I did not test installing any 5.4.154 packages at that point).
Only after I had installed kmod-mii before the sysupgrade did the bug occur at which point I stopped testing.
Is it only an issue with the metadata? After all the first symptoms I noticed were the eth1 device (usb lan adapter) being present but not working and not being able to install the 5.4.154 packages after the 21.02.1 upgrade.
Is "Perform reset" and setting up from there ok or is it not a valid workaround?
should both be ok based on the above theory...
(i'd only excluded restoring backups from the tests as it would have complicated things before we knew more about what was happening)
I guess well find out exactly when developers take a look;
afaik;
eth1 would not have worked due to no drivers really being present (or thereabouts) (expected)
opkg would not have worked due to the metadata telling opkg that different versions of stuff are installed on the system when they are not (expected)
you can further validate these things prior to reset with;
