Help Understanding OpenWrt Update Process /"Keep Settings" and Package Update Process

Installing and Using OpenWrt
1

Was reading through the docs, and am confused on the proper update processes.
So let's start with my question on Openwrt version upgrades.

Uncheck/clear the “Keep settings” checkbox especially for major version upgrades, so that new defaults will get applied. Keeping settings may be possible for minor upgrades, but there is always a risk of incompatible settings. (more info regarding the "Keep settings" checkbox and its use cases).

Is what is stated for keeping settings. Following the link it provides, it states this:

Only check the “Keep settings” checkbox on minor OpenWrt → OpenWrt bug fix upgrades that are known to not change the config structure.

Does this mean, if I am going from 19.07.5 to 19.07.6, I can keep settings checked no issues?
Now what about 19.06.9 to 19.07.1 for example? Is it safe to keep settings checked?
And now I assume going from 19.00.0 to 20.00.0 would be considered a major upgrade, and keep settings should be unchecked?
(ignore my poor use of version numbers, you should be able to understand what I am trying to ask.)

It is really frustrating to not be able to keep settings, is there no reason openwrt doesn't have a way to handle this automatically? I see the suggestion of a script, however this puts a lot more work on end users, and also if you setup via luci it makes it more annoying because now you have to go and convert (see) everything to cli to make the script. This also implies the user knows how to script.

I really don't want to have to deal with this everytime I have to upgrade, this is really tedious. For stuff like security upgrades, it makes it even more frustrating because it kinda forces me into needing do reconfigure everything or my system is more vulnerable. Has there been any considerations in streamlining/automating this process? Or at least making it so something like luci backup before updating, and importing after update will work flawlessly after version upgrades?
Manually reconfiguring everything is extremely tedious, and if you don't remember a setting it could mess your whole configuration up. It's just not user friendly at all. :frowning:
This is my biggest issue with OpenWRT personally, besides that I really love OpenWRT.

My second related question, is with updating/upgrading packages. In all my distros, the package manager can generally be ran and deal with updating everything at once no issues. For example on Arch, a simple sudo pacman -Syu.
However, the docs for OpenWRT suggest that this should not be done.
Now this confuses me, for one, how am I supposed to update each package manually, if I don't know which I should be updating first? Secondly, this means if using cli, I have to generate a list of all packages, and manually enter each one to check for updates? Extremely tedious. If using LUCI, it's a lot easier, however again, that first issue of knowing which to update first.
I fail to understand why I cannot just "update all".

2

OpenWrt is designed for limited-flash-space routers with resource constraints. The package manager has only limited functionality.

Don't think it is like a desktop distro.

How to do everything in router with 8 MB flash, of which 6 MB is already consumed by the existing firmware...

1 Like
3

This is understandable.
Could a package that gives the package manager this functionality, not be made possible?

Also still doesn't really help in answering the proper update process. But I understand if you aren't sure, and just replying to that part. Thank you.

1 Like
4

The designed practice is to use sysupgrade every now and then to upgrade whole firmware, instead of continuously upgrading individual packages.

Again, starting from limited flash space and the need to have reset ability, the design is to have read-only firmware (in /rom) and all changes in overlay/

That means that if you upgrade a package, you will have both the old package version in rom/ and then new package in overlay/ although thanks to overlayfs you only see the new package in /

That means that for most routers you would quickly run out of space if you try to upgrade everything individually.

(You can upgrade without space issues onlyntuose packages that are installed by you and are already on overlay/ )

2 Likes
5

No, it's not 'merely' down to the package manger, but also to the packaging (and its approach) itself - before even thinking about device specific constraints.

[Disclaimer: the following is a very simplified view on some aspects of how normal desktop distributions work, reality is more nuanced and deals with a more complex ecosystem]

Q: How does a general purpose distribution handle in-place package upgrade?

A: Mostly by allowing various forms of co-installability.
What this means is proper ABI versioning of libraries, symbol versioning and allowing to keep multiple ABIs of a library co-installed at the same time, to be used by different programs. So /usr/bin/foo can still use libfoo1, while /usr/bin/bar already depends on libfoo2 (and technically, nothing prevents /usr/bin/baz to require libfoo42 at the same time). Although distributions try to get rid of older/ orphaned (no longer updated) library ABIs as soon as possible, in-place upgrades depend on this being possible at least temporarily (while the upgrade is commencing, so tools (including the package manager itself) using either ABI can continue to work, despite being in the middle of the upgrade). This has consequences both for packaging (you need to split out libraries, rather than shipping the tools and libraries (at least those used by other packages as well) in a single package, properly into individual packages), just as well as for the system requirements on the system resources of your target system (vulgo, "router"). More packages (more packages == more concurrent ABI versions of the same package) mean significantly more complex dependency resolution by the package manager (more, and more complex-, package meta data ending up in the package repository index), which as a consequence means higher RAM requirements at runtime on your target (opkg can't work on devices with 32 MB RAM anymore) - but at the same time you roughly need at least 3 times the permanent storage (libfoo1, libfoo2 - and the downloaded libfoo2-x.y.ipk during the upgrade process, before- and while it is being unpacked). In terms of very basic libraries, the libc implementation for example, the situation would get even more messy, as old ABI versions (possibly including intermediary ABIs) would have to remain on storage for longer spans of time, including a whole zoo of packages depending on either of them (OpenWrt just switched from uclibc to the totally binary-incompatible musl just ~5 years ago, Debian and other distros had to move in-place from libc5 to libc6 about 15 years ago - and even with a proper package manager in place (dpkg && apt) and just upgrading from the old ABI to the newer one of the same libc (glibc) this was a disruptive process for developers and users). Keep in mind, OpenWrt just had to abandon support for devices with 4 MB flash, devices with 8-16 MB are still very common among typical OpenWrt targets, even devices with more flash (NAND/ eMMC) might not actually reserve that much more to be used for kernel && rootfs (e.g. the rather high-end ZyXEL Armor Z2/ NBG6817 comes with a 4 GiB eMMC, but the maximum firmware size is 4+64 MB, due to the vendors partitioning scheme).

Another large topic would be finding a way to upgrade the kernel…
On the face of it, on a desktop distro, this is also accomplished by being able to keep multiple kernels co-installed (so again, 3-times the space requirements, actually more, considering the runtime re-generated initramfs and having to retain 1-2 versions of that), but it also depends on the kernel being part of the rootfs (rather than having to remain on a separate partition). This suddenly requires a bootloader which can find the kernel on the rootfs, mostly grub2 these days, but your off-the-mill router usually can't do that - they come with a very minimal OEM fork of (ancient-) u-boot variants instead, which generally has no concept of filesystems, but reads the raw kernel from fixed flash offsets. Sure, this can be solved by adding an intermediate bootloader (maybe upstream u-boot, maybe grub2), maybe something custom, at the expense of even more storage.

At some point you have to decide what your target is, something like a desktop (and SBCs like RPi, Allwinner/ sunxi, Rockchip, etc. are rather desktop-like in this respect) or traditional embedded platforms which make up the bulk of devices supported by OpenWrt (and these devices aren't even deeply embedded platforms), as this has deep implications about the very basics of your operating systems, its packaging and package manger. If you are longing for the feature set of a full general purpose distribution, you also need to provide the hardware resources required for that - for Debian this would be 2 GB main storage and 128 MB RAM (actually, while formally specified as such, you won't really survive a default boot and upgrade with less than 384 MB RAM - yes, it's possible to drop RAM usage via quite invasive configuration changes, but not really down to 128 MB). Neither opkg, nor OpenWrt's packaging (how packages are split up) are capable of full in-place upgrades, so you would have to switch to a better (but larger) package manager - and even with a focus on size constraints, you'd still be in the 800 MB - 1 GB area as minimum storage requirements, with corresponding RAM requirements >=128 MB) to deal with the dependency resolution as well. With a more complex and fine-grained packaging, you'd also require a lot more developer time for handling this and keeping it alive, requiring easily 3-8 times the manpower available now.

2 Likes
6

Ok, understandable.
Now I am still just confused on the proper process to go about updating, both the system and packages.

With the system, the actual way the docs describe it I posted in my original post.
Again, what versions can I safely be selecting "keep settings"? 19.06.9 to 19.07.0 to 19.07.1, etc? And once it reaches 20.00.0 I should not keep settings and reconfigure from scratch?
Or should I be re-configuring after every center digit upgrade-19.06.9 to 19.07.0 for example?

And as for the times I need to do reconfiguration, are there any easier/simpler ways through LuCI besides writing the shell script, or manually just writing down everything, going back in and re-configuring it all and hoping I got everything?

As for the packages, what exactly is the proper way to update them. hnyman seems to state

You can upgrade without space issues onlyntuose packages that are installed by you and are already on overlay/

Which if I understand this properly, suggesting that when doing upgrades to OpenWRT, it upgrades all the preinstalled packages? And you only have to worry about updating the packages you specifically install?
So I should be fine to just hop in LuCI and click update on all the packages I've manually installed?

Thank you

7

In general, as a rule of thumb:

For 18.06.0 --> 18.06.8 you can keep the settings (but user-installed packages (binaries) will be lost nevertheless).
For 18.06.8 --> 19.07.5 you should start from scratch, using your old configuration as reference.
For 19.07.0 --> 19.07.5 you can keep the settings (but user-installed packages (binaries) will be lost nevertheless).
For 19.07.5 --> 21.xy.0 you should start from scratch, using your old configuration as reference.

For keeping configuration over the maintenance it shouldn't matter if you go directly from .0 to .x or follow each minor Upgrade over time, the configuration should remain largely compatible.

Disclaimer: the above is a rule of thumb, not gospel. Everyone tries hard to avoid breaking (configuration-) changes, but sometimes they are unavoidable (e.g. swconfig --> dsa on selected targets coming up with 21.xy.0). In order to minimize the breakage, it's always strongly preferred to contain the introduction of these breaking changes for major upgrades and not minor service releases.

The wish for some magic automated migration for swconfig to dsa configurations has delayed the introduction for several years, as this is a rather major beaking change. The problem is that both are conceptionually very different, while you can achieve the same functionality using either method, there is no direct 1:1 mapping when it comes to how to set it up.

2 Likes
8

in addition to the above...

lets assume you could... would you really want to?

routers are in a large part a security device...

just because you 'update' what is said to be updatable...

there is no guarantee remnants of any backdoor / trojan / rootkit aren't happily sitting alongside your freshly updated programattica... waiting for profile.d to run so they can re-alias/wrap themselves around your 'up to date' system...

9

there is no guarantee remnants of any backdoor / trojan / rootkit aren't happily sitting alongside your freshly updated programattica

Of course, but I fail to understand the logic here. Whether you update individually, or all at once what is the difference?
I can understand if you're suggesting doing the reset before updating versions because of this, however I fail to understand what updating packages individually vs all at once would make any difference to that issue?

10

Disclaimer: the above is a rule of thumb, not gospel. Everyone tries hard to avoid breaking (configuration-) changes, but sometimes they are unavoidable

Is there usually a note on releases that say whether or not you can keep settings safely or reset from the previous version released?

11

No. Too many different CPU targets etc. and too many old versions. Some changes happen only for some router families etc. E.g. when mvebu routers switched from old swconfig to DSA switch config.) This is an universe of approx. 50 CPU targets and 1000+ routers, so it is quite different than traditional x86_64 desktop distros where the only difference may be additional functionalities in new CPUs.

Some times a sysupgrade block is built into the system, e.g. if you try to sysupgrade a recent mvebu router to the current master, keeping settings is prevented. But that blocking logic is only built to the newest 19.07.6 (and maybe 19.07.5?). The older firmwares naturally know nothing about these new blocks as the mechanism was not yet in place when 19.07.0 was released in back in 2019.