18.06.2 - Upgrading packages

Quick question, installing 18.06.02 worked perfect.
When looking what software packages had upgrades I got the following list,
Are they safe to upgrade with 18.06.02?

opkg list-upgradable
luci-lib-ip - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-theme-bootstrap - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
netifd - 2018-11-19-4b83102d-2 - 2019-01-31-a2aba5c7-2.1
luci-app-firewall - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-proto-ppp - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-mod-admin-full - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-base - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-proto-ipv6 - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-lib-nixio - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci-lib-jsonc - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1
luci - git-19.020.41695-6f6641d-1 - git-19.034.35162-a0ae6c9-1

1 Like

“Never” upgrade packages as a matter of practice.

Search is your friend

1 Like

@jeff can you explain why ?

I believe the packages that come with a system are generally stored compressed on a squashfs filesystem (on the ROM), so replacing the package doesn't actually replace them -- but merely puts the new package uncompressed on the overlay filesystem (the older package is still there).

So you have to be careful of a number of things here -- including running out of space on your overlay device. While this may not be a problem if you are using an additional USB drive as an external root you can still end up in scenarios where a non-captured dependency on some base package ends up clobbering things.

So ideally you just reinstall a new image with enough to get the system to boot - and then add all your extra packages on top.

2 Likes

Further, at least at this time, ABI versioning is not done (yet), so you can end up with incompatible packages. Probably five threads on user problems with that in just the last week.

5 Likes

Ok, I can understand this, it makes sense.

But now I wonder why are those updates being made public? Why is someone making the effort of compiling and publishing some packages that nobody is supposed to use?

Thanks!

If you're installing your packages after a flash of a ROM, or you're using them with the image builder, that almost always gets you a self-consistent, current set. (Caveat there is if you're doing it just as they are being updated.)

opkg list-upgradable is the :devil:

2 Likes

Would it be an idea to add a flag in LUCI, only showing updates of user installed packages ( so masking of all packages that are part of the flash image package.

2 Likes

Ok, so the image builder scenario makes sense, thanks for the clarification.

On the other hand, if I am not supposed to upgrade a package after flashing, why are packages already included in the stable build being updated?

If those updates are important for the people using the image builder, why aren't they important for those who flash a stable release?

I think the problem is that publishing a new version of a package gives the impression that there is something wrong with the previous version, but at the same time people is getting the message that they should not upgrade their packages.

If you use the image builder, you'll be assembling your image from the latest versions.

Yes, there's a fine balance. Some updates are of security importance. The vast majority of them are just bug fixes or minor enhancements. Without reading the release notes of the upstream project, it is typically difficult to know the difference between the two. Things that are really critical (like the 802.11 vulnerabilities) have had the OpenWrt team providing a release, as I recall.

1 Like

There are many aspects:

  • earlier OpenWrt releases (before the LEDE architecture change) were monolithic "compile everything at once", and no packages of a release got updated. So, there was not problem with ABI as release packages remained contstant.

  • With the current build system, packages do get updated during the lifetime of a release. But the old ABI dependency checking in opkg is not 100% certain to indicate that all dependent packages do get re-compiled (& re-installed).

  • the package set at any given time is supposed to be coherent. If you compile or imagebuild, or install a release, you are supposed to get a coherent set to the packages installed at that time.

  • But if you opkg upgrade some packages in 18.06.x after installation, the opkg logic does not always recognize that a depending "upper" package needs to be updated, too. E.g. mbedtls get upgraded, but the version of libustream-mbedtls remains the same and does not get automatically updated, although it should be.

  • master has now ABI more directly in the package binary name, so the dependent packages should more clearly depend on an exact version of a library.

But in general, the whole idea of "let's upgrade all installed packages " is a new phenomenom, maybe due to larger flash spaces. Earlier there was no such option, because free flash space was minimal in a size-constrained device. OpenWrt opkg system is not really targeting that package upgrade approach.

9 Likes

Thanks @jeff and @hnyman for the info.

  • Why not bump libustream-mbedtls release version in the repo?
  • So reinstalling libustream-mbedtls should fix the issue?

Until detection of ABI changes is automated (first steps of capturing ABI information underway on master), knowing just what dependencies need to be updated would be a purely manual task. There's the possibility of ripple-through more than just one package deep, as well. The "sledgehammer" approach of "everything that could possibly depend on this, and everything that depends on those, and ..." works with an unconstrained file system, but between capacity and flash wear, isn't really appropriate for all-in-one routers.

  • libmbedtls -- 143 kB right there

https://openwrt.org/packages/table/start?dataflt[Dependencies_pkg-dependencies*~]=libmbedtls

  • 21 packages -- 2,102 kB (by hand, without enough coffee)

With users still clinging to 4 MB flash devices, and even 8 MB ones, another 2 MB for someone impacted by all of them can easily put them over the top. Of those, the apparently popular transmission packages are around 1 MB right there.

2 Likes

Even with those new ABI checks in place, you need to be aware that this won't ever become as thorough as for the large desktop distributions, which spend a lot of time and effort on doing this properly (including symbol versioning). That is a limitation of the available system ressources, opkg's abilities in general and most of all developer ressources - yes, these new changes will help, but they won't make it 'perfect' either.

2 Likes

So the conclusion is...

  • Do not upgrade packages, use supported releases and snapshots, or compile from the source and be happy.
  • Otherwise don't complain on the forum and be ready to deal with consequences yourself, such as manual dependency resolution and additional free space consumption, potentially leading to ABI-conflicts and soft-bricking.
5 Likes

The problem appears to be that updates to packages are put in a repository which is used by the released software which includes the tools for updating packages just like on almost every similar platform, creating a POLA violation.

This is made worse by the fact that most packages can be safely updated. I've been updating packages for years and only recently found one to be problematic (netifd).

Even worserer is that the idea of not updating packages only occurred to me when I came here and -after jumping all the hoops to find the forums, sign up, navigate to the right place, browse, find the relevant thread, find the relevant posts, try to check the authority of the poster, etc. - read a thread or two on the subject.

The best solution is not obvious, but as currently released and documented, it should be expected that all good admins will find out how to update and do so (and probably fix any problems they encounter while doing so). Some relevant discussion in Okpg upgrade safeguards?

2 Likes

This note is a little late in the game if you've already upgraded, but...

OpenWrtScripts github repo has an opkgscript.sh script that will:

  • write out your current set of installed packages before flashing (making a list of packages that aren't part of the original firmware image), then
  • (after flashing the new image) read that package list, and reinstall them.
4 Likes

I'm trying (really hard) to make sense of what is written in this thread and might need some help from those of you more knowledgeable.
I have a D-Link DIR-860L-B1 that is currently running 18.06.1 and was about to upgrade it to 18.06.2, but I do have a few extra packages installed:
(during the initial installation I added)

opkg install kmod-usb-serial
opkg install kmod-usb-serial-option
opkg install usb-modeswitch
opkg install comgt
opkg install luci-proto-3g
opkg install usbutils
opkg install kmod-usb-storage
opkg install kmod-usb-storage-extras
opkg install kmod-fs-vfat
opkg install kmod-nls-utf8
opkg install kmod-nls-cp437
opkg install kmod-nls-iso8859-1
opkg install kmod-fs-ext4
opkg install kmod-fs-msdos
opkg install kmod-fs-ntfs
opkg install luci-lib-px5g
opkg install px5g-standalone
opkg install libustream-openssl
opkg install kmod-ledtrig-netdev

By looking at the short release note:
https://openwrt.org/releases/18.06/notes-18.06.2
I learned that the kernel was upgraded:

both 4.9.x and 4.14.x being LTS branches I do not understand the discussions about API/ABI breakage.

Then, about the installed packages upgrades, process that I might need to run myself, by reading the official OpenWRT docs:
https://openwrt.org/docs/guide-user/additional-software/opkg
https://openwrt.org/docs/guide-user/installation/sysupgrade.cli

I learned that opkg is able to automatically resolve dependencies, thus, I'm not able to follow some of the posts here in this thread.

Bottom line - how should I proceed now?
Upgrade the flash by using LuCI and flash:
https://downloads.openwrt.org/releases/18.06.2/targets/ramips/mt7621/openwrt-18.06.2-ramips-mt7621-dir-860l-b1-squashfs-sysupgrade.bin
The configuration will be preserved but I'm not sure about the extra packages I installed.
If those extra packages will remain (will they still run?) and opkg will inform me that some are upgrade-able, how should I proceed? Upgrade them with opkg?

Or, should I backup my actual config, flash the new 18.06.2 by removing everything, then install all the packages again, as new, and put the configuration back? I hope this won't be the case, it'll be overkill and I'll need to bring down the network, get the router, prepare it and install it back (it's not in my possession ATM)

Any clarifications/hints appreciated! Thanks!

Assuming that the packages are "well behaved" (and your list appears to be), the configuration (if any) will be preserved by sysupgrade. You can check what is preserved with sysupgrade -l and add to it, if needed, by editing /etc/sysupgrade.conf

During a ROM upgrade, any packages you previously installed after flashing are wiped out when the overlay is reset. While the saved config is restored, the "binary" packages are not.

After you upgrade, you can install new versions of the packages you added using okpg, if they are not already present in the ROM you flashed. If there's already a version in the ROM, it is not recommended to upgrade it.

As with any upgrade, preserving your config in a safe place (off the router) is highly recommended.

2 Likes