Remote Upgrading Devices With nonstandard configs

I have a couple different devices mostly spread around my families residences where the configs are non-standard and I'm not on-site to supervise upgrades. These configs have situations where there are nonstandard VLAN configs so that if you upgrade without keeping settings, the devices will become non-responsive. At the same time, devices are not generally supposed to keep settings across major upgrades. One is an AP running an 18 series, the other is a router running snapshot on an RPi4 in a router-on-a-stick VLAN config.

What's the best way to get these upgraded without borking my remote families setup?

For the RPi4 I could just burn the new config to an SD card and ship it to my sister. That's appealing! But for the AP not so much. I do have wireguard connection to the remote location where the AP is, through a separate router device which is not running OpenWrt.

Imagebuilder or building from source will both allow you to generate images containing all necessary packages (so you don't need them to do any packet installations post flashing) - as long as you stay within a stable release (or small steps between individual snapshots), the upgrade should be smooth and without pitfalls (you may have to add additional configuration files to /etc/sysupgrade.conf to retain over upgrades).

Major configuration changes, like they're happening for DSA or the various semantical configuration changes for /etc/config/network right now for 21.02.x, can't be covered that way and will need a more hands-on approach (remote access via VPN, specially crafted -pre-configured- firmware images or swapping out the devices via mail). Technically, rule-book based auto-configuration (à la ansible or using uci invocations) is also always an option, but usually more a topic for enterprise installations with many identical devices in the field - rather than the typical historically grown assortment of various devices in a family & friends settings.

The question is always to juggle between the extremes, the 'perfect' setup vs. something that's just 'good enough' but which can be maintained by the target users themselves (at least in emergencies) and striving for perfection can quickly become a curse in terms of ongoing maintenance.

Aware that I can build an image with all the needed packages. But can I build an image with the needed configs? And I'm not sure if one of the devices (a zyxel armor) will change between swconfig and DSA in the latest upgrades.

You can, whatever you place under files/ will end up in the generated image verbatim (e.g. files/etc/config/network and so on). You can also check the resulting images (at least in regards to the presence and contents of your injected configuration files) with binwalk, tar and unsquashfs.

ipq806x (as in the ZyXEL Armor Z2/ nbg6817) is still using swconfig in both openwrt-21.02 and master, there is a pending pull request to migrate ipq806x over to DSA, but that hasn't been merged into master yet (hopefully soon) and in any case won't enter openwrt-21.02 at this point (which will stay on swconfig for the remainder of its release cycle).

1 Like

Is there a simple way to have opkg output the existing packages in a way I can just import into the image builder?

hmm, personally I build from source, rather than using imagebuilder myself…

but:

# sysupgrade -h
[…]
        -k           include in backup a list of current installed packages at
                     /etc/backup/installed_packages.txt
        -T | --test
                     Verify image and config .tar.gz but do not actually flash.
[…]
        -l | --list-backup
                     list the files that would be backed up when calling
                     sysupgrade -b. Does not create a backup file.

could be relevant, you can also check the result of find /overlay/upper | sort for hints.

opkg status

seems to get me a list of all packages and I can see the various
Status: install user installed

so
opkg status | tr '\n' ',' | sed -e "s/,,/\n/g"

gets me a csv file I can grep for user installed packages :wink:

Though... it's kinda funky, lists things like "iw" and "ubus" as user installed... :frowning:

Upgraded packages are probably shown as user-installed.

1 Like

On the Zyxel router am I correct in thinking that sysupgrade will flash to the other partition from what booted, and if needed I can do the 3x on/off again and revert to the original partition?

If the other partition is still on stock firmware can I still flash a sysupgrade image?

Yes, it always sysupgrades to the other, currently-not-in-use, partition set.

I'm not so sure about this aspect, at least I haven't found a magic incantation to toggle the boot order on a non-responsive device so far, but (if you can boot the device) luci-advanced-reboot or nbg6817-dualboot can do the job; push-button tftp recovery works very reliably though (this procedure always overwrites the first partition set).

Yes, a potentially existing OEM firmware is overwritten the same way an OpenWrt image would be, in round-robin fashion (nbg6817-dualboot has some tricks up its sleeves if that is not desired). I have replicated the OEM upgrade behaviour in sysupgrade as well as I could, it should behave exactly the same way as the OEM firmware in every aspect (related to the upgrading) - in order to be compatible with each other (nbg6817-dualboot also works on the OEM firmware).

1 Like

Hmm... tftp recovery is a good trick I think, because I do have wireguard access to the remote location on a device that's separate from the AP. As a worst case I guess that could be used. It's so dicey though, remote upgrades when people rely on the network...