I'm building my own OpenWrt image from source. I do this update 2-3 times a year. When I update, most of the time I decide to include in the ROM some of the packages that were previously installed by opkg in extroot. After the update, there is a bit of a mess - newer package versions from ROM are overwritten by overlay's older versions left there from before the update.
It's hard for me to keep track of the changes I make between ROM versions.
Is there any advice about managing opkg's package lists? Find out what is installed and where? Find overlay conflicts/duplicates? How to remove packages from overlay while keeping newer ROM versions?
My solution now is to manually compare /rom and /overlay/upper and remove older files from overlay. I hope there is a better way of doing this.
if you build from source, don't bother (too much) about opkg, just build everything you need into the image (=y).
trim down your diffconfig to the essential config decisions and annotate them (see hnyman's community builds for an example workflow), make defconfig oldconfig will expand those for you.
keep the buildconfig/ seedconfig and all your changes (quilt patch series) under (at least local-) revision control (e.g. git).
runtime installed package contents (apart from their explicitly marked conffiles) do not survive a sysupgrade, if they do, you've done something very wrong (probably via sysupgrade.conf).
It is a LuCI app that retrieves a list of currently-installed packages, requests a new build from the ImageBuilder with those packages, then automatically installs that build.
Attended sysupgrade and diffconfig won't help me. I don't have a problem with building images. My problem is removing old packages from /overlay after sysupgrade.
I know that overlay should be emptied when sysupgrading. I'm not OK with that. I have many files in /etc, /root and /var that must remain there (on USB stick) after the sysupgrade. So I disconnect the USB stick when performing sysupgrade, leaving the UBI overlay instead.
There is one other reason I disconnect USB overlay before sysupgrade:
Last time I tried sysupgrade with keep config option, it tired to copy the USB overlay into UBI overlay! This is NOT what I want and it won't work. There is very little space left in UBI rw partition. Complete /etc for all USB packages won't fit.
I keep a minimal working config on UBI, with only basic router functionality, in case USB mount fails.
You can't retain an extroot over a sysupgrade, fighting against inherent design principles of opkg/ sysupgrade and OpenWrt (which are there for hard technical reasons) won't provide you with much joy, just more windmills in front of you.