In the Preparing for Upgrade of the wiki it recommends not keeping settings once every 12-16 months. I also see some recommendations that old configuration files not be restored after loading a new OpenWRT version.
A 12-16 month reconfiguration schedule seems pretty aggressive for most users. Is this a schedule most of you adhere to?
Since the configuration is file based, is the best practice recommendation to go through each configuration file line by line to verify compatibility with a newer firmware version and then reload it, or to just start from scratch with LuCi and go from there?
OpenWrt constantly evolves. From kernel changes and changes to accommodate constantly changing configurations and applications, to hardware capabilities that weren’t supported . . . It is, in many releases NOT recommended to carry over previous release configs.
That is the main reasoning on the caveat you ask about.
While most manufacturers typically abandon older platforms, and thus firmware updates, OpenWrt excels in keeping your hardware secure and properly functioning long after the manufacturers intended it to be viable.
The rule of thumb should be to make a backup of your current configuration before any upgrade and if in doubt, use it to restore your configuration from that backup. Typically you will find many can be used as is, and those that actually do need new configurations, the backup can to used to accommodate those easily.
For minor updates (e.g. 23.05.x to a different 23.05.x), I always preserve settings, but for major updates (e.g. 22.03.x to 23.05.x), preserving settings can sometimes cause problems or cause you to miss out on the benefit of new defaults for those settings you do not customize. Usually, there's a new major OpenWrt version about once a year.
First, backup your existing configuration.
On the command line, you can use the command uci show to get a list of all your OpenWrt settings, and use opkg list-installed to get a list of all installed packages. Copy and paste the output of those commands (separately) to two text files on your computer.
Temporarily reset the router to defaults and repeat step 2, using different text files on your computer.
Use the diff command to compare your customized settings lists to the default settings list, and then do the same with the package lists. Look through the comparisons and see what's notable. Now you have a quick reference you can use to migrate your customizations to a new major OpenWrt version or a new device.
The package lists will contain version numbers, which can add noise to the diff comparison. You only want the package names. An easy way to remove the version numbers is to paste the list into a LibreOffice spreadsheet (Edit → Paste Special → Paste Unformatted Text) using tabs as the delimiter, select the column with the package names, copy it, and replace the existing contents of your package list file. Do that for both package list files. Then do the diff comparison.
Many users will be overwhelmed with that task, and even once you have the delta (e.g. in diff -Nrup a/ b/ style), you still need to interpret it - and adapt the settings accordingly. Especially this transfer effort can get quite tricky, even more so if it touches configurations that are not part of the default settings - meaning you have no known-good (new) config to compare against.
The gist is, there is no guarantee for configurations to remain the same over time - they should be between maintenance releases of the same stable branch (and here that is a pretty strong 'should', bordering 'must'), they can't always be between different major versions. This is not an x-months question, but an oldstable --> newstable situation.
For obvious reasons configurations don't change nilly-willy, and where they are necessary, it's at least attempted to lessen their impact (e.g. via migration scripts, trying to update the settings automatically - in some cases this is possible, not in all - and be aware that these migrations scripts only take action during the sysupgrade, not if you then later important an old configuration tarball (that just replaces the current device settings verbatim, without checking anything). It is attempted to keep configuration syntax stable, but that only goes so far - just to raise some issues:
the big elephant in the room, swconfig <--> DSA, here this is a major semantic and syntactical change, changes need to be redone, there are no migration helpers in place. this is so big, that even the compat version is raised to hint at the magnitude.
this wasn't an easy decision, the initial DSA migration was delayed over and over again (around two years) in the hope for 'some' magical migration script that would ease this, but there is none, there can't be any - the semantics are too wildly different (before even thinking about identifying the corner cases, namely how to deal with situations that are neither swconfig nor DSA (plain network interfaces, switchdev (ipq807x/ nss-dp), virtual interfaces, etc. pp.).
fw3 <--> fw4, really a lot of effort was spent to retain the configuration as much as possible. For the general case there are no changes necessary, the exact same config can be retained and gets interpreted into equivalent results. Only very specific corner-cases, as in the previous method to include externally defines iptables calls from a file does not work anymore (must be changed)
we have dæmons that are being jailed from one release to the next, to improve their security exposure (case in point, dnsmaq a while back), while functionally an invisible change, it does still affect some config options (where to place pid files, which files are declared to be available in the jail, etc.), here we had a migration script in place - it works, …unless you then later import an old config tarball (it caught out a lot of users, even though all precautions were set, automatically)
we have things like the ifname change, both syntaxes are evaluated for a migration period, config are automatically adapted, but still, this is something that will catch some users in x-years from now
we get new settings that might need to be configured, etc.
with firmware sizes growing, you might have to adapt bootloader settings and partitioning between releases - strictly speaking this is orthogonal to (outside of) OpenWrt itself, but it still needs the required preparation or you're in deep problems.
device paths or interface names can change over time, these are not 100% stable
device naming (probing order) can change between major kernel updates
GPIO definitions (and subsequently LEDs) have changed over time
MAC extraction has changed over the years, sometimes to fix old omissions, sometimes just behind the scenes, but affecting cases where there are explicit MAC overrides configured
…
…
…
If you know what you're doing, what needs changing and why, you can do that - but if you aren't following development closely (and maybe didn't even read the release notes talking about them), you won't be in a position to answer these topics.
This is no clear-cut yes-or-no question, nor does it have easy answers.