Maintaining an OpenWrt Router

If you can manage to install openwrt initially then updating isn't any harder. Not sure what the issue is here.

If you install and config 5 or 8 packages with dependencies, it can be easy to forget what all you did manually months ago... so it often happens that you update, and then a couple days later you realize something you expected to work isn't working because you didn't reinstall it.

Like I need the wpad full package, and when I updated my 3 APs I only put it on one of them by accident, and so then the other two didn't even come up on the air except the guest network ... but I didn't notice for a couple days because everyone was connecting to the one AP that had it... that sort of thing.

When I was testing CeroWrt (2012-2014), I was flashing new images every week or so. Because I was testing firmware that was changing fast, I never kept settings. I created a script (probably like @dit_dah_dit's) so I could get to a baseline configuration reliably. It was wicked fast. But I knew exactly what I wanted to install each time. (If anyone is terminally curious, the script still lives at: https://github.com/richb-hanover/OpenWrtScripts#config-openwrtsh )

@dlakelan describes the kind of "maintenance" I now perform. I get the router set up (manually). At update time (usually with a new OpenWrt release), I flash the new version of the firmware, then I have to make sure all the packages get reinstalled. It would be quite convenient for OpenWrt to keep both settings and packages as part of installing a new firmware image. It would basically mean, "Get my router back to where it was, with everything updated."

And for @drbrains questions: Usually I do reinstall the new image, then use a script I found on the forum to replace/update packages. I keep settings and have never had a problem with OpenWrt (sample size of a half-dozen flashes, though). I believe I restored from backup one time, with good results (again, not a lot of track record).

I find it interesting, or should I say didn’t expect, that the experienced people here are using a clean image and added packages later on.

I was expecting a higher update rate (using intermediate “snapshots” obviously). I also expected a full custom image and using “keep settings”. Hence the question: did you run into problems where updated packages didn’t play nice with the old config files.

So that brings up a new question: given how easy it is to build your own custom image with all the packages included for your use case, why would you still install the packages after a clean image?

I guess I'm not surprised by this, but it's great that we tell each other how we operate. So here are my biases toward OpenWrt images:

  • I run a "semi-production" router: my wife and family expect "all the networking magic to work all the time." If I'm going to change anything, I need to have enough spare time both to install the new image/package and then to pay attention to how things are working over the next couple days to be sure I haven't broken anything.

  • Consequently, I generally only update when there is a new release image. (I continue to feel that a current, but un-upgraded OpenWrt release image is more reliable and secure than vendor firmware.)

  • Although I could learn to build my own image (I've been writing code professionally - 68K assembly, C, Python, Javascript - for 45 years), I have chosen to remain ignorant of the OpenWrt build system. It feels like a rabbit hole that could only suck up a tremendous amount of my time.

  • If everyone were to build their own image, then who will test the release builds?

  • If I have trouble, I feel comfortable dumping my story on the support forum because I can say, "I'm using 18.06.2 with X, Y, Z packages, and I see these symptoms..." The alternative ("I built my own image using a snapshot from a couple weeks ago...") is a much harder diagnostic effort.

That's why I push so hard for a good newcomer experience. Not only does it help them, but it can make my life easier, too.

3 Likes

I guess most of us are in the same boat on your first point: our family / friends expect it to work like magic all the time. I have however a little more faith that it will, so I don't plan on "next couple of days".

Only update on a new release. That means (at the moment): 1 or 2 times per year. How much effort in a new "maintenance system" should we invest for only a few times per year. (This is separate from new-install, easy-wizard type of stuff).

@richb-hanover, you should at least try installing and using the build system once. You spend a lot of time here on the forum (probably even more behind the scenes), so I think you should see its not the rabbit hole you think it is :wink:

As for asking help: you could choose to build the release version, and not a snapshot with X,Y, Z package.

The testing part, I actually don't know who is doing that, other than "all of us here". We test drive snapshots and release versions I think? The build-bot is just "some cloud computer" building images for "all" targets in my mind. A lot will not be tested at all I can imagine until some user has that actual device. Our devs may have a lot of devices between them, but for sure not every single device that we can build for.

So as far as "maintaining" goes: if we create a "generic" image with most-wanted features (and maybe some wizard), the "keep settings" should do a good job in allowing "novice users" to update their device without any problems.

First time installation is beyond the scope of this thread, and we will continue that in the other one.

Well it's a few times per year times how many people? I'm hoping millions... And if it makes updates easier then I'm hoping they happen faster and close vulnerabilities faster, so it's potentially a security issue

But thats the point: if all the packages are already in the image, and there were/are no issues with the config files, the keep settings is already doing a very good job.

So for Easy install...and update (maintaining), the packages should be included in the image. Much like it is with a OEM/vendor update.

Yeah, but they don't use every feature every day (can you say printers?), so I feel I need to be hyper-aware for a while, and be sure I'm not doing anything the day before their deadlines.

Only update on a new release. That means (at the moment): 1 or 2 times per year. How much effort in a new "maintenance system" should we invest for only a few times per year. (This is separate from new-install, easy-wizard type of stuff).

This is the right question. My criteria: I would be completely content if we could guarantee that "keep settings" and installing (updated) packages would restore my router to its previous operating state after flashing a new image.

@richb-hanover, you should at least try installing and using the build system once. You spend a lot of time here on the forum (probably even more behind the scenes), so I think you should see its not the rabbit hole you think it is :wink:

Nope. Not gonna do it :wink: (You don't understand how that could throw me into a full anal-compulsive, gotta-understand-everything loss of a week or two of productive effort on important projects. I realize that there are Docker images, VMs, etc. all crafted and ready to make an image. I still am choosing not to do it.)

As for asking help: you could choose to build the release version, and not a snapshot with X,Y, Z package.

The testing part, I actually don't know who is doing that, other than "all of us here". We test drive snapshots and release versions I think? The build-bot is just "some cloud computer" building images for "all" targets in my mind. A lot will not be tested at all I can imagine until some user has that actual device. Our devs may have a lot of devices between them, but for sure not every single device that we can build for.

I sort of see this as herd immunity. I just use released images - it's an efficient
use of my time, and makes sure that someone tests releases (for my bog-standard Archer C7 & WNDR3800's)

So as far as "maintaining" goes: if we create a "generic" image with most-wanted features (and maybe some wizard), the "keep settings" should do a good job in allowing "novice users" to update their device without any problems.

First time installation is beyond the scope of this thread, and we will continue that in the other one.
[/quote]

Exactly. I'm thinking of naming my goal the Essential Secure Router It does "all the router kinds of things" that people want (and we can debate which packages belong).

And for the Maintenance side of things, we need the "Keep settings" update process to be smooth as silk.

I think we're all agreeing with each other. (At least, I'm trying to :slight_smile: ) To summarize, it would be a good goal to have:

  • A regular release schedule for OpenWrt for "stable" builds that include the "right set of packages" (to be determined) for an Essential Secure Router
  • An "Upgrade me to the latest OpenWrt release" facility that preserved all the built-in and manually added packages with their settings.

Do I have this right? Thanks.

2 Likes

That sounds about right to me. You could add to the upgrade me facility some kind of low-resource-requirement notification of availability in LuCI.

Each one of my scripts installs one "service" (not "service" as in a process/daemon, more like one "idea"; installing one "idea" may involve spinning up multiple daemons/processes). This allows me to either run a script or skip it if my customer doesn't need that particular service. For example, I have one customer that needs to use VPN for remote access, but another one doesn't.

  • 0a. Install a clean image of OpenWrt.
  • 0b. SSH into the router, set a password. scp the scripts into /tmp
  1. Run an initial "up2date" script, which basically performs an opkg upgrade <pkg-name> on all packages, since some packages have newer versions even with a fresh install of OpenWrt. Also installs luci-ssl, which to be honest I don't understand why it's not included as a part of the base OpenWrt core packages.
  2. Install USB-related packages. My preferred router to deploy to customers are Netgear R8000s, which come with two USB ports.
  3. If I am able, I encourage my customers buy a small (i.e. one of those tiny chiclet-sized) USB thumb drive to plug into one of the USB ports. If there is such a drive, then this script configures the system to write the syslog to a file on the USB drive. Also installs the logrotate package to rotate the syslog. This allows logs to become persistent across reboots of the router.
  4. Set up LuCI statistics graphs by installing collectd and luci-app-statistics. Set configs to have all of the graph data get written to the USB drive, so that graphs also become persistent across reboots.
  5. Install dynamic DNS, if they are using it.
  6. Install NUT if the router is connected to a UPS.
  7. Install an IKEv2 VPN using strongSwan. I stay far, far away from OpenVPN, WireGuard, etc. because telling customers they need to install an additional app on each device usually does not make for very happy customers (particularly the non-tech-savvy ones).
  8. Install a guest wi-fi network if desired.

At this point, I have some "non-standard" (i.e. customization) scripts which set up specific configs for each customer: DHCP reservations, DMZs, public IP passthrough, etc.

1 Like

We've been over this in the forum a bunch recently, in general this is a bad idea and breaks things! opkg isn't fancy enough like apt-get is to avoid lossage.

Other than that, your scripts sound like a generally good idea.

This is why I do it as the first step right after a clean install. If I brick the router because of running package updates, then I simply unwrap another new router and skip this script the second time. I have only ever bricked a router once doing updates, and I was able to unbrick it offline at home... and the reason for the brick was apparently a bug in a new netifd package which cut me off from accessing the router. The bug looks like it got patched later, and I haven't had any problems since.

And in another topic @vgaetera points to https://openwrt.org/docs/guide-user/installation/generic.sysupgrade#identifying_customizations that offers the tantalizing hint:

If I'm reading this right, the web GUI "Keep settings" could be able to make a list of the existing set of packages, which then could be available after the flash for (re)installation.

3 Likes

I like this as a requirement. But I was about to add that the the other two bullets, but I then wondered whether it's better to do this in the router or whether people could subscribe to a "Notify me when there's a new update" on OpenWrt.org...

1 Like

Only people like us would want this notification in the portal. Normal people wouldn’t want to login into their router unnecessarily.

Suppose you hear somehow about there being a new update. Would you rather deep dive into the downloads page, or go to your router and have it have a button "download and install latest version from openwrt.org". The router itself has all the information it needs about it's own hardware... It should be able to find the sysupgrade image and check the sha256 against the one on the download page on it's own.

1 Like

However, the normal user is not logging into their router to see if there is an update. So some e-mail notification would be nice. After that people could log into their router and click the “update” button.

2 Likes

Example how others are doing it (update notification):
AVM's Fritz!OS checks regularly for updated firmware on AVM servers.
If there is a new version available, the user gets notified via email (configurable via Fritz!OS GUI).

I find this a very comfortable solution.

2 Likes