How to upgrade x86 OpenWrt?

From the doc it seems all I have to do is executing sysupgrade -v path.img, but I kept encountering two issues:

#1
sysupgrade kept complaining that Image metadata not found.

I built the image by following the Quick Image Building Guide. It built successfully, but seems to lack metadata. I tried enabling "Record files checksums in package metadata" in menuconfig, but it didn't help. I wonder how I could make the final image contain the metadata?

#2
If I forced it to upgrade by specifying -F, after the image is flashed and machine rebooted, nothing seemed to have changed. I specified the new image to have bash built-in, but after rebooting, bash didn't exist.

Am I doing it wrong? What's the correct way to upgrade x86 openwrt?

P.S. I want to replace dropbear with openssh-server, and busybox with bash for the x86 build. Is it something that's officially supported through a custom build? I simply deselected dropbear and selected openssh-server, but couldn't find where to deselect busybox.

BusyBox is not just a shell for OpenWrt, so it's unlikely you can remove it easily.

Also, OpenWrt in not a general-purpose Linux distribution, it may not fit your needs the best.
Perhaps you should use something like Fedora/CentOS/Ubuntu/Debian.

OpenWrt upgrades are significantly more problematic than a general-purpose Linux distribution.
And this is relevant for both system upgrades and package upgrades.

Thanks for the quick reply. OpenWrt actually fits my needs perfectly, since I need a linux based router system. I just wonder if I could take the chance to have some tools in their full version since the host is a x86 machine.

And the need to upgrade an existing system is quite universal I think.

I used to be able to just flash the image for non-x86 OpenWrt. I was hoping the same easiness could be apply to the x86 version.

I wouldn't try to upgrade your installation. I'd keep it in case it all goes wrong which lets face it happens easily and just did. I presume you've got oodles of disk space because your running openwrt on x86 hardware as all disks are huge to openwrt. Openwrt on x86 is absolutely mustard as you can multi multi multi boot and experiment without any great consequences.

No experience of it but you might want to look at lxc containers to run a full Linux system in a VM for the extras.

Are you using the squashfs or the ext4 version? The squashfs version is more like what runs on smaller routers, it would be more amenable to upgrading with settings saved.

Busybox does lots of things besides the ash shell. To make the shell bash, just add the bash package don't remove busybox.

Thanks guys. It's a bummer that the general message is that openwrt just isn't designed for x86, you are on your own if you do decide to use it.

I found an article online that suggested swapping the kernel manually and then upgrading all opkg packages. I'll give it a shot, if that doesn't prove successful maybe I have to consider other systems.

@mk24, yes, I'm using ext4 to make openwrt utilize all disk space. If I switch to squashfs, is it possible to expand the partition size to occupy the whole disk space?

No you can't expand squashfs but you can build yourself at any size.

I wouldn't go so far as to say openwrt just isn't designed for x86 at all. Personally (and I've no desire to start a debate on this) I'd say it's the best platform for it because of its flexibility and CPU power - nothing comes close if you're running openvpn or a web server.

You can also consider pfsense or opensense... Or multiboot with them... And then return back to openwrt because of its great support and multitude of packages and functionality.

I think the message should be that x86 is a bit different, there's a whole bunch of different opportunities and you should use openwrt in a slightly different way to a commercial plastic box router.

I had been using extroot on OpenWrt 18.06 x86_64, but the squashfs image I tried for 19.07 had not enough free space to utilize extroot, although I didn't test the latest minor release.

@hgl I run OpenWrt x86_64 in a VM as my main firewall/router. It works incredibly well and to be honest, I quit pfSense in favor of OpenWrt. Happy to elaborate more on the "why" if anyone is interested.

But to your questions I upgrade my VM all the time because I run tweaked master snapshot builds and update quite a bit. I run the ext4 filesystem and upgrades are just as easy as on a standard router. Given my VM runs on an SSD, the process takes just under 20 seconds to update, reboot, and get back to functional.

If you are using the ext4 filesystem, you just upload the *-generic-ext4-combined.img.gz file via the web interface (or scp + sysupgrade if you prefer CLI). That's it. :slight_smile:

2 Likes

Yes, it should be fine if you avoid customizing builds and upgrading packages.
Otherwise, this thread would not exist.

Could you elaborate on that? I customize builds, yet still upgrade my x86_64 VM with the updated builds with no trouble. As for upgrading packages, it’s the same situation as on a non-x86 unit. There are some packages that can be upgraded via opkg, but anything that is tightly integrated with the kernel likely will have to “upgraded” through a rebuild.

1 Like

It is not the same for other general-purpose Linux distros.
APT/DNF typically respect existing symlinks and do not overwrite them unlike OPKG.

Based on my understand of @hgl's OP, it sounds as though his whole issue is not feeding the right type of upgrade image to sysupgrade to begin with.

@hgl, could you reply with the full path.img value that you are passing to sysupgrade?

2 Likes

I think I know why I failed to sysupgrade: I passed the uncompressed image. I guess that's also why it complained the image had no metadata. Thank you so much for showing me the file name you used. I'll try again some time later.

Also a couple follow up questions if you don't mind me asking:

  1. Do you flash the MBR or UEFI version of the image? Will flashing a image also upgrade the boot partition? I'm asking because I have an UEFI only system, and I have to swap out the boot partition in the 19.07.03 image with the one in the snapshot image for the flashed openwrt to boot. I wonder if the next version still doesn't have official UEFI image, will I be able to directly flash the MBR image.
  2. Once flashed, does your ext4 partition shrink to the size specified in the image?
  3. Once flashed, what happens to the outdated packages and kernel modules? Are they automatically removed or disabled? Is executing opkg upgrade enough to bring everything back to normal?
  4. Once flashed, do files in locations other than /etc/config preserved?

If I could just sysupgrade the (correct) ex4 image and everything just works, that would be dreamy.

I'm quite also interested in your VM set up. Do you mind share a little bit?

I'm under the impression that a router is both the first and last defense regarding network security. Wouldn't running OpenWrt in a VM make the attack surface bigger?

But my x86 router is pretty powerful and has much space left after OpenWrt is installed. I don want to somehow run a full distro alongside it. I'm also eyeing for the docker support in OpenWrt 20, which seems to make it possible to run a full distro directly in it.

Do you use something like ESXi as the hypervisor?

I will do the same trial next weekend, as right now I have Installed my AtomicPi with stable version 19.07.3 by tweaking the boot partition and Grub partition. Both partitions (root and boot) are EXT4 and the grub is fat 16, but the WIFI driver is not working on kernel 4.XX and also USB ethernet I am using, is not in default installed. My plan to use 5.xx Kernel with snapshot image and I have expanded my root partition to the max around 15GB (internal eMMC).
just wondering if the setup what @_FailSafe doing, can be applied to my AtomicPi without too much reconfiguration. Thanks

I do run ESXi 6.7 and multiple network-related VMs (DNS, grafana + influxdb for stats collection, etc.) in addition to my OpenWrt VM. The OpenWrt VM itself has four vCPUs, though it can actually run current workload with two vCPUs on my core-i7@3.4Ghz. I give it 512MB RAM, though it honestly could run comfortably at 384MB or so.

I added an Intel 1Gb quad-port NIC into my ESXi box (I use the onboard NIC for ESXi Mgmt) and currently assign one of the quad-port interfaces to a LAN vSwitch (which trunks multiple VLANs to my upstream switch). A second quad-port interface goes to my WAN vSwitch, which is only attached to my OpenWrt VM's WAN vNIC and on the physical side, my cable modem. Obviously, my LAN vSwitch connects to a LAN vNIC on the OpenWrt VM. I also have a separate DNS vSwitch which is used for the vNICs on my DNS VMs and connected to a third vNIC on my OpenWrt VM.

The DNS vSwitch configuration is not really necessary, but it's a little cleaner in my head and easy enough to add in a virtual environment.

The main downside to a configuration like this is that my ESXi box can't be remotely managed if the OpenWrt VM goes down for any reason. But from a stability standpoint, that VM never crashes, so the only real "outage" situation I encounter is a true internet down (i.e. ISP goes offline) scenario. In that case, even a physical router won't save me. But, I do have all my critical VMs set to auto-start in the proper sequence and my ESXi host machine is set to return to previous power-state upon power loss (via BIOS setting). Any time I have had my ESXi host go down (power outage, for example), it just powers back on by itself and all my VMs come back online shortly thereafter.

Feel free to read up on a configuration like this and draw your own conclusions, but personally, I am good with the separation of traffic via vSwitches and consider that to be secure enough in my case. Having used ESX/ESXi in professional contexts, this configuration is actually quite common.

2 Likes

Thanks for the delineation, it's very interesting, and made me want to give ESXi a try.

I'm actually considering switching to pfsense, with a linux distro running alongside it to run docker, etc, to make use of the x86 system more thoroughly.

Curiosity question... what is behind your motivation for using pfSense?

I'm not trying to start a flame war, but it's mostly the fact that OpenWRT gears more toward wireless routers, and the result of that includes:

  • most commands it offers are of stripped down versions.
  • aggressively clean up files, like package index, etc.
  • no first-class support for IDS/IPS systems.

Which aren't necessarily benefits for x86 systems. But these aren't major, I'm still quite ambivalent.

1 Like