How to upgrade x86 OpenWrt?

Installing and Using OpenWrt
21

Understood and I was not asking the question to set you up for a war on behalf of OpenWrt or pfSense. I was genuinely curious as one who has used both, fairly extensively, and ended up sticking with OpenWrt even on x86.

Objectively, what commands does pfSense offer that you wish you had (or had more thorough versions of) on OpenWrt? I have found that a lot of the full and proper commands/libs can be had through a custom OpenWrt build.

OpenWrt does favor “temp” stuff a lot, which makes sense given its history as a router based distro. Historically memory-bound devices and the desire to minimize writes to disk have, and still are, shaping the way it functions as an OS. I have overcome some of that by altering the locations (via symlinks or modifying paths in configs) of some files, like the DHCP lease file, so they survive reboots.

On the IDS/IPS front, you are correct. The gold medal goes to pfSense on that one from a feature availability perspective. That is, assuming you have the horsepower in your x86 box to turn all that on. I found pfSense to be very memory heavy for the additional packages you enable. Especially with things like pfBlockerNG. (I prefer Pi-hole to pfBlockerNG now anyway, FWIW)

A couple more thoughts between the two...

I found the traffic shaping and QoS configuration in pfSense to be downright maddening at times. It affords you a ton of configuration options (as does the whole of pfSense), but requires a ton of knowledge to know how to tweak all the pulleys and levers. Even after weeks of tuning, I still could not achieve the same level of consistent bufferbloat reduction in pfSense that I get with OpenWrt SQM (CAKE) with about 30 minutes of tuning.

I much prefer pfSense’s GUI version of firewall rule configuration to OpenWrt’s GUI. It’s much easier to visually see what you’ve got set up hierarchically with rule precedence in pfSense because each interface has a separate tab with just its rules. OpenWrt’s firewall GUI, while more “pretty”, is far less functional to me. Along those lines, pfSense makes it so much easier via either web console or pftop to quickly view firewall logs in real-time.

The available packages for pfSense are significantly smaller in variety and availability. There are pros and cons to that, of course. A pro is that you get well-curated and functionally tested/stable packages that just work. The cons are that there are a limited number of pfSense devs who have time to test new packages and approve them. So the package selection tends to stay pretty static and they very much favor lagging version upgrades in favor of stability (not necessarily a con). It’s definitely a more tightly controlled environment as opposed to the very open, community driven approach of OpenWrt.

Lastly, one of pfSense’s strengths is also one of its biggest deterrents. It provides one with hundreds of textboxes, dropdowns, checkboxes, and radio buttons to tweak about anything you can dream up on a firewall/router. Even with decently advanced network knowledge, I was finding myself having to refer to pfSense docs a lot to get clarity on many of the settings. A lot of times that led me down the road of landing on pfSense forums. In order to not offend anyone, let me just say that OpenWrt’s community forums are top notch in the level of friendly support offered. OpenWrt forums are generally (sure, there are some exceptions) filled with people who care to help others understand concepts without belittling them. That cannot be understated, and it is one of the main reasons I love to stick with OpenWrt. It has a great community that I haven’t found a lot of other places.

Anyway, I’ll wrap this up here because I just realized I wrote a book. Strictly from a stable network perspective, both pfSense and OpenWrt will get you there. It’s the value-adds that have to be weighed. :slight_smile:

EDIT 1
I lied... I wasn’t done. :joy: Another thing I just remembered that I meant to call out is that OpenWrt has been [thankfully] very accepting of WireGuard. It has not been well accepted by pfSense to date. In fact, here is an example of both their WireGuard stance and what is a very common style of forum response.

4 Likes
22

this is a good question: the short answer is yes, however, hypervisor stacks have come a long way...

providing you pay attention to not inadvertently exposing host or parallel guestOS's over a bridged wan or "untrusted" link, the risks are as high as the hypervisor vendors exposure + code quality + update schedule... the prevalence of the vmware and virtualbox brands ( amongst others ) speaks volumes on that front...

as with most things exposure related... risk and risk assessment/management are commensurate to your attack surface... data centre level protection is on another level...

3 Likes
23

Thanks for taking the time to give such a rich response.

I, for example, tried swapping dropbear with openssh in a custom build (for ed25519 support), but it seemed luci was designed to operate with dropbear. It's good to know you are having good results. Maybe I had done something wrong, will give it another try soon.

IDS/IPS and traffic shaping are more of educational than practical purpose for me, so I wasn't really complaining :slight_smile:, and thanks for bringing SQM to my attention.

I haven't used pfSense intensively, so wasn't familiar with its community culture. Looks like I made good choice by starting with OpenWrt.

All in all, I think you just manage to swing me toward OpenWrt.

A daydream question though, from this Wikipedia list, it's apparent that there are quite few Linux-based router systems that focus on x86. Do you have any experience with them? I personally never heard any of them. Assuming I wasn't living under a rock, I wonder why they never gain as much traction as OpenWrt does? I guess it's either pfSense absorbed most x86 router users, or OpenWrt already fits the needs quite well?

24

That's a good point.

I actually don't really have a powerful x86 machine as a router (I did, but then I changed to a much less powerful one). I guess I'll just move most "server services" to a dedicate box in the LAN.

25

Also to respond to some of my own questions for prosperity:

Will flashing an image also upgrade the boot partition?

Yes. My grub wait time was reverted after flashing

Once flashed, does your ext4 partition shrink to the size specified in the image?

Yes.

Once flashed, what happens to the outdated packages and kernel modules?

All gone.

Once flashed, do files in locations other than /etc/config preserved?

Nope.

1 Like
26

This should depend on the sysupgrade configuration:

2 Likes
27

Exactly. @hgl I failed to mention this as one of the additional things I have tweaked and tuned. I have multiple additional entries for saving either single files or entire directories so they survive an upgrade. It takes just a few minutes to configure the /etc/sysupgrade.conf file and saves a ton of time post upgrade.

I also leverage the /etc/rc.local file to check that certain files/services exist on each restart (which obviously happens post-upgrade) as well. There’s one service I use which isn’t a proper OpenWrt package yet, so my rc.local ensures the service is there at reboot and if not, it pulls it from GitHub, installs it, and sets it to start automatically. Things like that help make the system very update friendly.

My /etc/sysupgrade.conf file for example:

Summary 
root@OpenWrt:~# cat /etc/sysupgrade.conf
## This file contains files and directories that should
## be preserved during an upgrade.

/root/
/etc/dhcp.leases
/etc/firewall.reload
/etc/iproute2/rt_dsfield
/etc/netdata/
/run/
/usr/lib/netdata/charts.d/sqm.chart.sh
/usr/lib/sqm/ctinfo_5layercake.qos
/usr/lib/sqm/ctinfo_5layercake.qos.help
/usr/lib/sqm/functions.sh

In case you're wondering, I created the /run directory and set my nlbwmon and collectd configs to store long-term data there instead of a subdir in /tmp. Though I have stopped using collectd in favor of netdata, I still keep nlbwmon long-term data in /run so it survives reboots & upgrades. There is likely a more ideal spot for that sort of data outside of /run, so if anybody has suggestions I would love to hear them.

And FWIW, here is my /etc/rc.local just to give an idea of some things I do with it:

Summary 
root@OpenWrt:~# cat /etc/rc.local
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

# Create a symlink to /etc/dhcp.leases if it doesn't exist already...
[ ! -f /tmp/dhcp.leases ] && ln -s /etc/dhcp.leases /tmp/dhcp.leases || echo "INFO: /tmp/dhcp.leases already exists."

ip6_global=$(/sbin/ip -6 addr show eth1 | grep global)
while [ "$?" != "0" ]; do
    sleep 1
    echo "Waiting for IPv6 global address on LAN..."
    ip6_global=$(/sbin/ip -6 addr show eth1 | grep global)
done
echo "Got $ip6_global. Proceeding with ip6neigh handling."

if [ -f /usr/bin/ip6neigh ]; then
  /usr/bin/ip6neigh oui download
  /usr/bin/ip6neigh restart
else
  echo "ERROR: ip6neigh does not exist in /usr/bin. Installing now..."
  /root/ip6neigh-setup.sh install
  /usr/bin/ip6neigh enable
  /usr/bin/ip6neigh start
  /usr/bin/ip6neigh oui download
  /usr/bin/ip6neigh restart
fi

/usr/bin/iperf3 -s -D

exit 0
2 Likes
28

For a year now, I building up a VMWare 7.x enviroment on a HPe Microserver with 4 SSD's.
I use Freenas as the under laying storage (run it virtual)
For now I run Domoticz and zigbee2mqtt with an USB Texas Instruments LAUNCHXL-CC1352P-2 in production.
Very stable setup!

I started this setup to make stable, allmost profesional and energy efficient home lab/server.
And if I need new server hardware, it is easy to migrate.

This week I started to implement OpenWrt x86 with a USB Netgear a6210 wireless adapter.
The first test It all seems te work more then great..I need to do some wireless stability tests.
I want to combine it with a virtual appliance Sophos XG that will be my first line of defence to the internet.

But I can't get Openwrt to work like a dump AP:

I like it to see others are also have fun with VMware to build there perfect setup.

1 Like