Using a router as a general-purpose server

Hardware Questions and Recommendations
22

For what its worth, as an nerdy amateur I went down this route for OpenWRT being router running family email server + wordpress site:

Main router = x86_64 core i5 (4 core / 4 thread) + 8GB RAM +120GB SSD runs at 6.7watts idle.
OpenWRT installed on USB stick and a NVMe SSD manually password encrypted for data storage.
WiFI M.2 Qualcomm NIC for 5g (WPA3 LAN + WPA2 GUEST + client isolation), plus Mediatek USB 2g (WPA3 LAN + WPA2 GUEST + client isolation)

Openwrt runs OpenVPN, Exim, Dovecot, ClamAV and Qemu-KVM VM.
Ports 25,80,443,587,995,1194(actually something else) open to WAN. 80+443 forwards to VM.

Qemu VM runs Ubuntu 22.04 LTS and has Apache, MariaDB, Wordpress.

I figured Wordpress is the most likely to get hacked so stuck that on the VM in its own zone.

OpenWRT also runs Crowdsec (registered to a few block lists) and Fail2ban (monitoring Exim/Dovecot and OpenVPN).
So stuff gets blocked before even trying and the stuff that does try gets a banned in short order.

OpenWRT Firewall zones:
The LAN cannot access the VM zone and vica versa without going via the VPN zone (which is OpenVPN). OpenVPN uses preshared key + password protected key and the keys and PKI are kept on a separate encrypted machine (only decrypted when needed). SSH is behind the VPN.
Ubuntu VM also runs own firewall.

I would have gotten round to putting fail2ban on the VM or perhaps setting up Crowdsec to report back to main instance. Except it turned out no longer to be required as Wordpress not used. So actually the VM is down and 80+443 are closed again.

Have thought about moving Exim + Dovecot into the VM now Wordpress is not needed.

But anyway. understand its risky (but don't appreciate how risky because amatuer). I keep on top of patching and CVE notifications. And rely on Crowdsec + Fail2ban.

Seems to work ok. Would have liked an extra 2 cores and 8GM RAM, but at the time the 7th gen i5 was much cheaper as a second hand ebay purchase.

23

No, you're right. I'm planning to run a couple of containers or VMs on the device.

24

Thanks a lot for sharing this.

But while we're here, wouldn't WireGuard be better off installed on a different machine (just like the web-server, etc., would) with only forwarding the WG ports?

25

It is one option, but I fail to see why you wouldn't run WG on the router itself?

  • it is much easier to set all the routes properly
  • less overhead
  • no NAT and no issues with stale connection tracking entries
  • also, Linux WG implementation:
    • doesn't dynamically allocate memory when processing packets and provably doesn't have "use after free", "out of bounds" and similar bugs
    • doesn't parse untrusted content - you have to know the private key to send packets that are going to be parsed
26

Some people run VPN endpoint separately probably due to the amount of users (more users, higher CPU load), but for x86 router it should be OK to have a few concurrent users.

27

You should really, really get a separate server!
Think about this:

  • Security! Keep it simple. Use a single entry point to your network. How many doors are there at your house? There is a reason why a security hole is called a backdoor...
  • Separation of concerns: each component does what it's best suited to do.
  • Flexibility: You can upgrade the router, and you don't have to reconfigure everything.
  • Storage: Think about what are you going to store! Imagine the storage failed. What would that mean? How much work will you need to put to restore the data... if it's even possible. Even the simplest thing you are storing will take effort on your part to restore.
  • You should really want to be running RAID 1 which only works on a separate server.

You want a all-in-1 device use a server... any computer will do, even some VPS providers are running on commodity hardware, and handling fail-over in software.

  • I prefer x86 for the flexibility of changing components:
    • Hardware video decoding for your media server? Add a CPU with better iGPU, or even GPU
    • More RAM for your security cameras... plenty of choices even up to 128 GB
    • You want your file server with an extra 4 x m.2 running at PCIe 4x4? Skip the GPU and add an expansion card, just good luck getting that data speed over the network
    • You want your server to have SFP+ 10GbE... expansion card
    • Data security: if one of the storage drives fails, you have a copy on the other one
      • HDDs experience data rot. I have a picture from years ago that looks exactly like this RAID1 partially protects against this
      • SSDs also experience data rot and have a limited number of writes, though you may never reach it
      • Again: Think about what your are storing? What if you lose the data? That picture from when you were younger? That bitcoin key on a HDD thrown in the landfill? Or that website that it took you a while to set up. Some can be restored, but you will still need to put the work into it.

It's no wonder the ATX standard lived so long. The flexibility it offered in 1995 is still valid today. Just think about the effort you are going to be putting into selecting the components for a small factor PC vs... going to any online store and picking any of the components that are compatible

28

+1; however that cuts two-ways. If the data to be stored is not sensitive then security is not a big issue, e.g. if you just want to share out your media library within your house.

You typically can keep OpenWrt configurations across updates, and simple file sgaring is not hard to set-up in the first place so eveb recreating the config from scratch is generally not a big challenge.

That is true, but also completely independent on where/how you make that storage accessible.

I respectfully disagree:
a) you can get external units that do raid1 internally and present as a single raw USB disk to the outside (so no separate server required)
b) a beefy enough router (say a raspberry pi5, intel N100, or similar) will easily do raid1 and better
c) depending on the nature of the shared files, on-line redundancy might not actually be required...

You might want that, but that s IMHO not a generally agreed upon requirement... (and personally I would try to avoid SFP as much as possible, compatibility is not nearly as universal as one would hope/desire).

That is not wrong, but as shown above, you can create redundant storage also for routers and redundancy is not always a requirement anyways.

Now, I am not saying your arguments are bad and you are certainly free to have and share these, but I want to push gently back on assuming these will be shared universally and must be true for everybody.

29

That is exactly my point... :slight_smile: the widest compatibility is a for PCIe card that you plug into the motherboard.

if you just want to share out your media library within your house

How much effort did you put into getting together that library?
Do you have another copy on your computer?
That is what I call a "Manual RAID 1". How much effort are you putting to keep the libraries in sync?

Here it the recommendation from Mr. Jeff Geerling himself:

Now regarding:

single raw USB disk to the outside (so no separate server required)

Will you be able to get at least 1 GbE for a client reading from it? Storage (HDDs, SSDs) -> USB interface -> Router -> Client. What about if you want to upgrade to 2.5GbE?

a beefy enough router

I look at it this way: I would not be willing to put that server in the front yard behind a locked gate. I would rather have it inside the house.

Below he describes the ideal setup. Redundancy is always a requirement. You have put in some effort into generating that data, even a media library. If you lose the data would need to put that effort again. In reality what will happen if you lose it is that you will only regenerate the bare minimum and some will be lost.
I would say that having a local RAID 1 server is an excellent starting point. Get a cheap second hand hand computer. All it needs is 2 or 4 SATA drives. In my setup reading off a RAID 1 of 2 x 8TB HDDs saturates a 1 GbE. Moving the bottleneck to the HDDs would require 2.5GbE networking.
The next upgrade is possible (NVMe drives + 10GbE) with: two expansion cards (PCIe cards for both) or MB/CPU/RAM swap (again: you have more options) vs entire system replacement for a router (less options).

30

Well, my point was SFP compatibility sucks, so maybe forego SFP... really the only reason to go SFP is iy you expect wanting to change the physical layer or at time of purchase you do not know that layer (or you painted yourself in a corner and need DAC)... my point was less that a router/server striuctly needs PCIe slots...

Not much.

Of course... (on a btrfs raid5 mnade oput of 4 disks, given btrfs raid issues, not a terrible secure set-up, but I will not run raid1 as I lack the resources to invest half my storage into redundancy.)

None...

No, but here is the point, it is plenty enough for a couple of VLC clients to play mp3 audio or even play DVD rips (fewer in parallel). I do not want to upgrade to 2.5 Gbps, the disk is USB2 so all of that would be a fools errand... Look this is a home media-mostly-music sharing use-case and not a NAS storage to feed a herd of high performance workstations with oodles of data to chew through...

Optimality/Ideality is in the eye of the beholder...

Not in my world, I have data that is precious and that I will try to guard and replicate and data that can be recreated quite easily and for that case redundancy is not very high on my priority llist. I accept that for your data that might be different, but technically that invalidates the 'always' in your claim somewhat...

I might have used the wrong word with calling this a media library, it is a relatively simple folder structure that contains individual MP3 files or DVD ripped folders. Should that disk die I likely will copy the same folders onto a new disk, there is little to get lost and little thinking will be involved.

And that is exactly what I want to avoid, my router runs 24-7 (VoIP reachability and consumes around 13 Watts), a cheap second hand computer even if exactly as efficient as the router will already double my continuous power draw... but what you describe will eat considerably more Watts, at that point I would drop the idea of having the media library accessible via the network completely.

Again, by all means go wild and do this with your data, but for my use case of supplying a family of 5 with access to a shared pool of MP3s to play in the background, that solution would be complete over-kill. Don#t get me wrong, I am not saying you should not be doing that (and I can envision use-cases where I would do the same), and likely I would NOT recommend to operate something like this from a router.

P.S.: I prefer written articles over videos, as video often takes up more time for a given content than text with graphs.