Hi guys!
Happy New Year 2022!
This is my first post here, but I'm a reader for a more than a decade, so...thank you at all of you sharing your experience and knowledge, putting energy and time into this project!
As a disclaimer, I'm not an engeneer or any near, so sorry if I use any non-sense...anyway, my question is as follows:
How capable is RPI4(4GB ram) with latest OpenWrt build for home lab and networking?
My home lab setup is:
Modem: ISP with 150 Mbps download and 50 up,
router: xiaomi redmi ax6 + unmanaged switch
proxmox hypervisor (3 VMs running: Home Assistant, AdGuard Home, Nextcloud own instance, Bitwarden)
2 Raspberry Pi's (media streaming (HDD for media files for kids) + mqtt server for smart home) - wired connection
smart home (51 devices on wifi and 8 zigbee devices, Zigbee Hub is WiFi)
3 laptops (mostly on wifi 5GHz)
3 smart phones
TV (streaming and media server for video and music)
smart home is growing (about 20 devices are more to come in our home),
network traffic is on busy days quite high (with Nexcloud and Home Assistant running/managing/compiling/automating...).
The problem is that my router (xiaomi redmi ax6) is crashing wifi (first 5GHz and few mins later 2.4 connection) and then also ethernet connections..which makes smart devices (wifi connected) almost unusable and unreliable, ultimately I had to reset it and it took time to get all devices up and connecting into Home Assistant. I added additional wifi range extender and network is more stable (wifi is really slow).
Now I try to troubleshoot the whole network and use RPI4 as main network router and using xiaomi router as AP, swaping unmanaged switch for managed L2 switch, separating IOT devices into separate VLAN network and putting some AP's (IOT and home network) is planned. But I'm concerned about RPi4 if it could handle all the work? Before buying new infrastructure I would like to check where the best/affordable path is...or just invest in some industrial grade hardware?
What do you think? Thank you in advance for all advice!
The only concern I can see is if your end goal includes internally routing video (IP cameras, streaming media) and file transfers between VLANs. The RPi 4 is surprisingly good but it has a total of 1 lane of PCIe rev2. That's around 4Gb/s which sounds like it ought to be a lot, and it would be all you're doing is benchmarking three or four example connections with iperf3. But in real life traffic management makes only a fraction of that practical; add that sort of thing to latency-sensitive tasks like gaming and Skype and the Pi's not going to keep up.
You'll want a bit of headroom for the worst case. Recommend you go x86_64, something with more lanes of PCIe rev 3 or better. Even an old thin client from ebay will do you. Something like the HP T730 with quad NIC would be plenty, but people have caught on to this and the days of getting them cheap are over. Consider an SFF Core i5; Sandy Bridge (circa 2011) or later would be plenty.
One useful thing about an overpowered core router is that you can put other tasks on it; naturally these tasks should be closely related to routing: a confusion of purposes can cause a mess when downtime for one results in downtime for another. (For example, I wouldn't use my fileserver for routing, as convenient as that would be). But I run mqtt (mosquitto) and AdGuard Home on my router, the latter in an LXC container. Both of these benefit from being able to send/receive on any or all of the router's networks and therefore don't need to be routed themselves.
As far as processing power goes you'd be more than fine with a ARMv7 (32-bit) however keep in mind that there's no hardware crypto acceleration and USB ethernet may not be everyones cup of tea. PINE64 RockPro64 (with either USB or PCIe NIC) might also be a slightly more powerful option which also does hardware crypto if that's of interest. You might also with to look at another distro if you're going to utilize somewhat powerful (capable) hardware and have a better selection of packages and smoother upgrade path. x86 might also be a viable option depending on hardware...
OK, I get it, maybe to weak and I don't want bottlenecks.
What about virtualizing openwrt in proxmox. The machine I'm running proxmox on is Intel Core i5 7500T with 32GB RAM. The memory and CPU resources are not near 50%. I'm planning to install second gigabit NIC and I already ordered TP-Link SG-2218 16 port gigabit managed switch. Do you thnik it should do?
That will be plenty: containerizing OpenWRT works well, with a few caveats:
While OpenWRT is now available in the standard LXC image repositories, I've had much better outcomes using @mikma's generated ones: https://github.com/mikma/lxd-openwrt
You will lose flow offloading. It may not matter much for most purposes, especially with that hardware and your WAN bandwidth. But fat single connections like VPN links, big individual downloads and internal video streams will use more CPU than they otherwise would. But again, this probably won't matter on this hardware. Edit: If you're virtualizing with KVM rather than containerizing, this problem doesn't occur: since OpenWRT's own kernel will be used, flow offloading will work. The tradeoff is that virtualized networking, even using the thin virtio driver with macvlan, still incurs noticeable CPU overhead.
See my comment above about a confusion of purposes: using the same machine for routing and other unrelated things means that downtime or maintenance for one purpose can result in unnecessary downtime for the other. I used to run a router containerized on my fileserver (just like I said not to do). But when my fileserver needed significant attention for any reason it meant I had to get my internet connection some other way while I did it, just for example.
Not trying to scare you off; I like containerization a lot. The lack of flow offloading is rarely a concern except when doing large individual transfers on weak hardware -- and most people find they have to turn off flow offloading anyway once they get into traffic shaping. Currently I'm doing it the other way around: I run OpenWRT natively an x86 router and I use its fairly basic LXC support (command line only, the luci app doesn't support my use cases very well) to run a couple of containers for AdGuard Home and a couple of other things whose uptime requirements are essentially the same as the router's.
I guess it should be fine, not really a big fan of running containers/VMs for firewalls for home usage but it should be fine. You might want to consider using pfsense etc if you're going to this route.
I'll try first by adding managed switch and separating networks by putting low quality ESPs on separate wifi AP (and VLAN). I might try RPi4 router because it is already up and running with usb3 ethernet. Really don't have time to learn pfsense/OPNsense (but considered virtualizing it on proxmox if RPi4 will not keep up with network load).
It would be useful to have two devices acting like a router (similar to cluster ).
Just one more question before I start to rebuild my home network...and before I buy some new equipment.
I have RPi4 with OpenWrt running, for a few days all the AP's and wired connections are through unmanaged switch. The WAN connection from modem comes to USB 3.0 ethernet adapter on RPi4 (works great btw), the LAN port on RPI4 goes to unmanaged switch.
Before I get a new managed switch I was considering two options for seting up network (VLANs etc):
Using two lan ports on RPI4
WAN connection ---> ethernet adapter USB3.0 on RPI4
LAN port RPI4 ---> managed switch
Using one lan port on RPI4
Modem connection ---> managed switch (WAN VLAN x)
RPI4 LAN connection ---> managed switch (LAN VLAN y)
Which route is better (easier for a user) to setup? Which device (switch or router) should handle DHCP server, VLANs, etc.? Sorry for this question if it's stupid, been reading stuff about VLANs for a few days and trying to understand some things before I start to mess with the network.
Thanks!
The first. Especially as you already have it working.
The router should generally do things like DHCP, DNS, etc. VLANs will need setting up on any device that understands VLAN tagging (in a home environment that's likely to be just the router, managed switch, and possibly APs.)
Connect WAN cable to port1 on switch, LAN cable to port 2.
Then, configure switch LAN10 as tagged, LAN as tagged, ports: WAN -> port1 and 2, LAN-> port 1 not a member, rest ports are untagged (for now).
PVID -> port 1 is VLAN10.
Is this the logic?
Edit: I'm have eyes on this switch: TP-LINK TL-SG2218. That should handle this?
).