Is microsegmentation feasible with OpenWrt using stacked VLANs?

Installing and Using OpenWrt
1

I’ve been setting up VLANs for security and while it’s better than nothing, I’ve been realizing more and more that it’s not really what I want.

Having a big pool of “IoT devices” separate from a big pool of “PoE cameras” from a big pool of “homelab servers” from a big pool of “personal computers” is still, iiuc, a long ways from a “zero trust”–style architecture.

The thing is, there’s usually very little need — at least as a default status quo — for devices within each pool to talk to each other either. So it’s great that my PoE cameras can’t hack directly at the homelab servers, but why can the cameras probe at each other? Likewise with the servers, most of the services running thereon don’t need any larger blast radius than themselves alone. Certainly there would be a few connections between them here and there, but for the most part they should all be isolated from each other too.

I think what I want, rather than an IoT vlan is an IoT firewall zone. Likewise rather than a services network, those services would be grouped in a zone. And I would turn off forwarding within most all of these zones. Really most of the interesting work happens between zones: I don’t shares files between individual laptops, or have much interaction between hosted services, but rather “all the isolated laptop zone devices” should be able to access “all the isolated host zone devices” kinda in funnel fashion!

Where I’m stuck is that with OpenWrt it seems like implementing this would require setting up a separate interface per device and/or service in the UI! And then since these devices/services are physically all over the house plugged into a bunch of different auxiliary managed switches I’d also need a separate VLAN per device/service as well? That’s a lot of set up and a lot of VLANs!

Looking into how many VLANs is “too many VLANs” I found some discussion on more enterprise-grade networking forums they’re talking about not using VLANs for security. Instead they’re using port isolation and using routing/firewall to manage traffic. But iiuc port isolation doesn’t really work in OpenWrt, and especially not with DSA switches?

Is there any reasonable way with OpenWrt to set up a “subnetwork” where all the devices are isolated by default? Just thinking in terms of how switches themselves work it seems fairly infeasible without either this mythical “port isolation” (i.e. packets never go directly between ports 1–7 but only the uplink port 8) or VLANs.

Now the wiki says that OpenWrt supports not just 802.1q but also 802.1ad VLAN standards. The latter being stacked (“QinQ”) VLANs! What I imagine using this for would be giving each device a nested VLAN ID within the overall VLAN class. So I’d still have an IoT VLAN but inside that each device would be on its own. Has anyone else done this? I don’t see much discussion here at all about stacked VLANs much less using them this… ambitiously…

Am I crazy for wanting this? Is it basically an impractical pipe dream with consumer grade hardware? Or am I missing some other angle here, some creative application of IPv6 or Wireguard or distributed routing or …?

2

If you make everything go through the router, then you'll bottleneck your network speed. Most of the time you have say a VLAN and if laptop A needs to talk to laptop B it just does ARP / NDP and finds laptop B's MAC address and sends packets direct to laptop B at layer 2.

This is fast, the router never even sees the packet, and the switch can operate at full line speed. Furthermore if laptop C and D are doing the same thing, they also operate at line speed through the switch fabric which can do gigabit between any two ports simultaneously for all pairs basically.

As soon as you need to route between laptop A and laptop B, then some router has to see all the packets and do firewalling. So that's slower and/or requires BEEFY hardware to do this, and is gonna max out at like a gigabit or two TOTAL between all pairs in your network.

So, you can do it, but you probably don't want to as you're essentially talking about a VLAN for each device and the router routes everything to everything else, and also you break all the multicast/broadcast based network discovery stuff (like for streaming things and mDNS and printer discovery and whatever).

I think segmenting by purpose is sufficient. All the cameras together, all the printers together, all the IoT devices together (or segmented by sub-purpose)... you're already doing a LOT.

Further, put a firewall on every device that supports one (laptops, desktops, servers, NAS, etc) and you're probably just about maxed out the practical security stuff.

3

That’s the thing. I’m already doing this, for basically everything!

My cameras live in one VLAN… while the NVR server lives in another. So I guess if the cameras want to send each other 4K valentines directly we’re all set but otherwise my router’s got to keep up.

My workstations live in one VLAN… the NAS lives in another. So in the very rare occasion I connect directly from one laptop to another, not going to complain about layer 2 packet handling that afternoon! But 99% of the time the file I need is on the server, and my router is deciding which packets can/can’t go there. Meanwhile the kids are installing who-knows-what on their computers which have 24/7 access to however my less-technical spouse has configured their laptop, all grouped in the home network :thinking:

That’s what I mean by funnel. What’s the use of this:

(cameras pwning each other at GbE speed)    =|
   (servers pwning each other at GbE speed) =| router
 (laptops pwning each other at 11be speed)  =|

When what I really want is this:

laptop A   =\         /= inkjet printer
laptop B   =\         /= laser printer
laptop C   =\         /= NAS server
               router
NVR server =/         \= camera 1
                      \= camera 2
                      \= camera 3

In practice, my router is keeping up with ± the resulting traffic already so I’m not worried about that. What I am worried about is why my inkjet printer would want to be talking to my laser printer anyway. There’s no reason for it to be doing that! And knowing that thanks to them being grouped together in a shared layer 2 network it could already be doing that without the router even seeing those packets is not a selling point, sorry :-/

4

First of all, apologies for the tone of my prior post. I was typing the heck out of a tangled mess of doubts that had been growing against my own network design but it kinda came out as against your reply, sorry.

I still do kinda want what I want, but… it’s complicated. Meanwhile I think you’re onto something.

From this perspective I’ve been doing it all wrong. Because yes, setting up Avahi to forward mDNS has been on my list for years now. On my desktops/laptops I can configure a printer using a hostname or static IP, but we lost the ability to print from our phones when the printers moved to their own VLAN.

Maybe that should have been my first clue.

The PoE cameras are probably an even better example. I adamantly stated the cameras are on the Camera VLAN and the NVR server is on the Server VLAN.

But why though?

The NVR container on that server should probably, simply, be on the cameras’ VLAN, no? That would be a ton of traffic that could then be switched more efficiently like you say. And at what risk? I can still block that VLAN to/from the WAN, adding a possible exception on the router for the NVR guest itself to do system updates and whatnot. (Though that can actually happen via its host machine in practice….)

Similarly, why aren’t the household printers (and probably even the homelab servers…?) simply on the “Home” VLAN? Is a printer any less trustworthy than the Facebook app on my spouse’s phone, or whatever warez the kids are finding for their computers? [There’s a whole digression here about outdated firmware and whatnot but at the end of the day I simply just don’t see a way to let the kids use the printer from their computers in a way that isolates the printer from their computers! So again, why did I think separating the home printer from the home computers was the right idea?]

This circles back. I don’t think you’re wrong, but it’s kindof a much better way of explaining why I want what I want at least: basically I wanted this, but centrally managed. But thank you for pointing this out. Because as much as I wish there were an easy way from my central OpenWrt router to say “this server should only accept connections from this one other server”, there’s already an easy way to configure that on that one server. I should get better at that!

1 Like
5

How many printers have hacked another printer, then generated some kind of ad-hoc network to spread?

Don't do this. If you want to see the reason, unplug your router and watch your cameras stop recording.

Yeah. You are over complicating your home network. If you ever get sick or pass away, no one is going to be able to fix it if it breaks.

2 Likes
6

I tend to agree that (for home and small business use) a simple VLAN structure is usually sufficient. Moving to a true zero trust architecture is extremely difficult and will break things you would normally expect to work.

That said, if you really want to go to the level of port isolation and the like, I'd do this with a managed switch that supports this function and layer that on top of the VLAN structures already in place. This way you can specify what ports can talk to other ports on the same L2 network and the rest gets handled at L3 for the inter-VLAN routing/firewalling.

7

Use a firewall on the server then.
There you can have fine granular access policies.

8

just having a bit of a rant here, at the end of the day when someone says ‘vlan’ I think ‘multiple networks over one cable’, that’s the basic point of it

if you have something planned where you aren’t doing that you should probably think a little about your proposed setup

9

I think youre well on your way towards a better system. Youre not wrong that the one printer doesn't need to talk to the other printer. But yeah the NVR should absolutely be on the same VLAN as the cameras...

One thing I'm doing is I have a cfengine server and I push policy to all my Debian machines using it. It let's me configure firewalls centrally and have them deploy to each machine.

you'll probably say that everyone has windows or macos machines and thats a valid concern, so just work on fixing that problem by converting everyone to linux :wink:

I'm also beginning to move more things to guix, which again means central config and I can push the config via guix deploy from a build VM

10

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.