SOHO Network Topology and the like; care to share?

Installing and Using OpenWrt Network and Wireless Configuration
21

Right, so you have all your LEDE boxes authenticate to the switch, only authenticated boxes will get access to the Admin VLAN. You have the LEDE boxes listen on the Admin VLAN and only have their administration interfaces on that VLAN. In order to admin a LEDE box you have to access the Admin VLAN yourself. If we were going to color code it let's call it Brown or something.

One way to admin, would be to walk up to and plug into the managed switch, and authenticate to the port via 802.1x. Now you're sure that only you can admin things.

Suppose instead you'd like to also admin without walking up to the Zyxel switch and plugging to the port. You could create a separate wifi SSID on say the LEDE box that firewalls the green subnet, and hook up that wifi network to the brown vlan. Then you could authenticate to this admin wifi and do admin without physically plugging to the switch, but still authenticated. You'd have to disconnect from the green wifi, but this is pretty painless, and you could maintain a wired connection to the green wifi. Or you could get yourself a USB wifi dongle and have two wifi connections simultaneously.

If you want to admin from the green network itself, now you lose the authenticated/encrypted/separate nature of your admin network. It's not advisable as connecting to the brown VLAN and bridging it to a "brown" wifi SSID is pretty easy and much more secure.

22

Perfect. This makes all sense. Thanks again

23

Sure, hope it works out for you. Give it all a try and let us know.

Also, I'd advise multiple ways to access the admin vlan, so you can't easily get locked out. For example enable an authenticated port on the switch, as well as an encrypted authenticated wifi SSID. Maybe also have a device on the VLAN that has a walk-up console, whether that's your central router box or a serial port on one of the LEDE boxes, whatever.

I guess in case of lockout, worst case, you reset the Zyxel back to factory settings using the reset button, and then reconfigure it (so make sure to have settings backups).

Here are some tips for the Zyxel switch:

  1. Immediately download the latest firmware and apply it. Among other things, it fixes a GUI bug that causes the web console to have white-on-white text in some browsers.

  2. After configuring your settings, they apply right away but aren't permanently saved until you click "save" in the upper right hand portion of the screen.

  3. Backing up your settings is pretty painless, do it after every major reconfiguration, right after clicking save.

  4. Configure your LAG groups first before doing anything else, for example if you're bonding multiple NICs on the central firewall.

  5. The QoS features of the switch are very useful, consider if you need them, particularly for IP Phones. Same is true at each LEDE box, setting up QoS can be very useful, for example to reduce the effect that the guest network has on the main network functions, and/or to prioritize admin traffic very high so you can admin things even during a packet storm. Also bandwidth limits can be useful for guest network and/or IoT network.

1 Like
24

All points well taken. Thanks

25

Iā€™ve received my Zyxel. My test setup is now:

Modem > Tomato Router (172.16.1.x) > Zyxel > OPNsense (laptop with a single NIC); and a single LEDE device hanging off the Zyxel.

Tomato connects from port 4 to port 1 on the Zyxel. The Zyxel is configured to use DHCP for its IP. OPNsense is connected on port 2 to the Zyxel. The sticking point for me right now is how to get OPNsense into play; to get the WAN interface to work. The LAN is assigned 192.168.1.x/27.

Iā€™ve created Vlan ID 10 (WAN_0010) on both the Zyxel and OPNsense. Iā€™ve tried various Vlan configuration settings on port 1 and 2 for OPNsense to pick up a DHCP address from the 172.16.1.x on WAN, but all failed.

Hosed my access to the Zyxel and had to reset my device several times. On an interesting note: when resetting the Zyxel it reboots to the manufacturer default settings; except it wonā€™t revert back to the static IP. Instead it remains stuck on DHCP. Weird.

Are you familiar enough with the Zyxel GUI to suggest what settings to use for the Port and Vlan Port under Vlan configuration? I spent quite some time on this, but itā€™s not gelling for me. It may be that I have the physical and logical topology all confused; but damn this shouldnā€™t be that difficult. Itā€™s frustrating to be stuck this early in the setup. Ha, ah... ; )

Cheers

opnsense
Selection_027

26

On the Zyxel, you need to go to the vlan port screen and make sure that the port your OPNsense is connected to is a member of both vlan1 for untagged frames, and the vlan 10 with tagged frames. Then you need to tell OPNsense to use tagged vlan 10 on its eth interface. That part I don't know about, because I have no experience with OPNsense.

Finally, you need to look at the port the tomato is connected to and make it a part of vlan 10 untagged so that every frame received from tomato "becomes" a part of vlan 10 when received by zyxel.

hope that helps.

27

Done that.

When you configure the interfaces from the CLI on OPNsense (and this is similar to other distributions) you have the option to assign LAN and WAN to an interface. I only have one physical interface. So I assigned LAN to the actual physical interface em0 and WAN to the VLAN interface em0_vlan10 that tagged for Vlan 10 (WAN). This is where I was not sure, if my grasp of physical vs logical was on target. But I believe this should be right?

Ah, I think I did Vlan 10 tagged for this port. I will change that to untagged. Vlan 1 remains untagged for this port as well?

Thanks a bunch!

28

No, you can only have one untagged VLAN per port, because it has to decide when it receives an untagged frame which vlan it should be in, and it needs to be unambiguous. Every frame from Tomato needs to be a part of vlan 10, and since you aren't making tomato tag its frames, when the zyxel receives the frame it needs to know to tag it with 10, so hence untagged frames are tagged 10 (that's what the port/untagged membership means).

29

I've been separating all my "untrusted" devices onto their own SSIDs for years, each with their own VLANs (as well as guest networks). Not only does this help in the event of a compromise with the device, but also keeps them from snooping more than just their own kind. On their own VLAN, you can not only restrict devices from "phone home" actions (for example, TP-Link home automation devices connect to the TP-Link cloud even though you "turned it off"), but also provide locally managed "custom" DNS. I run unbound for DNS and kea for DHCP on a non-LEDE/OpenWRT host rather than dnsmasq for both clearer and more flexible configuration, as well as a belief that the security of those two products is greater than dnsmasq.

The more complicated your drawing, the harder it will be to understand "what if" questions.

Firewalls are sort of the opposite. The fancier the GUI, the less likely it is that you can understand the rules they produce.

You may have problems with WDS and VLANs, as VLANs are an Ethernet concept and, at least last I checked, weren't supported as an 802.11 standard. You'll probably need a Level 2 encapsulation to get out to that remote group if you have more than a couple VLANs to bridge. You'll see recommendations of using OpenVPN or the like, but the throughput when running on home-grade WiFi platforms is pretty poor. If only a couple, and your WiFi drivers support it, you can use multiple SSIDs, one for each VLAN you want to bridge.

As for myself, I have always assumed that WiFi is not secure and that it was the responsibility of the attached devices to encrypt, so tunneling without encryption is sufficient for me (GRE, for example). Hence one of the reasons my fireplace switch is still manual; it would annoy me if someone turned on the lights in the middle of the night, but at least it wouldn't burn down my house.

Going along with this, I don't "trust" any WiFi-connected devices to have direct access to my internal networks, even if they are "known" devices. You might want to consider requiring authenticated VPN access.

I'm not sure how you're managing the DHCP from the modem, either for IPv4 or for IPv6 with the /60 that many Comcast head ends will supply. Also, you need to be able to access the modem's management interface. The way that I handle it is with a "box" that is directly connected to the modem that isolates the rest of the network from potential DHCP changes, as well as allowing my infrastructure-level management VLAN to access the modem's interface. My main firewall always knows it's outside interface address is fixed, simplifying the rules.

I separate my network devices' management interfaces from those for service hosts as I very seldom need to manage the network infrastructure. Your VLAN configuration needs to be protected as strongly as your firewall rules. VLANs are pretty much "free" with decent hardware. There is some risk of a rogue device coming online and "guessing" the right VLAN to get more access than they should. To help mitigate this, all segments require the correct VLAN tagging for the port; nothing is assumed at the switch.

My setup is a minimal Ubuntu system running on PCEngines hardware in front of the modem that handles DHCP from Comcast and provides one level of firewall with netfilter (not iptables). DHCP is handled by dhcpcd, as unpatched ISC DHCP doesn't handle both the interface and the /60 PD.

Everything is handled by Cisco SG300-series switches. The Ubuntu "talks" to my main firewall, running FreeBSD and "raw" ipfw. LEDE/OpenWRT handles the WiFi portion of the links, associating SSIDs with VLANs and either routing them over wired or over 802.11s backhaul between the units to the main firewall. With a single point of connection between the various network segments, it's easier to visualize and rationalize the paths and what data flows where.

DHCP is supplied by kea and the Cisco DHCP relays built into the switches. These Cisco switches can restrict the ports on which DHCP servers can be located, "preventing" a rogue DHCP server from impacting your entire network. Local DNS is provided by unbound which goes "straight to the source" rather than trusting Comcast's servers. DNS is custom for the network served, providing local knowledge "on a need to know basis". NTP is local. Service hosts are typically run within FreeBSD jails, with full virtualized network stacks in each jail.

File servers run FreeBSD on mirrored ZFS. Rolling snapshots provide at least a little short-term hope against ransomware. Backups are the fall-back. I do not run any SMB services.

If you don't fully understand IPv6, you might want to get things working with IPv4 first, then slowly introduce IPv6. Just remember that assuming that NAT magically protects your network is asking for trouble.

1 Like
30

That's all I do right now as far as segregation is concerned. I have all phones, camera, media devices in separate Vlan with its own SSID.

Hopefully that's something I will moving towards to. More segregation, more SSIDs. All cloud cameras in one Vlan, all Androids and iOS devices in separate one, etc. Seems to me like it could easily get too complex fast.

You might be right. If so, I will need to redesign my whole approach. I though I saw some options for Vlan settings on the Engenius that I will use. I'll see.

When you say internal network, you mean trusted network part of your internal network or are you talking about the known devices on your untrusted network that are on a separate Vlan? You would want to use VPN for that?

Could you elaborate on that? How do you enforce that through the Vlan settings?

Using Vlans, correct?

Your setup is probably too complex for me. i am already struggling with just getting the Vlan porting figured out. :stuck_out_tongue_winking_eye:Thanks for sharing though! i can tell you put a lot of thought into it. Curious; is this for business or home?

31

Ok, I now got OPNsense (OPNs) to pick up an IP from 172.16.1.x.; after I reconfigured the Zyxel as you suggested; and after I blew away the interface configurations on OPNs and assigned the physical interface em0 as WAN. I havenā€™t tested anything else yet. That probably will have to wait until the weekend.

On the Zyxel
port 1 is Vlan 10 untagged; Vlan 1 tagged
port 2 is Vlan 1 untagged; Vlan 10 tagged

I wasnā€™t sure, if port 1 Vlan 1 tagged is correct?

My understanding of Vlans feels fragmented. I've re-read your instructions and explanations above and they make sense and I watched some more videos; but I donā€™t feel any more confident. Maybe thatā€™s partly because I donā€™t find the Vlan portion of the Zyxel to be very intuitive. Although the device is sold and was a great recommendation.

The concept of all of this seems straightforward enough, but every next step I take seems like a brand new puzzle.

For example, Zyxel allows for more than one Vlan to be selected as untagged for one port; or maybe thatā€™s unique to the native Vlan 1; or its an idiosyncrasy of the GUI?

Under Port settings, should I leave everything for now as it is, in particular Accepted Type and Ingress Filtering, or will this come into play later? What port needs to be a Vlan trunk? I thought the port on the Zyxel that the OPNs is connected to should be a trunk and each LEDE? Now I am not so sure anymore. Argh.

Do I need to mess with Membership options such as Forbidden and Excluded for each port / Vlan?

The management Vlan confuses me as well because on the Zyxel it defaults to the native Vlan 1. Do I need to create a separate Vlan to securely manage my devices (OPNs, LEDE, etc.) with 802.1x as we discussed?

Mon Dieu! :scream:

I think I severely lack an understanding of what logical steps to take when designing the architecture of a Vlan topology and how that translates then into what settings to use. Thereā€™s a disconnect. I think my drafts are garbage. Itā€™s a mesh of physical and logical topology.

Do you know of any good resources that explain Vlan methodology or approach? Like short 10-15 minute graphics driven tutorials that just throw out a couple of simple examples, but emphasize the best practices approach, like first you need to think about this, then that, etc. What Iā€™ve found thus far is either overly complex or too simplistic. The Zyxel videos on this were utterly useless.

Sorry for these ramblings. Feel free to ignore me. Ha, ha...

32

Once you get the hang of VLANs, it is deciding on topology and numbering (including your IPv4 and IPv6 subnetting) that is the challenging part, not the implementation. I'm old school and just think about them as Ethernet cables when doing my planning.

For carrying VLANs over WiFi, you'll either need each on their own SSID, or a tunnel. There is a good write-up of using GRE at Vlan trunk over wlan Be aware that busybox doesn't implement ip tunnel as configured in at least the TP-Link Archer C7 images and a custom build is required. There isn't an easy way to replace busybox on a running system https://bugs.lede-project.org/index.php?do=details&task_id=1380 but it can be done very carefully by extracting the self-build package yourself.

I consider every WiFi device to be untrusted. For a WiFi-connected device to get access to resources on any of my internal nets, it needs to be via a secure and authenticated path; ssh or VPN.

I don't know the workings of your switch, but on the Cisco SG300-series, you can require all packets coming in to be VLAN tagged on a port-by-port basis. Combined with the VLANs associated with the port, this means that you have restricted to only the VLANs you intend to be accessed through the port (or switch-to-switch trunk, for that matter).

It's a home network and it's not all that crazy to work through. I start by thinking about resource type (UPS/power management, switches, access points, service hosts, end-users, printers,...), what services they need (DHCP, DNS, NTP,...), and how and from where they should be accessed (management interfaces and service interfaces both).

On implementing VLANs on a switch, finding the screen or CLI command to give you the information you need can be challenging. For any connected device that is VLAN-aware, you generally set that VLAN as tagged on the port. If a connected device is sending untagged packets to the switch and expecting the same in return, you need to set a PVID (default VLAN) for the port, and generally allow untagged packets for the PVID on that port. When I know the remote device only sends VLAN-tagged packets, I block untagged ingress and egress from the switch.

Maybe some ASCII art will help?
Switch port set for PVID of 100
Configured VLANS on switch are 100, 101, 102, 103

There is physically only one "wire" but, in my head, I sometimes think of them as individual wires when reasoning out how things will work.

"Dumb" remote device

  • VLAN 100 is untagged
  • VLANs 101-103 not enabled
    Any untagged packets on the wire become VLAN 100 internally
    All tagged packets are rejected (assuming you have set it that way for the port)
VLAN 100 internal <-> port <-> (untagged) on wire <-> printer
VLAN 101 internal  X  port <-  VLAN 101   on wire <-  rogue
VLAN 102 internal  X  port <-  VLAN 102   on wire <-  rogue
VLAN 103 internal  X  port <-  VLAN 103   on wire <-  rogue

"Smart" remote device, only uses VLAN 101 and 102

  • VLANs 100, 103 is not enabled
  • VLANs 101-102 tagged
    Edit: Even when only using tagged, you generally need to set the PVID. Some switches require that it be a VLAN configured for the port. 
VLAN 100 internal  X port <-   (untagged) on wire <-  rogue
VLAN 101 internal <-> port <-> VLAN 101   on wire <-> remote server
VLAN 102 internal <-> port <-> VLAN 102   on wire <-> remote server
VLAN 103 internal  X port <-   VLAN 103   on wire <-  rogue

"Mixed" remote device, untagged packets, as well as uses VLAN 101 and 102

  • VLAN 100 is untagged
  • VLANs 101-102 tagged
  • VLAN 103 is not enabled
VLAN 100 internal <-> port <-> (untagged) on wire <-> remote server
VLAN 101 internal <-> port <-> VLAN 101   on wire <-> remote server
VLAN 102 internal <-> port <-> VLAN 102   on wire <-> remote server
VLAN 103 internal  X  port <-  VLAN 103   on wire <-  rogue

While I've shown the "rogue" on a non-operational VLAN, there is nothing to prevent them from snooping the wire and seeing that VLAN 101 and VLAN 102 are "valid" VLANs and configuring their own interface for one or both of those VLANs. This is where something like 802.1X port authentication comes in, if you really think you need it. For many people, password/keyed authentication to their devices is the weak link; a compromise of a "trusted" device is much more likely than an untrusted device appearing on a trusted VLAN.

For switch-to-switch trunking, I set things up with everything on VLANs, no untagged packets. In general, the more explicit you are with things, the less chances for surprise.

Speaking of surprises, make sure you know how to factory-reset your switch before you think about changing the management VLAN from its default (which you probably should do). I've messed up more than once on that myself!

1 Like
33

To get VLANs, remember how they are implemented. Each packet is either an untagged packet, or a "tagged" packet with a particular VLAN number. They travel down the wire and are received by devices that understand VLANs and interpreted as if they were coming on a "different wire"

So, you can set a PVID for a particular port, and then when an untagged frame is received it will be "internally tagged" inside the switch.

If a tagged frame is received, the switch understands the tag and it becomes a part of that particular VLAN, so it can only go out on ports that belong to that VLAN...

From a security perspective you set up each port so that it only accepts the minimum set of VLANs that are needed to perform the tasks your network requires.

So for example, coming from your Tomato/modem you'll be receiving untagged frames at the switch, you want them to go into vlan 10, so you set up PVID 10 for that port, and this port ONLY belongs to vlan 10.

Trunking is only needed when you want unknown vlans to pass through your switch. In your situation I don't think it will ever be needed to have trunking set up.

1 Like
34

Ha, ha... I can definitely appreciate that.

Actually that's what got me confused thinking of them as cables, whereas I believe it might work better for me to first think about the logical layout and how the data should flow and then translate that to the physical topology. Listening to couple networking professionals vent their own frustrations, put things in perspective for me. Apparently terminology may vary between vendors and sometimes even between models by the same vendor (Cisco). So that was interesting.

I took a look at my docs last night. The Engenius should support VLANs in WDS Bridge mode, but I will still need to confirm.

So, if you have a trusted network, untrusted network and maybe a guest network, you require each user to connect to that subnet which is on its own VLAN SSID via OpenVPN for example? So each device needs to have a client installed. Don't you guests hate you? Just thinking out loud. :face_with_raised_eyebrow:

Yes, I think that's possible. Like I said, I think some of the terminology that some of the vendors use differs and somehow the Zyzel has the VLAN configuration laid out doesn't gel with me. That's probably just me. Probably just need some more hands-on time with this.

I think that was my first little epiphany. When I first started thinking about the layout, I went top to bottom (devices farthest away); but I think it much better the other way around when it comes to VLANs.

That makes sense. Once I get my network flow working, I will review each port and be as restrictive by ingress and egress as possible. Is that the right approach?

Thanks for this. It help clear up a few points.

Well, my switch comes with it and Daniel thinks 802.1x is at least supported one way to the LEDE devices I am going to use. So what the heck. Unless I've run completely out of patience by the end of this project. :laughing:

Not sure I follow?

So the uplink ports between each switch need to be marked as 'Trunk' in addition you need those same ports on each swith to be tagged for all VLANs that the uplink is supposed to carry?

Oh, yes! Very wise words indeed!

Thanks for your time helping me out with this. It's appreciated.

Cheers

35

No, at least in the Zyxel, their notion of "trunk" is that the switch will pass vlans set up by other switches that it doesn't know about. So for example you have Cisco switch somewhere with VLANs 99 100 101 coming out some port that then connects to your Zyxel port 1, and another Cisco switch with same vlans connected to port 2...

The zyxel is not configured to use 99 100 101 at all, so if you make port 1 and 2 a trunk, it will just happily accept any vlan tag there and pass it along back and forth between port 1 and 2...

In your scenario all the VLANs that are valid on your network should be ones that the Zyxel knows about and has explicitly enabled on each port that they can appear on. So nothing should be "trunk"

1 Like
36

Thanks man!

37

Ok, so trunks behave the same in a VLAN scenario as they do when connecting (trunk, uplink) two physical switches with no VLANs. I thinks some of the fog is lifting. :grinning:

Thanks Daniel!

38

Specific advice to get you going:

Keep one port "open" and accepting all VLANs and accepting untagged packets that go direct into the management vlan, while you configure the other ports

To see which VLANs your ports should accept, just look at your diagram. Each color is a VLAN. So what I see is:

Tomato: red, vlan 10 untagged with ingress filtering only untagged frames allowed

LEDE box with blue/green at bottom: only blue and green vlans, make them both tagged on that port. Make the LEDE box tag its frames.

OPNSense: needs to communicate to all the vlans so its port should belong to red, green, blue, purple and orange, make them all tagged

DMZ: you're connecting to a "dumb" switch here, easiest is to make this port belong to only orange vlan untagged

LEDE Box at the top: make its port belong to purple and blue vlans tagged

on each port in the zyxel you'll enable ingress filtering so that it can only accept VLANs that the port belongs to.

1 Like
39

Thanks for these specifics. I will be testing some of this out with the equipment that I have on hand this weekend.

40

No, guests log in to the guest WiFi with access to the outside world -- they have no need to access anything else.

If you have an unauthorized user or application on a "trusted" system, your firewalls, network separation, 802.1X authentication, ... are not going to help you. Personally, I'd spend significant time there before going on to "niceties" like 802.1X.

It depends on your switch's terminology. I have to look at the docs every time I adjust the configuration of my Cicso switches. I never remember clearly the difference between, as I recall, "access", "general", and "trunk" including which options each allow.

:+1::relaxed:

I've perhaps assumed that you have wireshark installed. If not, get it up and going! It's a lot easier to "see" the traffic than with tcpdump