DSA VLAN default settings: is it segregated network?

grrr2 · June 29, 2021, 7:57am

Hi,

With the introduction of DSA network configuration I realized that something is not working as should out of the box. Most probably I miss something silly.

(1) VLANs are supposed to be segregated, isolated virtual networks without any automatic routes between them, i.e. members of VLAN1 should not be able to access VLAN2 members by default. They are different L2 networks, normally adding L3 routing could connect them. Am I correct?

(2) Additionally there is iptables firewall with zone feature, creating two interfaces each member of their own VLAN and put them to different zones, I would assume an additional layer of segregation, i.e. zone_VLAN1 could not access zone_VLAN2 without explicit config. Am I correct?

(3) Playing with 21.02-xx though it seems no matter what kind of VLAN method I use (bridge or vlan_filtering) for some reason the zoned, VLAN'ed networks have visibility on each other. What is missing, how can I make sure two VLANs are really segregated?

To put into context let's assume the following physical network:

Two router-switch combos, namely R1 and R2 each with 1x WAN (eth0) and 4x LAN (eth1-eth4) ports. R1 is connected to ISP via WAN port, R1 and R2 is connected via eth3 port on each router. Want to use a network called 'lan' and other called 'guest' to segregated trusted vs untrusted end points. (eth4 is used for console access thus can be ignored.)

Therefore I did the following (config is same on both R1/R2 unless otherwise noted):

eth1-eth3 ports are merged to a bridge called sw0
with vlan_filtering on, created VLAN1 (vid=1) and VLAN2 (vid=2). this results two new 802.1q devices, namely sw0.1 and sw0.2 with respective Vlan ids assgined
eth1 is untagged in VLAN1
eth2 is untagged in VLAN2
eth3 is tagged for VLAN1 & 2
R1 acts DNS, DHCP server and gateway to WAN
created interface called 'LAN' from sw0.1, ip address 10.0.0.1/24 (R1) and 10.0.0.2/24 (R2). On R2 gateway is pointing to 10.0.0.1 (R1). In zone 'lan'.
created interface called 'GUEST from sw0.2, ip address 20.0.0.1/24 (R1) and 20.0.0.2/24 (R2). On R2 gateway is pointing to 20.0.0.1 (R1). In zone 'guest'.

Strangely though from R2 guest interface pinging R1's lan interface is working:

root@r2:~# ping -c2 -I 20.0.0.2 10.0.0.1
PING 10.0.0.1 (10.0.0.1) from 20.0.0.2: 56 data bytes
64 bytes from 10.0.0.1: seq=0 ttl=64 time=1.011 ms
64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.309 ms

--- 10.0.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.309/0.660/1.011 ms

root@r2:~# tcpdump -i eth3 -e arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
07:30:12.459129 xx:xx:xx:xx:6a:80 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.1 tell 20.0.0.2, length 28
07:30:12.459471 xx:xx:xx:xx:8d:e7 (oui Unknown) > xx:xx:xx:xx:6a:80 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Reply 10.0.0.1 is-at xx:xx:xx:xx:8d:e7 (oui Unknown), length 46
07:30:12.459745 xx:xx:xx:xx:8d:e7 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 64: vlan 2, p 0, ethertype ARP, Request who-has 20.0.0.2 tell 10.0.0.1, length 46
07:30:12.459825 xx:xx:xx:xx:6a:80 (oui Unknown) > xx:xx:xx:xx:8d:e7 (oui Unknown), ethertype 802.1Q (0x8100), length 46: vlan 2, p 0, ethertype ARP, Reply 20.0.0.2 is-at xx:xx:xx:xx:6a:80 (oui Unknown), length 28

I don't think this is expected, or is it?

Any idea what I miss please? Is it a problem with my config? I have false expectation regarding how VLANs should work?

Thanks.

trendy · June 29, 2021, 8:17am

yes

yes, as long as the default policy for forward is drop/reject.

You cannot have 2 active gateways the way you have. Only one gateway has precedence and that one is used for traffic coming from lan or guest vlan. To really achieve your goal, you need to do Policy Based Routing and you have 3 options:

mwan3 package
pbr package
a set of rules/routes for each internet connection.

grrr2 · June 29, 2021, 8:25am

Hi,

I don't have two WAN gateways. The only WAN gw is on R1. The default gateway on R2 is set to pointing to R1 basically (per VLAN).

trendy · June 29, 2021, 9:29am

I understood the same and my answer still applies. You cannot have 2 gateways on R2 per vlan as you have them. You still need sourced based routing to direct packets coming from the guest vlan to use the guest gateway.

grrr2 · June 29, 2021, 12:48pm

Sorry but I don't really see why the gateway config causing VLAN leakage ... so i tested what you suggested:

config interface 'lan'
        option proto 'static'
        option device 'sw0.1'
        option ipaddr '10.0.0.2'
        option netmask '255.255.255.0'
        option defaultroute '0'

config interface 'guest'
        option proto 'static'
        option device 'sw0.2'
        option ipaddr '20.0.0.2'
        option netmask '255.255.255.0'
        option defaultroute '0'

root@r2:~# ip route
10.0.0.0/24 dev sw0.1 scope link  src 10.0.0.2
20.0.0.0/24 dev sw0.2 scope link  src 20.0.0.2
192.168.1.0/24 dev eth4 scope link  src 192.168.1.134
root@r2:~# ping -c2 -I 20.0.0.2 10.0.0.1
PING 10.0.0.1 (10.0.0.1) from 20.0.0.2: 56 data bytes
64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.276 ms
64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.203 ms

--- 10.0.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.203/0.239/0.276 ms

... and the exact same happens.

As said I don't see how gateway is related to the problem because I think (could be wrong):
Default gateway (or probably better call default router) is a last resort direction: if don't know where to send than send to def. gw. Which means it is not L2 but L3 (as it is an IP address of a host) but VLAN is in L2.

Basically I have two extended broadcast domains over two physical devices which are connected via direct cable.

With the simple routing table (after removing gateway IP as you suggested) traffic flow is slightly different:

root@r2:~# tcpdump -e -i eth3 arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
12:13:27.451693 xx:xx:xx:xx:6a:80 (oui Unknown) > xx:xx:xx:xx:8d:e7 (oui Unknown), ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 28
12:13:27.451942 xx:xx:xx:xx:8d:e7 (oui Unknown) > xx:xx:xx:xx:6a:80 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Reply 10.0.0.1 is-at xx:xx:xx:xx:8d:e7 (oui Unknown), length 46
12:13:27.531511 xx:xx:xx:xx:8d:e7 (oui Unknown) > xx:xx:xx:xx:6a:80 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 2, p 0, ethertype ARP, Request who-has 20.0.0.2 tell 20.0.0.1, length 46
12:13:27.531567 xx:xx:xx:xx:6a:80 (oui Unknown) > xx:xx:xx:xx:8d:e7 (oui Unknown), ethertype 802.1Q (0x8100), length 46: vlan 2, p 0, ethertype ARP, Reply 20.0.0.2 is-at xx:xx:xx:xx:6a:80 (oui Unknown), length 28

It looks the problem is more due to default config, i.e. reverse path filtering is disabled by default. When switch it to 'strict':

root@r2:~# ping -c2 -I 20.0.0.2 10.0.0.1
PING 10.0.0.1 (10.0.0.1) from 20.0.0.2: 56 data bytes
^C
--- 10.0.0.1 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

root@r2:~# tcpdump -i eth3 -e -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
12:29:30.896774 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 20.0.0.2 > 10.0.0.1: ICMP echo request, id 8266, seq 0, length 64
12:29:31.899308 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 20.0.0.2 > 10.0.0.1: ICMP echo request, id 8266, seq 1, length 64
12:29:35.900433 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 28
12:29:35.900652 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Reply 10.0.0.1 is-at xx:xx:xx:xx:8d:e7, length 46

root@r1:~# tcpdump -e -i eth3 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
12:29:30.896918 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 20.0.0.2 > 10.0.0.1: ICMP echo request, id 8266, seq 0, length 64
12:29:31.899397 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 20.0.0.2 > 10.0.0.1: ICMP echo request, id 8266, seq 1, length 64
12:29:35.900536 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 46
12:29:35.900578 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Reply 10.0.0.1 is-at xx:xx:xx:xx:8d:e7, length 28

As expected traffic from VLAN2 (via interface IP 20.0.0.2) is not reaching 10.0.0.1 in VLAN1.

And of course it is working as expected within VLAN1:


root@r2:~# ping -c2 -I 10.0.0.2 10.0.0.1
PING 10.0.0.1 (10.0.0.1) from 10.0.0.2: 56 data bytes
64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.320 ms
64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.412 ms

--- 10.0.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.320/0.366/0.412 ms

root@r2:~# tcpdump -i eth3 -e -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
12:33:35.557237 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.2 > 10.0.0.1: ICMP echo request, id 8766, seq 0, length 64
12:33:35.557484 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 8766, seq 0, length 64
12:33:36.558264 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.2 > 10.0.0.1: ICMP echo request, id 8766, seq 1, length 64
12:33:36.558576 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 8766, seq 1, length 64
12:33:40.634367 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 28
12:33:40.634574 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Reply 10.0.0.1 is-at xx:xx:xx:xx:8d:e7, length 46
12:33:40.717806 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 46
12:33:40.717833 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Reply 10.0.0.2 is-at xx:xx:xx:xx:6a:80, length 28

root@r1:~# tcpdump -e -i eth3 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
12:33:35.557983 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.2 > 10.0.0.1: ICMP echo request, id 8766, seq 0, length 64
12:33:35.558052 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 8766, seq 0, length 64
12:33:36.559071 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.2 > 10.0.0.1: ICMP echo request, id 8766, seq 1, length 64
12:33:36.559131 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 102: vlan 1, p 0, ethertype IPv4, 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 8766, seq 1, length 64
12:33:40.635096 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 46
12:33:40.635134 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Reply 10.0.0.1 is-at xx:xx:xx:xx:8d:e7, length 28
12:33:40.718316 xx:xx:xx:xx:8d:e7 > xx:xx:xx:xx:6a:80, ethertype 802.1Q (0x8100), length 46: vlan 1, p 0, ethertype ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 28
12:33:40.718632 xx:xx:xx:xx:6a:80 > xx:xx:xx:xx:8d:e7, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Reply 10.0.0.2 is-at xx:xx:xx:xx:6a:80, length 46

So is it possible that default config is not correct?

Or let me ask differently:
If need to use two (or more) Owrt enabled devices for example due to wifi reception problems and want to segregate trusted and untrusted end devices, but want to keep management centralized (i.e. only one device to provide WAN, DNS, DHCP etc services, the other Owrt devices to work as dumb AP/switch) is VLAN a solution or something else? If creating two VLANs for un/trusted network and assigning different iptables zones is not enough, what else may be required to have a fully isolated two networks? If VLAN is not the right approach than what could be?

mk24 · June 29, 2021, 1:12pm

When the router holds an IP address on two networks the kernel has L3 routes between them, and a bare kernel would freely route between them. Thus the firewall is an integral part which restricts such forwarding. For a highly restricted guest network this is typically done by blocking all traffic from guest to any private IP, which would include the other LAN. Guests are only allowed to use destination addresses that are on the Internet. Note that this does not prevent guest traffic from forwarding to the Internet through your gateways with private IPs, as long as the final destination is on the Internet.

Of course you need a couple of other rules so that guests can use your router's DHCP and DNS services.

This can all be done at the fw3 /etc/config/firewall level, which abstracts you from dealing with iptables rules directly.

trendy · June 29, 2021, 1:27pm

I don't see the issue with default configuration. What you are trying to do there is to force asymmetric routing. You send the ping via a link that normally would not be used and reply is supposed to come back over a different link. Apart from that, if you have INPUT policy accept on R1 lan zone, then everything will be accepted, regardless of the source.

VLAN is a solution, but not the way you implemented it.
You'd need to have an IP address only on the LAN interface of R2, without wan links. The GUEST interface would be unmanaged, that is without IP. Then you trunk the lan and guest vlan over the uplink to R1, where you enforce the firewall policy (lan->wan , guest->wan)

grrr2 · June 29, 2021, 2:52pm

What you are trying to do there is to force asymmetric routing. You send the ping via a link that normally would not be used and reply is supposed to come back over a different link.

Where you see different links please? To be on the same page: R2 - no WAN. The two VLANs are extended between R1 and R2 via eth3 tagged. I appreciate if you can correct my thinking here:

		wan
		|
R1 		eth0 	eth1 	eth2 	eth3  --+
VLAN1	-		u		-		t		|
VLAN2	-		-		u		t		|
										|
R2 		eth0 	eth1 	eth2 	eth3  --+
VLAN1	-		u		-		t
VLAN2	-		-		u		t
		|
		discon
		nected

I think this setup would mean that I have two networks (VLAN1/2) on both devices and they are actually connected via eth3 to compose two extended networks: so an end device on R1:eth1 and device on R2:eth1 are in one broadcast domain (VLAN1); R1:eth2 and R2:eth2 are in a different broadcast domain (VLAN2). I assumed this setup would mean that L2 frames would go untagged within R1 for example and egress tagged via R1:eth3 which is received on R2:eth3 and due to vlan_filtering on R2 tag is removed and put frames to correct VLAN on R2. If this thinking is wrong then my bad sorry.

As @mk24 explained that in kernel L3 routing exists so GUEST interface should be unmanaged as suggested by both of you.

Before we can close, would you please help me understand what do I think wrong with all these VLAN assumptions of mine? E.g. ethX.Y is a VLAN tagging notation meaning outgoing traffic is tagged? But does not say anything about incoming traffic either it is tagged or untagged it will be accepted? Hence the need for vlan_filtering?

trendy · June 29, 2021, 4:02pm

vlan1 and vlan2. You are sending the ping over vlan1, according to the destination, however the response will come back from vlan2, again according to the destination.

No, this is correct thinking.

Strictly speaking, if you had just an interface, then yes that would be true. However here you have a switch, and its configuration is slightly different.

It depends. If you allow untagged it will be accepted. If not it will be discarded. Also if it comes from a wrong vlan it will be discarded.

I think you are overcomplicating it. What you are trying to accomplish is simple. Trunk on eth3, one management interface for R2, assign ports to each vlan, manage everything on R1.

grrr2 · June 29, 2021, 6:09pm

vlan1 and vlan2. You are sending the ping over vlan1, according to the destination, however the response will come back from vlan2, again according to the destination.

Ok I see that but keep in my mind I did not want this, I expected actually this would fail - but learned it is not the case.

Thanks very much for guidance and explanations!

trendy · June 29, 2021, 6:36pm

It's not that hardened by default to mitigate such scenarios from inside the lan. Lan is generally accepted/trusted. On the other hand of course this is not a default configuration that you are experimenting with.

You're welcome!

system · July 9, 2021, 6:37pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.