IOT isolation with vlans in multi-AP environment

shorted.neuron · February 16, 2021, 4:50am

I've been using openwrt for years on a single router setup. Recently i moved into a much larger place, with lots of wifi-unfriendly materials. So got some new little gl.inet b1300 routers, and have them set up nicely like this:

This is working great... the main router talks to the ISP and handles DHCP for the whole network. The router and APs all connect to an ethernet switch, and have static IPs all on the same network. They all have both 2.4Ghz and 5GHz radios advertising the same SSID, but different channels for each. 802.11r fast roaming is turned on and everything is working great. This is my "trusted" network.

Now I am trying to figure out how to layer an untrusted IOT network on top of it. I have in particular a machine that manages video cameras. It has ethernet interfaces which need to talk to cameras hooked to IOT wifi in a different part of the house.

I've read many other forum posts, and I understand adding another SSID, a subnet for it, and how firewall rules would work between them.
What I cannot get my head around is how to handle this in the backhaul layer. On my existing backhaul, this is untagged ethernet. What I had thought to do was create a "lowtrust" interface in each openwrt, and bridge that to the wireless sub-AP that has the new IOT SSID. This seems to work if i use DHCP at each individual AP.

Then i thought I would be able to tie these separate IOT wlans together by bridging them to a tagged vlan. Therein lies my troubles. It works only between openwrt2 and openwrt3. As soon as I configure the switch in openwrt1 (the "main router"), then it recycles the wan port and from there on is unable to DHCP on wan to get address from my ISP. its weird!

I think that the only problem is on the "main" router, so have attached its configs here:

/etc/config/wireless (same in working or broken state)

config wifi-device 'radio0'
        option type 'mac80211'
        option hwmode '11g'
        option path 'platform/soc/a000000.wifi'
        option htmode 'HT40'
        option channel '1'

config wifi-iface 'wifi2ghz'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option key '*****'
        option ssid 'trusted'
        option ft_over_ds '1'
        option mobility_domain 'beef'
        option ft_psk_generate_local '1'
        option ieee80211r '1'
        option disabled '0'
        option ieee80211w '1'
        option encryption 'psk2'

config wifi-device 'radio1'
        option type 'mac80211'
        option hwmode '11a'
        option path 'platform/soc/a800000.wifi'
        option disabled '0'
        option htmode 'VHT40'
        option channel '149'

config wifi-iface 'wifi5ghz'
        option device 'radio1'
        option network 'lan'
        option mode 'ap'
        option key '*****'
        option ssid 'trusted'
        option ft_over_ds '1'
        option mobility_domain 'beef'
        option ft_psk_generate_local '1'
        option ieee80211r '1'
        option disabled '0'
        option ieee80211w '1'
        option encryption 'psk2'

config wifi-iface 'wifilow2'
        option ssid 'lowtrust'
        option encryption 'psk2'
        option device 'radio0'
        option ieee80211r '1'
        option ft_over_ds '1'
        option key '********'
        option mode 'ap'
        option ft_psk_generate_local '1'
        option mobility_domain 'dead'
        option isolate '1'
        option network 'lowtrust'

/etc/config/network in "working" state:

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.9.1'
        option ifname 'eth0'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'
        option broadcast '1'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config interface 'lowtrust'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.13.1'
        option type 'bridge'
        option igmp_snooping '1'
        option ifname 'eth0.13'

I change it just a little bridge lowtrust to the switch, then the whole thing goes to shit and wan will not dhcp anymore. It does however work in that two untrusted clients can now ping each other (firewall lock down coming a bit later) across two different access points.
/etc/config/network which breaks wan:


config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.9.1'
        option ifname 'eth0.1'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'
        option broadcast '1'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option vid '1'
        option ports '0t 3 4'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option vid '13'
        option ports '0t 3t 4t'

config interface 'lowtrust'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.13.1'
        option type 'bridge'
        option igmp_snooping '1'
        option ifname 'eth0.13'

Have i got this all wrong? Can someone explain a better way to handle an isolated IOT or guest network across many access points that are set up this way?

shorted.neuron · February 16, 2021, 4:56am

BTW, both "openwrt2" and 3 in the diagram above (the plain APs) use different static private IPs and do not have a wan interface at all. In their case, the switch_vlan config blocks like this achieve the desired effect. Its just the 'main router" openwrt1 that has wan troubles after applying this config.

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 3 4'
        option vid '1'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '0t 3t 4t'
        option vid '13'

trendy · February 16, 2021, 10:41am

First of all, is the switch managed? If not, the result is undefined. Other unmanaged switches pass tagged frames, whiles others don't.
According to documentation the internal switch of the B1300 uses port 5 for wan, however in your configuration eth1 is used.
Are you running the official OpenWrt, or the modified stock GL inet?

mpa · February 16, 2021, 6:14pm

OpenWrt on the IPQ40xx SoC still has some problems with the VLAN configuration.
There are many threads about this issue here in this forum.

One workaround that was mentioned is to keep VLAN 1 and VLAN 2 untouched, and create custom VLANs numbered 3 and up. Do not use LuCI to change the switch configuration, only the command line (uci or vi).

I am not sure whether the integrated switch supports mixing untagged and tagged VLANs on a port.
It might be worth trying to set all VLANs to tagged where a port carries multiple VLANs.

shorted.neuron · February 16, 2021, 7:06pm

Thanks guys. Switch is not managed, however I have verified using two systems talking to each other and dumping the wire that it does pass tagged frames unaltered, and doesnt seem bothered by mixed tagged and untagged.

I am using official openWRT 19.07.06. I was scared of the gl-inet flavor due to being based on ancient 15.05 stream.

Device naming gets really confusing. is eth0 the same thing as switch port 0? is eth1 the same thing as switchport 5? Thanks for the hint that IPQ40xx has trouble with VLANs. I will try the workaround with vlans numbered higher than 3. I guess i do have just enough ports that i can put vlan1 on one output port and all tagged vlans on a different port so that they are not mixed. That only leaves port0, which Luci calls "CPU" in the switch config screen. I've seen advice that it should always be set as tagged, and others not. I guess i dont understand the relationship between eth* devices and the internal switch. I'll try to read up on that. @trendy , which doc were you looking at exactly? https://openwrt.org/toh/hwdata/gl.inet/gl.inet_gl-b1300
or something under here? https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=04d3308b6248ef21a6f0bc3378b342906c2d2865
Wont be able to report back for a day or more anyway, because everyone is telecommuting over the thing today!

Cheers!

trendy · February 16, 2021, 9:34pm

This one.

shorted.neuron · February 16, 2021, 10:32pm

Gotcha, thanks @trendy ... i didn't spot it there the first time, but now see it at #switch_ports_for_vlans ... makes more sense now.

Care to eyeball my "fix" im going to try?

# first we gotta fix vlan1 aka LAN
uci set network.@switch_vlan[0].ports='0t 3'   # tag to cpu, untagged to port 3

# set up vlan13 on the switch (currently not configured at all on the switch)
# switch_vlan[1] needs to stay away from vlan "slot" 2 (which is what Luci wanted)
# so may as well make it match the 802.11q vlan tag aka vid 13
uci set network.@switch_vlan[1]=switch_vlan
uci set network.@switch_vlan[1].device='switch0'
uci set network.@switch_vlan[1].vlan=13   # get it off vlan "slot" 2 which is wan port
uci set network.@switch_vlan[1].vid='13'
uci set network.@switch_vlan[1].ports='0t 4t'   # tag to cpu, tagged to port 4

uci changes
# if all is good, 
# uci commit

trendy · February 16, 2021, 11:09pm

Should be fine. You can always make some small (or big) corrections by editing the config files directly.

shorted.neuron · February 16, 2021, 11:15pm

For others who may come across this thread, using "swconfig" does indeed show why allowing Luci to assign vlan2 to vid13 did not work for me. Heres "working" configuration with just my untagged vlan and WAN ports. Notice port 5 (aka eth1 aka the WAN port) as suggested earlier is hooked to vlan2:

# swconfig dev switch0 show | grep -A 5 VLAN
VLAN 1:
	vid: 1
	ports: 0t 3 4 
VLAN 2:
	vid: 2
	ports: 0t 5

trendy · February 17, 2021, 10:15am

If you want to use the wan port, remove port 5 from vlan 2 and add it under another vlan.

shorted.neuron · February 18, 2021, 2:17am

Thanks much for the help, i finally got it applied, and works like a charm when avoiding vlan2.
I'll mark this solved, but one last thought... is there any other good way to provide a guest or IOT LAN across several APs? I kinda feel like my config is a kludge.

For posterity, config that matters was /etc/config/network:

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdf6:308c:542d::/48'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ipaddr '192.168.9.1'
        list dns '208.67.222.222'
        list dns '208.67.220.220'
        option igmp_snooping '1'
        option ifname 'eth0'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'
        option broadcast '1'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option vid '1'
        option ports '0t 3 4'

config switch_vlan
        option device 'switch0'
        option vid '13'
        option vlan '13'
        option ports '0t 3t 4t'

config interface 'lowtrust'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.13.1'
        option type 'bridge'
        option igmp_snooping '1'
        option ifname 'eth0.13'

and of course swconfig now shows what everyone else expected:

# swconfig dev switch0 show | grep -A 3 VLAN
VLAN 1:
        vid: 1
        ports: 0t 3 4
VLAN 2:
        vid: 2
        ports: 0t 5
VLAN 13:
        vid: 13
        ports: 0t 3t 4t

trendy · February 18, 2021, 2:41pm

Move these under wan interface, where they are reachable from.

I don't see anything wrong with your setup. If you use dumbAPs from the other side you'll have to trunk all the vlans back to the main router. Alternatively, you could terminate the guest and iot on the APs and use lan to backhaul them, but then guests or iots won't be able to communicate internally.

shorted.neuron · February 20, 2021, 2:57am

Thanks much Trendy. The IOT can largely be isolated to their nearest AP, but the darn cameras all need to feed back to (anything that I control). So Mac and IP filters for those at each AP. Tagged looks the way to go, and I'll have correct 802.1q switches soon. Thanks for all the help!

system · March 2, 2021, 2:57am

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.