Help setting up LAN and two VLAN's over same adapter (mvebu + DSA)

cybrnook · June 16, 2021, 4:31am

I have a need to setup some VLAN's, but would need a little guidance to do it. My router is a ClearFog Base unit (mvebu) with which I am using it with Dual WAN. So, for the three LAN ports it has ( 2 x Gbe and 1 x SFP), both RJ45 Gbe ports represent WAN and WANB, where the SFP port through a transceiver to RJ45 is my LAN connection. I am currently on the 21.02 branch and am current as of this morning.

I ultimately would like to keep my primary LAN network 192.168.1.1/24 available with DNSMASQ and the backing DHCP server.
At the same time, I would now like to add:
GUESTVLAN - 192.168.2.1/24 (With DNSMASQ as backing DHCP)
IOTVLAN - 192.168.3.1/24 (With DNSMASQ as backing DHCP)

I would like all three to run over the main LAN link I have now. From that point, I can configure my switches and AP's directly to use VLAN tags 10 and 20 to route traffic accordingly, creating layers of separation.

I followed some posts earlier on and think I have a basic setup today running for my bridge switch:

config interface 'loopback'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'
	option device 'lo'

config device
	option type 'bridge'
	option name 'switch0'
	list ports 'eth2'

config bridge-vlan
	option device 'switch0'
	option vlan '1'
	list ports 'eth2'

config interface 'lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.1'

config interface 'wan'
	option proto 'dhcp'
	option delegate '0'
	option peerdns '0'
	list dns '127.0.0.1'
	option metric '10'
	option device 'eth0'

config interface 'wanb'
	option proto 'dhcp'
	option delegate '0'
	option peerdns '0'
	list dns '127.0.0.1'
	option metric '20'
	option device 'eth1'

config interface 'wginterface'
	option proto 'wireguard'
	option delegate '0'
	option private_key 'xxxxx'
	option listen_port '51820'
	list addresses 'xxxxx'
	option metric '30'
	option force_link '1'

config wireguard_wginterface
	option description 'xxxxx'
	option public_key 'xxxxx'
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'xxxxx'
	option endpoint_port '51820'
	option persistent_keepalive '25'

config device
	option name 'switch0.1'
	option ipv6 '0'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'eth2'
	option ipv6 '0'

config device
	option name 'wginterface'
	option ipv6 '0'

Would someone be willing to lend a hand to show me how to do this correctly? @anomeome have any spare time?

cybrnook · June 16, 2021, 7:23pm

I feel like I have to be close and am missing something stupid silly. I got the VLAN interfaces created:

I have firewall zones created:

I checked the box in each interface to setup DHCP (I have dnsmasq full installed):

In my switches I have the VLAN's created:

In my unifi controller and AP, I created the VLAN's:

But the moment I add one of the SSID's in my unifi AP to one of the VLAN groups, no traffic flows and no IP gets assigned out:

Any help is much appreciated.

cybrnook · June 16, 2021, 7:25pm

Here is /etc/config/network:

config interface 'loopback'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'
	option device 'lo'

config device
	option type 'bridge'
	option name 'switch0'
	list ports 'eth2'

config bridge-vlan
	option device 'switch0'
	option vlan '1'
	list ports 'eth2'

config bridge-vlan
	option device 'switch0'
	option vlan '2'
	list ports 'eth2'

config bridge-vlan
	option device 'switch0'
	option vlan '3'
	list ports 'eth2'

config interface 'lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.1'
        option ipaddr '192.168.1.1'

config interface 'guest'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.2'
	option ipaddr '192.168.2.1'

config interface 'iot'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.3'
	option ipaddr '192.168.3.1'

config interface 'wan'
	option proto 'dhcp'
	option delegate '0'
	option peerdns '0'
	list dns '127.0.0.1'
	option metric '10'
	option device 'eth0'

config interface 'wanb'
	option proto 'dhcp'
	option delegate '0'
	option peerdns '0'
	list dns '127.0.0.1'
	option metric '20'
	option device 'eth1'

config interface 'wginterface'
	option proto 'wireguard'
	option delegate '0'
	option private_key 'xxxxx'
	option listen_port '51820'
	list addresses 'xxxxx'
	option metric '30'
	option force_link '1'

config wireguard_wginterface
	option description 'xxxxx'
	option public_key 'xxxxx'
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'xxxxx'
	option endpoint_port '51820'
	option persistent_keepalive '25'

config device
	option name 'switch0.1'
	option ipv6 '0'

config device
	option name 'switch0.2'
	option ipv6 '0'

config device
	option name 'switch0.3'
	option ipv6 '0'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'eth2'
	option ipv6 '0'

config device
	option name 'wginterface'
	option ipv6 '0'

And /etc/config/dhcp:

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option localservice '1'
	option ednspacket_max '1232'
	list server '127.0.0.1#5453'
	option noresolv '1'
	option dnsseccheckunsigned '0'
	option rebind_protection '1'
	list rebind_domain 'plex.direct'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	list ra_flags 'none'

config dhcp 'guest'
	option interface 'guest'
	option start '100'
	option limit '150'
	option leasetime '12h'
	list ra_flags 'none'

config dhcp 'iot'
	option interface 'iot'
	option start '100'
	option limit '150'
	option leasetime '12h'
	list ra_flags 'none'

anomeome · June 16, 2021, 7:35pm

So, no IP to the AP, or no egress from the AP.

cybrnook · June 16, 2021, 7:45pm

The AP itself has an IP assigned to it from my "LAN" network, which is what my primary network is off of (192.168.1.1/24).

But, I have 3 x SSID's on my AP:
Wi-Fi (192.168.1.1/24)
Wi-Fi (Guest) (192.168.2.1/24) - Should be over VLAN 2
Wi-Fi (IOT) (192.168.3.1/24) - Should be over VLAN 3

I would expect when I assign a device to the VLAN based SSID, dnsmasq on OWRT would assign out an IP from the .2 or .3 pool, am I thinking incorrectly?

anomeome · June 16, 2021, 9:34pm

So if I understand correctly, you want:

trunk port w VLAN 2, 3, and native VLAN 1, over eth2

as the unifi SW appears to be tagged 2 and 3.

cybrnook · June 16, 2021, 9:38pm

Exactly buddy!

My router unit itself only has 3 x NIC ports, 2 x standard RJ45 gigabit and 1 x SFP. Both RJ45's are being used for my WAN and WANB connections (Since I use dual WAN at home). So, the only remaining port that my unit has is the SFP, so I need that to serve everything on the LAN side, which would be Native VLAN1 for LAN and VLAN2 for GUEST with VLAN3 for IOT, all over eth2.

VLAN2 and 3 will be used strictly for the moment as Wi-Fi VLAN's. Meaning the existence of them for the moment will be limited to just wireless clients connecting to a respective SSID (Wi-Fi (Guest) for VLAN2 and Wi-Fi (IOT) for VLAN3.

anomeome · June 16, 2021, 9:44pm

Big salty grains here, but something like:

network

config bridge-vlan
	option device 'switch0'
	option vlan '1'
	list ports 'eth2:u*'
	option alias 'lan'

config interface 'lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '192.168.1.1'
	option device 'switch0.lan'

config bridge-vlan
	option device 'switch0'
	option vlan '2'
	list ports 'eth2:t'
	option alias 'guest'

config interface 'guest'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '192.168.2.1'
	option device 'switch0.guest'

config bridge-vlan
	option device 'switch0'
	option vlan '3'
	list ports 'eth2:t'
	option alias 'iot'

config interface 'iot'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '192.168.3.1'
	option device 'switch0.iot'

cybrnook · June 16, 2021, 10:12pm

Didn't work, at least not out of the box. Some strangeness I see and I am not sure if it's related to all the work that @jow is doing with DSA in Luci, but looking at the GUI it's showing as VLAN1 as tagged and primary, while in the network file itself it's untagged and primary. I believe that's red herring, but maybe worth bringing up?

Also, @jow , there is a typo on the Bridge VLAN filtering page, notice the "Enable VLAN filterering"

anomeome · June 16, 2021, 10:15pm

Not sure where 21.x is at with commits, what about:

bridge vlan
brctl show

cybrnook · June 16, 2021, 10:16pm

Bridge seems not to be included.

root@OpenWrt:/etc/config# bridge vlan
-ash: bridge: not found
root@OpenWrt:/etc/config# brctl show
bridge name	bridge id		STP enabled	interfaces
switch0		7fff.d263b41a53d6	no		eth2
root@OpenWrt:/etc/config#

EDIT: One thing I am doing in the meantime, I am going through my switches once more and changing all ports to "tagged" for VLAN traffic over 2 and 3. They were set to "untagged" prior.

anomeome · June 16, 2021, 10:25pm

You will probably want to add that to your build:

Summary

root@mamba:/# bridge vlan
port              vlan-id  
lan4              10 PVID Egress Untagged
lan3              10 PVID Egress Untagged
lan2              20
                  30
                  4094 PVID
lan1              20
                  30 PVID Egress Untagged
itch0             10
                  20
                  30
                  4094
wlan0             10 PVID Egress Untagged
wiot0             30 PVID Egress Untagged
wlan1             10 PVID Egress Untagged
wiot1             30 PVID Egress Untagged

cybrnook · June 16, 2021, 10:27pm

Adding it now and will do a recompile and we can pickup from there. Thanks @anomeome , you've always been good to me

cybrnook · June 16, 2021, 10:44pm

Okay, so I'm a dumbass I think the fact that my switches had the ports set to VLAN 2 and 3 as "untagged" prevented the tagged traffic from flowing. Once I changed them to "tagged" ports for 2 and 3, magic happened.

Here is my config at the moment, which seems to be working as expected, dhcp and all:

config device
	option type 'bridge'
	option name 'switch0'
	list ports 'eth2'

config bridge-vlan
	option device 'switch0'
	option vlan '1'
	list ports 'eth2:u*'

config interface 'lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.1'
	option ipaddr '192.168.1.1'

config bridge-vlan
	option device 'switch0'
	option vlan '2'
	list ports 'eth2:t'

config interface 'guest'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.2'
	option ipaddr '192.168.2.1'

config bridge-vlan
	option device 'switch0'
	option vlan '3'
	list ports 'eth2:t'

config interface 'iot'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.3'
	option ipaddr '192.168.3.1'

I am not sure if eth2 on VLAN1 really needs to be called out as untagged and primary :u* but I will leave it for now. Thanks @anomeome

Also @jow , I am not sure if it's expected or not, but I also noticed something strange when it comes to the VLAN's we added by hand. What I mean is that if I go into /etc/config/network and add by hand:

config bridge-vlan
	option device 'switch0'
	option vlan '3'
	list ports 'eth2:t'

config interface 'iot'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option device 'switch0.3'
	option ipaddr '192.168.3.1''

Then issue /etc/init.d/network restart, I can go into luci and look at Network->Interfaces->Devices and my VLAN(s) appear as expected:

BUT, if I click the "configure" button next to one of the newly created (greyed out) VLAN's and make 0 changes BUT I do click "Save", then 2 pending changes appear:

At that point my greyed out VLAN definition becomes a defined "Device" in the devices list, no longer called out as "VLAN (802.1q)". I am not sure if that is expected behavior or not?

cybrnook · June 16, 2021, 11:27pm

@anomeome what package contains the bridge command? I added "bridge" under "Base System" but that wasn't it.

anomeome · June 16, 2021, 11:29pm

Search of ip-bridge should find ti.

cybrnook · June 17, 2021, 12:06am

root@OpenWrt:~# bridge vlan
port              vlan-id  
eth2              1 PVID Egress Untagged
                  2
                  3
switch0           1
                  2
                  3

jow · June 17, 2021, 5:02am

Kind of expected, not a reason of concern at least and no effect on the underlying configuration.

cybrnook · June 17, 2021, 1:45pm

@jow did you see these two items as well? (the replied to post) There is a typo on Bridge VLAN Filtering page as well on the same page LUCI is not properly showing the flags set for a given port.

jow · June 17, 2021, 2:21pm

Will fix the former, could not reproduce the latter this morning.