Weired untagged VLAN behavior : bug?

[maximushugus]

Hello. I have a problem with my configuration.
On my network I have :

• a routeur (a Pfsense server)
• a managed switch
• 2 OpenWrt router as "dumb AP" broadcasting the same WiFi (the same SSID and password)

I recently upgraded my 2 OpenWrt box (Xiaomi Mi Router 3G v1) from 19.07.6 to 21.02.1.
On each OpenWrt routeur, I had until the upgrade :

• a VLAN (it's VLAN 3) for my LAN, on untagged port 1 + port 2 + port "wan" + Wifi "lan"
• 2 VLANs (VLAN 50 and 99) for my IOT network and GUEST network, as tagged only on the "wan" port + Wifi "Guest" or Wifi "IOT"

My switch was configure with untagg VLAN 3 (for lan) and tagged VLAN 50 and VLAN 99 for the port my OpenWrt AP were connected.

After upgrading to 21.02.1 I had to reconfigure my VLAN with DSA.
So I first tried to put the same config, by creating a bridge with all of my ports on each OpenWrt AP, and creating untagged VLAN 3 for each ports and tagged VLAN 50 and tagged VLAN 99 on the "wan" port.
I have my Wifi LAN configured as part of the LAN (with VLAN 3) network.

Here is the problem with this configuration : when I connect a WiFi client on the first AP, it works, I get an IP and connection to LAN and internet. But as soon as I connect to the second AP, I cannot get an IP anymore on the second AP and neither ont the first.

Here's the tests I have done :

• First I thought it was a Fast transition (802.11r) problem so I disabled it, and manually disconnacted from the first AP by turning off Wifi and reconnecting to the second AP : the problem is still here, I cannot get IP from the 2nd AP, and when tring to reconnect to the first AP, I cannot get an IP adress anymore.

• I configured my OpenWrt AP with Egress untagged with and without "PVID" for the VLAN 3 on the "wan" port
  

  
  
  or
  
  
  
  But the same problem is still here

So I tried to make network captures to understand when I have the problem (after disconnecting from first AP from which I obtained an IP adress and connecting to the second or reconnecting to the first) and not getting one anymore :
Each test have been done with both configuration from screen capture as shown above with the same results

• From the Wifi client Wireshark capture I see the DHCP ask (broadcast) but not response
• From my Pfsense routeur I see the DHCP client asking for an IP and I see the response and the IP sent back
• A capture on the WiFi interface from my OpenWrt I see the DHCP ask (broadcast) but not response
• A capture from the br0 interface I see the DHCP ask (broadcast) but not response
• A capture from the br0.3 (VLAN 3 on the bridge) interface I see the DHCP ask (broadcast) but not response
• A capture from the "wan" interface (from the wan port on which the OpenWrt AP is connacted to the switch) I see the DHCP ask (broadcast) but not response
• I did a port miroring on my central switch from the port on which is connacted my OpenWrt AP to an other port and did a capture of this miroring : I see the DHCP client asking for an IP and I see the response and the IP sent back

So to conclude with both of the confiuration (screen capture) : DHCP asks goes from my client, to the AP, to the Pfsense router, there is a response made, that passes my main switch to the AP but the OpenWrt doesn't see (and doesn't give it to the client) the DHCP response.

But I saw on the ifconfig command that the number of packet on "eth0" interface (I believe this is the CPU swich of my OpenWrt AP on which every ports are connected) was much higher than on "br0"+"wifi".

So I did a packet capture one "eth0" : I saw tons of "broken" LLC packets.

The last thing I did was to put every VLAN on tagged on the OpenWrt AP :

And to tagg every VLAN on my central switch for the port on which the OpenWrt AP is connected.
With this config everything works as expected, I get IP and the amount of packet on "eth0"="br0"+"wifis".

So is this me that didn't understood the way DSA VLAN are supposed to work for untagged VLAN on OpenWrt or is there a bug for untagged VLAN on my OpenWrt routers ?

Thanks and sorry for long post

[dandik]

Just a simple question, did you delete the firewall from default config?
since that would mess with wan port .

And did you create an unmanaged layer 3 interface for vlan3 and set device to br0.3,
and set wifi to use that as the network ?

Make sure opwenwrt is not running a dhcp server , which might interfere .

Also if no dice with config just for tests create a managed vlan3 interface on br0.3 device
and dhcp server to see if wifi devices can get ip from openwrt.

[mk24]

Tagged and untagged on the same cable is not a best practice, and not well supported on consumer-grade hardware. Generally you should set up trunk cables to have all VLANs tagged like you did in the end.

Mixing tagged and untagged should only be considered when you have to connect to someone else's network that is set up that way and you can't change it.

[maximushugus]

@dandik

Just a simple question, did you delete the firewall from default config?
since that would mess with wan port .

There is still firewall on the OpenWrt AP but I deleted WAN rules, the firewall is juste here to keep IOT and GUEST client to be able to access gui or SSH the OpenWrt AP. Rules for LAN allow everything so I don't thinks this is the problem.

@dandik

And did you create an unmanaged layer 3 interface for vlan3 and set device to br0.3,
and set wifi to use that as the network ?

I didn't tried this one but as there is no DHCP and firewall allowing everything on LAN it should change anything (I think)

@dandik

Make sure opwenwrt is not running a dhcp server , which might interfere .

No DHCP server on both OpenWrt AP (the DHCP is on my Pfsense router)

@mk24

Tagged and untagged on the same cable is not a best practice, and not well supported on consumer-grade hardware. Generally you should set up trunk cables to have all VLANs tagged like you did in the end.

As you can see this is what I ended with, and with tagged port for every VLAN and no untagged VLAN (so basicly a trunk port) it works.
But what i found weird is that with 19.07 OpenWrt it used to work with untagged LAN VLAN and tagged for everything else.
I like the old config (with untagged and tagged) because sometime I unplugged the OpenWrt to plug a computer on the switch port and the computer was on LAN VLAN "out of the box"

Thanks for both of your answers

[dandik]

I never had problems running untagged vlan with tagged on trunks .

I would reset the device and start fresh , delete firewall.

and just get a lan port working on device with each vlan you have on the trunk
for testing sake.

[maximushugus]

I will try this and i'll do a follow up here. If any else has an idea on the problem ?

[maximushugus]

I tried to reset one of my OpenWrt AP and reconfigure it the way I want : untagged VLAN3 (LAN) an all ethernet port and tagged VLAN50 and VLAN99 on the "wan" port.
The problem is still here ...
I tried disabling firewall : the problem persist.

As I told before, when the problem occures (after reconnecting to WiFi) there is a lot of LCC packet on the "eth0" device (cpu switch from what I understand).
So I think this is the CPU that doesn't know what to do with those packets.
So I tried disabling hardware and software acceleration (MT7621 for my Xiaomi Mi Routeur 3G v1) but it didn't change anything...

Is this an OpenWrt bug ??

Thanks a lot

[jaromanda]

To be honest, I think it's time to show your /etc/config/network file, because all you've shown is the bridge vlan config, and there's much more to configuring your network than that

and perhaps the /etc/config/wireless and /etc/config/firewall

also, either a diagram or a detailed description of your physical network setup would help

[maximushugus]

Yes, here are the configs files :

For /etc/config/network :

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option packet_steering '1'
	option ula_prefix 'xxxx:xxxx:xxxx::/48'

config device
	option name 'br0'
	option type 'bridge'
	option stp '1'
	option igmp_snooping '1'
	option multicast_querier '0'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'wan'

config interface 'lan'
	option proto 'static'
	option device 'br0.3'
	option ipaddr '192.168.3.2'
	option netmask '255.255.255.0'
	option gateway '192.168.3.1'
	list ip6addr 'xxxx:xxxx:xxxx:3::2/64'
	option ip6gw 'xxxx:xxxx:xxxx:3::1'
	list dns '192.168.3.1'

config bridge-vlan
	option device 'br0'
	option vlan '3'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'wan:t'

config bridge-vlan
	option device 'br0'
	option vlan '50'
	list ports 'wan:t'

config bridge-vlan
	option device 'br0'
	option vlan '99'
	list ports 'wan:t'

config interface 'IOT'
	option proto 'static'
	option device 'br0.50'
	option ipaddr '192.168.50.2'
	option netmask '255.255.255.0'
	option gateway '192.168.50.1'
	list ip6addr 'xxxx:xxxx:xxxx:50::2/64'
	option ip6gw 'xxxx:xxxx:xxxx:50::1'
	list dns '192.168.50.1'
	option defaultroute '0'

config interface 'INVITE'
	option proto 'static'
	option device 'br0.99'
	option ipaddr '192.168.99.2'
	option netmask '255.255.255.0'
	option gateway '192.168.99.1'
	list ip6addr 'xxxx:xxxx:xxxx:99::2/64'
	option ip6gw 'xxxx:xxxx:xxxx:99::1'
	list dns '192.168.99.1'
	option defaultroute '0'

config device
	option name 'wlan0'
	option multicast_to_unicast '1'
	option multicast_fast_leave '1'

config device
	option name 'wlan0-1'
	option multicast_to_unicast '1'
	option multicast_fast_leave '1'

config device
	option name 'wlan0-2'
	option multicast_to_unicast '1'
	option multicast_fast_leave '1'

config device
	option name 'wlan1'
	option multicast_to_unicast '1'
	option multicast_fast_leave '1'

config device
	option name 'wlan1-1'
	option multicast_to_unicast '1'
	option multicast_fast_leave '1'

For /etc/config/wireless :

config wifi-device 'radio0'
	option type 'mac80211'
	option channel '13'
	option hwmode '11g'
	option path '1e140000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0'
	option htmode 'HT20'
	option cell_density '0'
	option country 'FR'
	option txpower '10'

config wifi-iface 'wifinet0'
	option device 'radio0'
	option network 'lan'
	option mode 'ap'
	option ssid 'SSIDLAN'
	option key 'PASSPHRASE'
	option encryption 'sae-mixed'
	option ieee80211r '1'
	option nasid 'SSID1wlan0second'
	option mobility_domain '0812'
	option ft_psk_generate_local '1'
	option ieee80211k '1'
	option rrm_neighbor_report '1'
	option rrm_beacon_report '1'
	option ft_over_ds '0'
	option bss_transition '1'
	option ieee80211w '0'

config wifi-iface 'wifinet1'
	option device 'radio0'
	option mode 'ap'
	option ssid 'SSIDGUEST'
	option key 'PASSPHRASE'
	option network 'invite INVITE'
	option encryption 'sae-mixed'
	option isolate '1'
	option ieee80211r '1'
	option nasid 'SSIDGUESTwlan0second'
	option mobility_domain '1234'
	option ft_psk_generate_local '1'
	option ieee80211k '1'
	option rrm_neighbor_report '1'
	option rrm_beacon_report '1'
	option ft_over_ds '0'
	option bss_transition '1'
	option ieee80211w '0'

config wifi-iface 'wifinet2'
	option device 'radio0'
	option network 'IOT'
	option mode 'ap'
	option ssid 'SSIDIOT'
	option key 'PASSPHRASE'
	option isolate '1'
	option encryption 'sae-mixed'
	option ft_psk_generate_local '1'
	option nasid 'SSIDIOTwlan0second'
	option mobility_domain '5678'
	option ieee80211r '1'
	option ieee80211k '1'
	option rrm_neighbor_report '1'
	option rrm_beacon_report '1'
	option ft_over_ds '0'
	option bss_transition '1'
	option ieee80211w '0'

config wifi-device 'radio1'
	option type 'mac80211'
	option hwmode '11a'
	option path '1e140000.pcie/pci0000:00/0000:00:01.0/0000:02:00.0'
	option htmode 'VHT80'
	option country 'FR'
	option cell_density '0'
	option channel '52'

config wifi-iface 'wifinet3'
	option device 'radio1'
	option mode 'ap'
	option ssid 'SSIDLAN'
	option key 'PASSPHRASE'
	option encryption 'sae-mixed'
	option network 'lan'
	option ieee80211r '1'
	option nasid 'SSIDLANwlan1second'
	option mobility_domain '0812'
	option ft_psk_generate_local '1'
	option ieee80211k '1'
	option rrm_neighbor_report '1'
	option rrm_beacon_report '1'
	option ft_over_ds '0'
	option bss_transition '1'
	option ieee80211w '0'

config wifi-iface 'wifinet4'
	option device 'radio1'
	option mode 'ap'
	option ssid 'SSIDGUEST'
	option key 'PASSPHRASE'
	option network 'INVITE'
	option encryption 'sae-mixed'
	option isolate '1'
	option ieee80211r '1'
	option nasid 'SSIDGUESTwlan1second'
	option mobility_domain '1234'
	option ft_psk_generate_local '1'
	option ieee80211k '1'
	option rrm_neighbor_report '1'
	option rrm_beacon_report '1'
	option ft_over_ds '0'
	option bss_transition '1'
	option ieee80211w '0'

And for /etc/config/firewall :

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'
	option flow_offloading_hw '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option network 'lan'

config include
	option path '/etc/firewall.user'

config zone
	option name 'iot'
	option forward 'REJECT'
	option output 'ACCEPT'
	option network 'IOT'
	option input 'REJECT'

config zone
	option name 'invite'
	option forward 'REJECT'
	option output 'ACCEPT'
	option network 'INVITE'
	option input 'REJECT'

config forwarding
	option dest 'invite'
	option src 'lan'

config forwarding
	option dest 'iot'
	option src 'lan'

For a precise description of my network :

1. I have a Pfsense router with a Wan port on which is connected my FTTH internet from my ISP. From this Pfsense box, I have a trunk port connected to my switch (with VLAN3 VLAN50 and VLAN99)

2. My 2 OpenWrt acces point are connected on dedicated ports on the switch. From the switch side on those ports, I would like to have VLAN3 untagged, and VLAN50 and VLAN99 tagged.

3. On those OpenWrt AP, I would like to configure it as it was before the upgrade to OpenWrt 21 : a bridge of every ethernet ports (WAN LAN1 and LAN2 ports), with untagged VLAN3 on all of those ports , and tagged VLAN50 and VLAN99 on the "WAN" port wich is connected to the swich

But as explained befor, I can't get this to work with untagged VLAN3 between my switch and OpenWrt AP. Actually I have to set tagged VLAN3 and VLAN50 and VLAN99

I don't know if this is precise enought.

Thanks

[jaromanda]

@maximushugus

option network 'invite INVITE'

Is that a typo or actual config?

[maximushugus]

This was actually in my config, and I changed it to

option network 'INVITE'

But it didn't resolve the problem. What is even stranger is that there is no problem to connect and get an IP the first time I connect to Wifi on an access point. But if I reconnect (aver disconnecting or after switching from another AP) the problem occures and what comes from my central main switch as untagged VLAN3 seems to stay on "eth0" but not on "br0.3" as I explained and so there is no connection.