ER-X-SFP: VLANs not working properly with kernel 5.4

Got a bit lost in your network layout. To my understanding:

  • eth5 is not enslaved in a bridge
  • eth5 is the physical WAN port but acts as LAN port to a device (modem/router?) upstream (that provides DHCP)
  • eth5 is configured as trunk (tagged VID 200) with the upstream device that is wired into eth5
  • eth5 gets its IP (DHCP) from the upstream device

What/where is eth2 located? Because previously

vs recently

On the MT7621 platform, I made a patch to allow you to set a label on the CPU interface same as on a DSA port.
This patch is also in OpenWrt,
So instead that the CPU port is called eth0 it is now called dsa.
Because on my device the 6 ports are labeled as eth0 - eth5, they are in linux also eth0 - eth5.
So that is the mix up.

Now on my bench eth5 is set as wan port and is not part of the bridge.
The question of NPeca75 seems a good question.
So I wonder how I can made that same setup.
I tried to write down what I think is needed to configure this setup.

Because it is a bit new to me how to setup a DSA port and the bridge.
I may set it up incorrect.
Also mt7530 may still have some bugs that prevent us to geting the right results.

So what I wrote down are that the right steps?

hi @Thirsty
tnx for your work. For weekend i will try it because we will be denied to go out of house for 84 hours (god damn virus) so i will have a plenty time :smiley:

Hey All,

Thanks for all the posts above for the hints that they gave me. Additionally this post Xiaomi Router 3G V2 - VLAN with DSA helped me as well.

On the ERX-SFP, I have WAN connected to eth0 and br-lan connected to eth1. The rest of the ports are the ones I waned to play with. Here is what worked for me. Note, before the start of the script, eth2 and eth3 are not shown in bridge v. This is using the ramips-5.4+fixes branch from vDorst (thanks for all the excellent work).

ip link set eth2 up
ip link set eth3 up
ip link add name br-test102 type bridge
ip link set dev br-test102 type bridge vlan_filtering 1
ip link set dev br-test102 up
ip link set dev eth2 master br-test102
ip link set dev eth3 master br-test102
bridge v del dev eth2 vid 1
bridge v del dev eth3 vid 1
bridge v del dev br-test102 self vid 1
bridge v add dev eth2 vid 102
bridge v add dev eth3 vid 102
bridge v add dev br-test102 self vid 102
ip link add link br-test102 name vlan102 type vlan id 102
ip link set dev vlan102 up
ip add add 192.168.102.1/24 dev vlan102

On eth2 I have a device (192.168.102.2) which directly tags on the port. On eth3 I have a smart witch, which does the tagging and then forward untagged to another port which then goes to a device (192.168.102.3).

I can ssh from the ERX-SFP to either of the devices and from either of those devices to the other. I tried ping and ssh. Both work flawlessly.

Here is another example, just like above, but now I'm using eth4 as an untagged port for a new vlan100 interface. I also added a vlan101 interface. Everything is working.

ip link set eth2 up
ip link set eth3 up
ip link set eth4 up

ip link add name vlans type bridge                
ip link set dev vlans type bridge vlan_filtering 1
ip link set dev vlans up         
ip link set dev eth2 master vlans
ip link set dev eth3 master vlans
ip link set dev eth4 master vlans

bridge v del dev eth2 vid 1
bridge v del dev eth3 vid 1
bridge v del dev eth4 vid 1
bridge v del dev vlans self vid 1

bridge v add dev eth2 vid 100                            
bridge v add dev eth3 vid 100                            
bridge v add dev eth4 vid 100 pvid 100 untagged
bridge v add dev eth2 vid 101
bridge v add dev eth3 vid 101
bridge v add dev eth2 vid 102
bridge v add dev eth3 vid 102
bridge v add dev vlans self vid 100
bridge v add dev vlans self vid 101
bridge v add dev vlans self vid 102                 

ip link add link vlans name vlan100 type vlan id 100
ip link add link vlans name vlan101 type vlan id 101
ip link add link vlans name vlan102 type vlan id 102

ip link set dev vlan100 up
ip link set dev vlan101 up
ip link set dev vlan102 up

ip add add 192.168.100.1/24 dev vlan100
ip add add 192.168.101.1/24 dev vlan101
ip add add 192.168.102.1/24 dev vlan102

I would really like to be able to have the IP addresses set in /etc/config/network so that other things (most importantly OLSRd) can use the info. I tried proto none. It tried proto static with force_link 1. I tried making it a bridge with empty_bridge 1. Any ideas?

1 Like

Let me point out that I am not familiar with the patch and neither with the MT7621 (and its fabric), latter to my understanding being a router on chip with the CPU and the switch being integrated (as opposed to be being separate chips linked on a board).

Usually none of the CPU ports:

  • upstream facing the WAN exterior port (label)
  • downstream facing the switch

are driven by DSA (ethtool -i) and thus not VLAN tag manageable with bridge v command. Subsequent if the WAN port requires VLAN tag it would be with ip l. The CPU port facing the switch commonly does not require VLAN tagging.

As for the switch ports:

  • facing upstream the CPU are not exposed to DSA (by design) and thus there is no VLAN tag management (contrary to the legacy switchconf)
  • facing downstream LAN exterior ports (label) are exposed by DSA and VLAN tag managed with bridge v

Aside from the previously cited I found this documentation [3] helpful about VLAN tag management of DSA port (use scenarios) and the sequences of setting up.


[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/networking/dsa/configuration.rst

Well, what do you know. I can now set up the vlan10X interface in /etc/config/netowrk. I don't know why it didn't work earlier. The entries in /etc/config/network look like this:

config interface 'v100'
	option proto 'static'
	option ipaddr '192.168.100.1'
	option netmask '255.255.255.0'
	option ifname 'vlan100'

And after the commands in ER-X-SFP: VLANs not working properly with kernel 5.4 - #26 by pmelange are run (except the last ip link set dev vlan10X up and ip add add commands) the interface goes up with it's IP address.

I finally found the time to sit down and get the router working. It's a freifunk router using multiple tagged vlans for meshing and at the same time also acting as just a switch for other vlans.

eth0

  • OLSR vlans for rooftop devices. 201 - 206
  • management vlan 5
  • client network vlan 10

eth1

  • OLSR vlan for streetlevel devices; 211
  • management vlan 5
  • client vlan 10

eth2

  • OLSR vlan for streetlevel devices; 211
  • management vlan 5
  • client vlan 10

The following vlans are passed unmanaged through the switch on eth1 and eth2.
11 12 300 400 500 501 1000

eth3 is untagged on vlan 5
eth4 and eth5 is untagged on vlan 10

I call the script /root/vlans.sh from rc.local. This woks at boot, but not if I run /etc/init.d/network restart. If anyone has a suggestion on how to make this survive a network restart, please let me know.

/root/vlans.sh

ip link set eth0 up
ip link set eth1 up
ip link set eth2 up
# not eth3 eth4 eth5
#ip link set eth3 up
#ip link set eth4 up
#ip link set eth5 up

# create the bridge
ip link add name vlans type bridge                
ip link set dev vlans type bridge vlan_filtering 1
ip link set dev eth0 master vlans
ip link set dev eth1 master vlans
ip link set dev eth2 master vlans
#ip link set dev eth3 master vlans
#ip link set dev eth4 master vlans
#ip link set dev eth5 master vlans

# clear out vlan 1
bridge v del dev eth0 vid 1
bridge v del dev eth1 vid 1
bridge v del dev eth2 vid 1
#bridge v del dev eth3 vid 1
#bridge v del dev eth4 vid 1
#bridge v del dev eth5 vid 1
bridge v del dev vlans self vid 1

# set vlans eth0
bridge v add dev eth0 vid 5
bridge v add dev eth0 vid 10
bridge v add dev eth0 vid 50
bridge v add dev eth0 vid 201                   
bridge v add dev eth0 vid 202                  
bridge v add dev eth0 vid 203                  
bridge v add dev eth0 vid 204                  
bridge v add dev eth0 vid 205                  
bridge v add dev eth0 vid 206                  

# set vlans eth1
bridge v add dev eth1 vid 5                                                  
bridge v add dev eth1 vid 10                                                   
bridge v add dev eth1 vid 11                                                   
bridge v add dev eth1 vid 12                                                   
bridge v add dev eth1 vid 211                                                   
bridge v add dev eth1 vid 300                                                   
bridge v add dev eth1 vid 400                                                   
bridge v add dev eth1 vid 500
bridge v add dev eth1 vid 501
bridge v add dev eth1 vid 1000                                                   

# set vlans eth2
bridge v add dev eth2 vid 5                                                   
bridge v add dev eth2 vid 10                                                   
bridge v add dev eth2 vid 11                                                   
bridge v add dev eth2 vid 12                                                   
bridge v add dev eth2 vid 50                                                   
bridge v add dev eth2 vid 211                            
bridge v add dev eth2 vid 300
bridge v add dev eth2 vid 400                                                   
bridge v add dev eth2 vid 500                                                   
bridge v add dev eth2 vid 501                                                   
bridge v add dev eth2 vid 1000                                                   

bridge v add dev vlans self vid 5                                             
bridge v add dev vlans self vid 10                                             
bridge v add dev vlans self vid 11                                             
bridge v add dev vlans self vid 12                                             
bridge v add dev vlans self vid 50                                             
bridge v add dev vlans self vid 201                                             
bridge v add dev vlans self vid 202                                             
bridge v add dev vlans self vid 203                                             
bridge v add dev vlans self vid 204                                             
bridge v add dev vlans self vid 205
bridge v add dev vlans self vid 206
bridge v add dev vlans self vid 211
bridge v add dev vlans self vid 300                                             
bridge v add dev vlans self vid 400                                             
bridge v add dev vlans self vid 500                                             
bridge v add dev vlans self vid 501                                             
bridge v add dev vlans self vid 1000                                             

/etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'AA:BB:CC::/60'

config interface 'mgmt'
	option type 'bridge'
	option ifname 'vlan5 eth5 eth4'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '192.168.5.1'

config interface 'dhcp'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6hint 'f'
	option ipaddr '10.0.0.1'
	option ifname 'vlan10 eth3'
	option dns '46.182.19.48 80.67.169.40 194.150.168.168 2001:910:800::12'
	option type 'bridge'

config interface 'north'
	option proto 'static'
	option ifname 'vlan201'
	option ipaddr '10.0.1.2011'
	option netmask '255.255.255.252'
	option ip6assign '64'
	option ip6hint '1'

config interface 'nano_vlan_202'
	option proto 'static'
	option ifname 'vlan202'
	option ipaddr '10.0.1.202'
	option netmask '255.255.255.255'
	option ip6assign '64'
	option ip6hint '2'

config interface 'ost'
	option proto 'static'
	option ifname 'vlan203'
	option ipaddr '10.0.1.233'
	option netmask '255.255.255.252'
	option ip6assign '64'
	option ip6hint '3'

config interface 'beuth'
	option proto 'static'
	option ifname 'vlan204'
	option ipaddr '10.0.1.237'
	option netmask '255.255.255.252'
	option ip6assign '64'
	option ip6hint '4'

config interface 'nano_vlan_205'
	option proto 'static'
	option ifname 'vlan205'
	option ipaddr '10.0.1.205'
	option netmask '255.255.255.255'
	option ip6assign '64'
	option ip6hint '5'

config interface 'nano_vlan_206'
	option proto 'static'
	option ifname 'vlan206'
	option ipaddr '10.0.1.206'
	option netmask '255.255.255.255'
	option ip6assign '64'
	option ip6hint '6'

config interface 'meshlan'
	option proto 'static'
	option ifname 'vlan211'
	option ipaddr '10.0.1.211'
	option netmask '255.255.255.255'
	option ip6assign '64'
	option ip6hint 'b'

config interface 'tunl0'
	option ifname 'tunl0'
	option proto 'none'

config device             
       option type '8021q'      
       option ifname 'vlans' 
       option vid '1'          
       option name 'vlan1'            
                                        
config device                       
       option type '8021q'           
       option ifname 'vlans'  
       option vid '5'                                                       
       option name 'vlan5'
                            
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '10'                                                          
       option name 'vlan10'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '11'                                                          
       option name 'vlan11'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '12'                                                          
       option name 'vlan12'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '50'                                                          
       option name 'vlan50'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '201'                                                          
       option name 'vlan201'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '202'                                                          
       option name 'vlan202'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '203'                                                          
       option name 'vlan203'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '204'                                                          
       option name 'vlan204'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '205'                                                          
       option name 'vlan205'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '206'                                                          
       option name 'vlan206'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '211'                                                          
       option name 'vlan211'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '300'                                                          
       option name 'vlan300'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '400'                                                          
       option name 'vlan400'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '500'                                                          
       option name 'vlan500'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '501'                                                          
       option name 'vlan501'      
                             
config device                        
       option type '8021q'            
       option ifname 'vlans'  
       option vid '1000'                                                          
       option name 'vlan1000'

Update

There has been some developments and new my best working configuration is based off of https://github.com/openwrt/openwrt/pull/2942#issuecomment-629374851

Even though the setup above works, the example linked above accomplishes that same and look a lot cleaner.

2 Likes

Hi all,

I have run into a problem. It's quite a strange one.

In order for OLSRd to mesh with it's neighbors it sends broadcast packets to port 698. This is working for for ipv4, but with ipv6 it only works for a short amount of time. With tcpdump I can see that the router is sending the broadcast messages to (for example) vlan201. With tcpdump on the raw eth0 device, I can see that the tagged broadcast packets first start out fine then after a few minutes stop. When the packets stop going on eth0, I can still see them on device vlan201.

The strange thing is, if I restart the OLSRd service for ipv6 on a neighbor, the packets start going again across eth0 and the routers mesh again. But also just for a few minutes.

Why would restarting the service on a neighbor node have an influence on how the packets are being sent through the vlan setup? Any ideas or suggestions on how to proceed with testing would be greatly appreciated.

Hey, I got a kernel warning.

[27297.485661] ------------[ cut here ]------------
[27297.494911] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:448 0x80421a44
[27297.508995] NETDEV WATCHDOG: dsa (mtk_soc_eth): transmit queue 0 timed out
[27297.509002] Modules linked in: iptable_nat batman_adv xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT nf_nat nf_flow_table_hw nf_flow_table nf_conntrack_rtcache nf_conntrack ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG sfp nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 mdio_i2c libcrc32c iptable_mangle iptable_filter ip_tables hwmon compat i2c_algo_pca gpio_pca953x regmap_i2c i2c_dev ledtrig_heartbeat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ipip tunnel4 ip_tunnel leds_gpio gpio_button_hotplug crc32c_generic
[27297.626581] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.4.32 #0
[27297.638360] Stack : 00000000 807b0000 ffffffff 80075a94 00000000 00000000 00000000 00000000
[27297.654991]         00000000 00000000 00000000 00000000 00000000 00000001 8fc0dd50 fd7e7749
[27297.671606]         8fc0dde8 00000000 00000000 00000000 00000038 8056ef84 342e3520 2032332e
[27297.688226]         00000000 0000054e 00000000 00098f95 00000000 8fc0dd30 00000000 80421a44
[27297.704844]         00000009 00000000 00200000 00000122 00000002 80313f30 00000000 80780000
[27297.721458]         ...
[27297.726311] Call Trace:
[27297.726327] [<80075a94>] 0x80075a94
[27297.738095] [<8056ef84>] 0x8056ef84
[27297.745024] [<80421a44>] 0x80421a44
[27297.751950] [<80313f30>] 0x80313f30
[27297.758874] [<8000b0f8>] 0x8000b0f8
[27297.765798] [<8000b100>] 0x8000b100
[27297.772727] [<8055631c>] 0x8055631c
[27297.779652] [<80075b4c>] 0x80075b4c
[27297.786576] [<80027c34>] 0x80027c34
[27297.793501] [<80421a44>] 0x80421a44
[27297.800434] [<80027d0c>] 0x80027d0c
[27297.807361] [<80421a44>] 0x80421a44
[27297.814289] [<8009e0a8>] 0x8009e0a8
[27297.821216] [<80421748>] 0x80421748
[27297.828157] [<8008bbac>] 0x8008bbac
[27297.835087] [<8008bde8>] 0x8008bde8
[27297.842040] [<805753a4>] 0x805753a4
[27297.848967] [<8002c328>] 0x8002c328
[27297.855897] [<802bcee0>] 0x802bcee0
[27297.862827] [<80006cc8>] 0x80006cc8
[27297.869748] 
[27297.873114] ---[ end trace a8abf31cae3c8b0f ]---
[27297.882378] mtk_soc_eth 1e100000.ethernet dsa: transmit timed out
[27297.895263] mtk_soc_eth 1e100000.ethernet dsa: Link is Down
[27297.921058] mtk_soc_eth 1e100000.ethernet dsa: configuring for fixed/rgmii link mode
[27297.921580] mtk_soc_eth 1e100000.ethernet dsa: Link is Up - 1Gbps/Full - flow control rx/tx
[34802.388191] mtk_soc_eth 1e100000.ethernet dsa: transmit timed out
[34802.400560] mtk_soc_eth 1e100000.ethernet dsa: Link is Down
[34802.427502] mtk_soc_eth 1e100000.ethernet dsa: configuring for fixed/rgmii link mode
[34802.428039] mtk_soc_eth 1e100000.ethernet dsa: Link is Up - 1Gbps/Full - flow control rx/tx
[46839.251072] mtk_soc_eth 1e100000.ethernet dsa: transmit timed out
[46839.263467] mtk_soc_eth 1e100000.ethernet dsa: Link is Down
[46839.302311] mtk_soc_eth 1e100000.ethernet dsa: configuring for fixed/rgmii link mode
[46839.302808] mtk_soc_eth 1e100000.ethernet dsa: Link is Up - 1Gbps/Full - flow control rx/tx
[51932.366785] mtk_soc_eth 1e100000.ethernet dsa: transmit timed out
[51932.379262] mtk_soc_eth 1e100000.ethernet dsa: Link is Down
[51932.409932] mtk_soc_eth 1e100000.ethernet dsa: configuring for fixed/rgmii link mode
[51932.410398] mtk_soc_eth 1e100000.ethernet dsa: Link is Up - 1Gbps/Full - flow control rx/tx
[71948.502305] mtk_soc_eth 1e100000.ethernet dsa: transmit timed out
[71948.514733] mtk_soc_eth 1e100000.ethernet dsa: Link is Down
[71948.545621] mtk_soc_eth 1e100000.ethernet dsa: configuring for fixed/rgmii link mode
[71948.546004] mtk_soc_eth 1e100000.ethernet dsa: Link is Up - 1Gbps/Full - flow control rx/tx

I was really hoping that the move to kernel 5.4 would fix this.

1 Like

conversation regarding the kernel warning is continued here

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.