MTU issue: ZBT-WE826 (MT7620N) and BMX7 mesh network

Hi there!

I came across the strange issue with MTU, maybe someone had this case or developers can tell me some tips to diagnostics.

Ok, the problem is the packages more than 1428 bytes won't pass through BMX7 ipv4 tunnel:

# ping -s 1428 10.222.222.1
PING 10.222.222.1 (10.222.222.1): 1428 data bytes
1436 bytes from 10.222.222.1: seq=0 ttl=63 time=27.280 ms
1436 bytes from 10.222.222.1: seq=1 ttl=64 time=2.460 ms
1436 bytes from 10.222.222.1: seq=2 ttl=64 time=2.520 ms
1436 bytes from 10.222.222.1: seq=3 ttl=64 time=2.480 ms
^C
--- 10.222.222.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 2.460/8.685/27.280 ms
# ping -s 1429 10.222.222.1
PING 10.222.222.1 (10.222.222.1): 1429 data bytes
^C
--- 10.222.222.1 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

Same time the ipv6 between BMX7 nodes works fine. The BATMAN-ADV protocol also works fine.

I guess it concerns internal virtual switch and some mechanism in BMX7, but what?

So, I'll leave below the cpu information, my /etc/config/network file and swconfig


# cat /etc/openwrt_release 
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='19.07.7'
DISTRIB_REVISION='r11306-c4a6851c72'
DISTRIB_TARGET='ramips/mt7620'
DISTRIB_ARCH='mipsel_24kc'
DISTRIB_DESCRIPTION='OpenWrt 19.07.7 r11306-c4a6851c72'
DISTRIB_TAINTS=''
# cat /proc/cpuinfo 
system type		: MediaTek MT7620N ver:2 eco:6
machine			: ZBT-WE826 (16M)
processor		: 0
cpu model		: MIPS 24KEc V5.0
BogoMIPS		: 385.84
wait instruction	: yes
microsecond timers	: yes
tlb_entries		: 32
extra interrupt vector	: yes
hardware watchpoint	: yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb]
isa			: mips1 mips2 mips32r1 mips32r2
ASEs implemented	: mips16 dsp
Options implemented	: tlb 4kex 4k_cache prefetch mcheck ejtag llsc pindexed_dcache userlocal vint perf_cntr_intr_bit nan_legacy nan_2008 perf
shadow register sets	: 1
kscratch registers	: 0
package			: 0
core			: 0
VCED exceptions		: not available
VCEI exceptions		: not available

# cat /etc/config/network
config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0 1 2 3 5 6t 7'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '4 6t'

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth0.1'
	option proto 'static'
	option ipaddr '10.214.193.1'
	option netmask '255.255.255.224'
	option dns '208.67.222.222 208.67.220.220'
	option ip6assign '60'

config device 'eth0_1_30'
	option type '8021ad'
	option name 'eth0_1_30'
	option ifname 'eth0.1'
	option vid '30'

config interface 'eth0_1_30_ad'
	option ifname 'eth0_1_30'
	option auto '1'

config device 'eth0_2_mac'
	option name 'eth0.2'
	option macaddr 'f8:5e:3c:0d:83:c9'

config device 'eth0_2_30'     
	option type '8021ad' 
	option name 'eth0_2_30'  
	option ifname 'eth0.2' 
	option vid '30'        
                  
config interface 'eth0_2_30_ad'
	option ifname 'eth0_2_30'      
	option auto '1'

config interface 'mesh_w0'
	option proto 'static'
	option ipaddr '169.254.161.1'
	option netmask '255.255.255.248'

config device 'wlan0_30'
	option type '8021q'
	option name 'wlan0_30'
	option ifname '@mesh_w0'
	option vid '30'

config interface 'wlan0_30_ad'
	option ifname 'wlan0_30'
	option auto '1'

# swconfig dev switch0 show
Global attributes:
	enable_vlan: 1
	mib: Switch MIB counters
PPE_AC_BCNT0: 0
PPE_AC_PCNT0: 0
PPE_AC_BCNT63: 0
PPE_AC_PCNT63: 0
PPE_MTR_CNT0: 0
PPE_MTR_CNT63: 0
GDM1_TX_GBCNT: 0
GDM1_TX_GPCNT: 0
GDM1_TX_SKIPCNT: 0
GDM1_TX_COLCNT: 0
GDM1_RX_GBCNT1: 0
GDM1_RX_GPCNT1: 0
GDM1_RX_OERCNT: 0
GDM1_RX_FERCNT: 0
GDM1_RX_SERCNT: 0
GDM1_RX_LERCNT: 0
GDM1_RX_CERCNT: 0
GDM1_RX_FCCNT: 0
GDM2_TX_GBCNT: 0
GDM2_TX_GPCNT: 0
GDM2_TX_SKIPCNT: 0
GDM2_TX_COLCNT: 0
GDM2_RX_GBCNT: 0
GDM2_RX_GPCNT: 0
GDM2_RX_OERCNT: 0
GDM2_RX_FERCNT: 0
GDM2_RX_SERCNT: 0
GDM2_RX_LERCNT: 3
GDM2_RX_CERCNT: 0
GDM2_RX_FCCNT: 0

Port 0:
	mib: Port 0 MIB counters
TxGPC      : 56628
TxBOC      : 0
TxGOC      : 62551869
TxEPC      : 0
RxGPC      : 33233
RxBOC      : 0
RxGOC      : 3604198
RxEPC1     : 0
RxEPC2     : 0

	pvid: 1
	link: port:0 link:down
Port 1:
	mib: Port 1 MIB counters
TxGPC      : 0
TxBOC      : 0
TxGOC      : 0
TxEPC      : 0
RxGPC      : 0
RxBOC      : 0
RxGOC      : 0
RxEPC1     : 0
RxEPC2     : 0

	pvid: 1
	link: port:1 link:down
Port 2:
	mib: Port 2 MIB counters
TxGPC      : 0
TxBOC      : 0
TxGOC      : 0
TxEPC      : 0
RxGPC      : 0
RxBOC      : 0
RxGOC      : 0
RxEPC1     : 0
RxEPC2     : 0

	pvid: 1
	link: port:2 link:down
Port 3:
	mib: Port 3 MIB counters
TxGPC      : 0
TxBOC      : 0
TxGOC      : 0
TxEPC      : 0
RxGPC      : 0
RxBOC      : 0
RxGOC      : 0
RxEPC1     : 0
RxEPC2     : 0

	pvid: 1
	link: port:3 link:down
Port 4:
	mib: Port 4 MIB counters
TxGPC      : 58869
TxBOC      : 0
TxGOC      : 7723643
TxEPC      : 0
RxGPC      : 58530
RxBOC      : 0
RxGOC      : 25111891
RxEPC1     : 0
RxEPC2     : 0

	pvid: 2
	link: port:4 link:up speed:100baseT full-duplex 
Port 5:
	mib: Port 5 MIB counters
TxGPC      : 0
TxBOC      : 0
TxGOC      : 0
TxEPC      : 0
RxGPC      : 0
RxBOC      : 0
RxGOC      : 0
RxEPC1     : 0
RxEPC2     : 0

	pvid: 1
	link: port:5 link:down
Port 6:
	mib: Port 6 MIB counters
TxGPC      : 26229
TxBOC      : 0
TxGOC      : 29082805
TxEPC      : 0
RxGPC      : 61557
RxBOC      : 0
RxGOC      : 72309218
RxEPC1     : 0
RxEPC2     : 11593

	pvid: 0
	link: port:6 link:up speed:1000baseT full-duplex 
Port 7:
	mib: Port 7 MIB counters
TxGPC      : 0
TxBOC      : 0
TxGOC      : 0
TxEPC      : 0
RxGPC      : 0
RxBOC      : 0
RxGOC      : 0
RxEPC1     : 0
RxEPC2     : 0

	pvid: 1
	link: port:7 link:down
VLAN 1:
	vid: 1
	ports: 0 1 2 3 5 6t 7 
VLAN 2:
	vid: 2
	ports: 4 6t
1 Like

Same thing on 21.02.0

1 Like

The issue appears with VLAN standard 802.1ad and everything's ok with 802.1q

config device 'eth0_1_31'
        option type '8021q'
        option name 'eth0_1_31'
        option ifname 'eth0.1'
        option vid '31'

config interface 'eth0_1_31_ad'
        option ifname 'eth0_1_31'
        option auto '1'

I periodically forget about this bug, and iconic, open issues again and again when came across this problem. So I am going to collect all my issues and believe that someday BMX6/7 developers team will find some work around of 801.1ad's small MTU.

According to this and this link from qMp developers team's blog, the bug has already well known for a long time and reveals time to time depending on drivers. But I don't think that that's a good idea to move my MESH to 802.1q though and that's a bad idea to use MESH without vlan. So I decided to use 802.1q partly where it depends.