Bandwidth limited to slowest LAN port - Netgear R6850

Installing and Using OpenWrt
1

Disclaimer: I'm aware that measured speed/bandwidth over wireless will never be near the rates advertised on the box. This is about a reproducible, consistent speed drop that doesn't make sense, and not about not getting 1,3 Gbps throughput even though the box says so.

I have a Turris Omnia (mvebu/cortexa9) which is connected to my ISP via fiber converter. In a nearby room I have a Netgear R6850 (ramips/mt7621). The 5 GHz radio of the Omnia runs one AP wifi-iface with WDS, and the 5 GHz radio of the R6850 is configured as a STA/client with WDS connecting to the Omnia. The 5 GHz radios are used exclusively to bring the internet to the office room. No other clients connect to the AP, and the radios only run the single wifi-iface on each.

My computers are connected to the LAN ports of the R6850, getting internet access over the WDS link. Both computers have Gigabit NICs. One of the computers is a HP laptop with an HP Thunderbolt dock, and I use the LAN port on the dock. This dock keeps the LAN port alive at 10 Mbps speed when the laptop is turned off, but still connected. And this impacts all other devices linked to the R6850.

Waveform bufferbloat test gives consistent results when wired directly to the internet, so I use it as reference here. If I disconnect the laptop dock and do /etc/init.d/network restart on the R6850, and then run a speedtest on my main computer, I get between 150-210 Mbps upload and download. It varies within that, which is to be expected over wireless. I am happy with that result. However, if the laptop dock is connected to the R6850, the upload speed sits consistently at 9,2-10,8 Mbps. The download speed can fall slightly, but still end around 160-170 Mbps. So it appears that the 10 Mbps speed of the dock LAN port interferes somewhere else, lowering upload speed for all devices. If I disconnect the dock and run /etc/init.d/network restart, both upload and download speed immediately jump back to 150-210 Mbps. I can go back and forth disconnecting and reconnecting the dock, running /etc/init.d/network restart afterwards and running the speedtest right after, and go back and forth between 9-10 Mbps upload and 150-210 Mbps upload. This can't be explained by just wireless neighbor interference and solar flares.

I have experimented, and strangely it isn't necessary to physically disconnect the laptop dock. If the R6850 port connected to the dock is assigned to a different VLAN than the others, speed is good for the other devices.

I also have a printer connected to the R6850, which only has a 100 Mbps interface. If I avoid the dock, but have the printer connected, upload speed will sit at around 99-102 Mbps. As that is closer to the good reference of 150+ Mbps it isn't as easy to claim the same thing happens, but I can't bring upload speeds above 102 Mbps when the printer is connected. The same thing with VLANs also apply, if the printer port is moved to a different VLAN, speeds are better.

/etc/init.d/network restart is always required to get out of the bad state, after disconnecting the dock. Speeds stay at ~10 Mbps after physically removing the cable until I restart the network.

In addition, when the speedtest is running in the bad ~10 Mbps state, the SSH interface of the R6850 is really slow. I'm connecting to it from my other computer, and even if a 10 Mbps transfer is running while I'm logged in, I shouldn't see input delays while typing commands, but I do.

No amount of restarts or changes on the Omnia end appears to help, everything happens on the R6850. Both are now running the buildbot snapshot images built from main 23ac1ad9515588321970985859b2ee0cc8243f91, but I also reproduced the behavior running 23.05.5 on the R6850.

I know channel 36 is a default, but it's actually the least used channel in the vicinity that doesn't have DFS requirements, so it was chosen on purpose. I've also set the radios to run at VHT40 instead of the default VHT80, as I'm looking for good enough speeds with better reliability.

R6850 configs

# uci export wireless
package wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option path '1e140000.pcie/pci0000:00/0000:00:01.0/0000:02:00.0'
        option band '2g'
        option channel '1'
        option htmode 'HT20'
        option disabled '1'
        option country 'NO'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'OpenWrt'
        option encryption 'none'

config wifi-device 'radio1'
        option type 'mac80211'
        option path '1e140000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0'
        option band '5g'
        option channel '36'
        option htmode 'VHT40'
        option country 'NO'
        option cell_density '0'

config wifi-iface 'default_radio1'
        option device 'radio1'
        option network 'lan'
        option mode 'sta'
        option ssid 'WWDS7'
        option encryption 'psk2+ccmp'
        option key '*'
        option wds '1'
# uci export network
package network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option packet_steering '1'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '172.18.1.3'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option gateway '172.18.1.1'
        list dns '172.18.1.1'

config interface 'wan'
        option device 'wan'
        option proto 'dhcp'

config interface 'wan6'
        option device 'wan'
        option proto 'dhcpv6'

config device
        option type 'bridge'
        option name 'br-switch'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'

config bridge-vlan
        option device 'br-switch'
        option vlan '5'
        list ports 'lan1:u*'
        list ports 'lan3:t'

config bridge-vlan
        option device 'br-switch'
        option vlan '22'

config bridge-vlan
        option device 'br-switch'
        option vlan '30'

config device
        option type 'bridge'
        option name 'br-lan'
        list ports 'br-switch.5'

Omnia configs

# uci export wireless
package wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
        option band '5g'
        option channel '36'
        option htmode 'VHT40'
        option country 'NO'
        option cell_density '0'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'WWDS7'
        option encryption 'psk2+ccmp'
        option key '*'
        option wds '1'

config wifi-device 'radio1'
        option type 'mac80211'
        option path 'soc/soc:pcie/pci0000:00/0000:00:01.0/0000:01:00.0'
        option band '2g'
        option channel '1'
        option htmode 'HT20'
        option disabled '1'
        option country 'NO'
        option cell_density '0'

config wifi-iface 'default_radio1'
        option device 'radio1'
        option mode 'ap'
        option ssid 'OpenWrt'
        option encryption 'none'
# uci export network
package network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdcb:59fe:bbd8::/48'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '172.18.1.1'
        option netmask '255.255.255.0'
        option ip6assign '64'
        option delegate '0'

config interface 'wan'
        option device 'eth2'
        option proto 'dhcp'
        option peerdns '0'
        list dns '185.181.61.24'
        list dns '195.10.195.195'
        option hostname '*'

config interface 'wan6'
        option device 'eth2'
        option proto 'dhcpv6'
        option peerdns '0'
        list dns '2a03:94e0:1804::1'
        list dns '2a00:f826:8:2::195'

config device
        option type 'bridge'
        option name 'br-switch'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'

config bridge-vlan
        option device 'br-switch'
        option vlan '5'
        list ports 'lan2:u*'
        list ports 'lan4:t'

config device
        option type 'bridge'
        option name 'br-lan'
        list ports 'br-switch.5'

Any suggestions? I'm at a loss for where this might be. Could it be a limitation of the switch of the R6850, or maybe a switch driver issue?

2

This looks too complicated. It could be simplified as:

config interface 'lan'
        option device 'br-lan.5'
...

config device
        option type 'bridge'
        option name 'br-lan'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'

config bridge-vlan
        option device 'br-lan'
        option vlan '5'
        list ports 'lan1:u*'
        list ports 'lan3:t'

Maybe it will not fix the problem, but at least it's cleaner.

3

It actually does fix the problem. However, there is a reason for that setup. In the long run I want to run a GRETAP tunnel between the two devices, so they can carry separate VLANs over the single Wi-Fi link. That requires, as far as I have found, assigning each network interface to a bridge device, that bridges the switch VLAN (br-switch.5) and a GRETAP sub-device (gre4t-link.5).

I didn't do the full GRETAP setup because I found the problem isn't introduced by GRETAP, and it looked less confusing running a WDS link which demonstrates the same problem.

I may have to figure out if there are more ways to bridge the GRETAP sub-interfaces to the physical LAN ports, if the multiple layers of bridges is the issue.

4

Well, I'll be damned. :smiley:
Then I guess you need to make some compromise. Or run multiple SSIDs.