Belkin RT3200/Linksys E8450 WiFi AX discussion

_FailSafe · October 7, 2023, 1:31pm

Two questions for you, if I may...

Is WED consistently working for you on your build or does it seem to stop showing offloaded flows after some amount of uptime? My reason for asking is that @nbd had indicated there may be some WED regression and I've not heard any update around that. See here: https://github.com/openwrt/mt76/issues/754#issuecomment-1703703105
Are you seeing erratic connectivity on your RT3200 with the 6.1 kernel? I have tried twice to switch over to kernel 6.1 builds and each time (several weeks apart) I would see extremely consistent and unusable latency spikes on the ethernet side of the device. So much so that it makes it inoperable as an AP and it takes a long time to get a new image uploaded to it in order to flash back to a kernel 5.15 build.

I'm tempted to just test it again and see if the ethernet latency is back to normal with kernel 6.1. But if you can save me some time, I'm all for that too Thanks!

Update:
I just tried my kernel 6.1 build again (actually did a full make dirclean first) and my resulting image produces this type of behavior:

From the AP where I flashed my kernel 6.1 build --> 192.168.XX.5 being my gateway device:

root@AP-Office:~# ping 192.168.XX.5
PING 192.168.XX.5 (192.168.XX.5): 56 data bytes
64 bytes from 192.168.XX.5: seq=0 ttl=64 time=2058.193 ms
64 bytes from 192.168.XX.5: seq=1 ttl=64 time=1058.342 ms
64 bytes from 192.168.XX.5: seq=2 ttl=64 time=59.063 ms
64 bytes from 192.168.XX.5: seq=3 ttl=64 time=162.308 ms
64 bytes from 192.168.XX.5: seq=4 ttl=64 time=1457.042 ms
64 bytes from 192.168.XX.5: seq=5 ttl=64 time=458.088 ms
64 bytes from 192.168.XX.5: seq=6 ttl=64 time=3946.141 ms
64 bytes from 192.168.XX.5: seq=7 ttl=64 time=2946.792 ms
64 bytes from 192.168.XX.5: seq=8 ttl=64 time=1950.370 ms
64 bytes from 192.168.XX.5: seq=10 ttl=64 time=4960.722 ms
64 bytes from 192.168.XX.5: seq=11 ttl=64 time=3961.502 ms
64 bytes from 192.168.XX.5: seq=12 ttl=64 time=2961.972 ms
64 bytes from 192.168.XX.5: seq=13 ttl=64 time=1962.462 ms
64 bytes from 192.168.XX.5: seq=14 ttl=64 time=962.632 ms
64 bytes from 192.168.XX.5: seq=15 ttl=64 time=0.514 ms
64 bytes from 192.168.XX.5: seq=16 ttl=64 time=1527.167 ms
64 bytes from 192.168.XX.5: seq=17 ttl=64 time=527.417 ms
64 bytes from 192.168.XX.5: seq=18 ttl=64 time=1053.480 ms
64 bytes from 192.168.XX.5: seq=19 ttl=64 time=53.528 ms
64 bytes from 192.168.XX.5: seq=20 ttl=64 time=0.535 ms
64 bytes from 192.168.XX.5: seq=21 ttl=64 time=63.599 ms
64 bytes from 192.168.XX.5: seq=22 ttl=64 time=404.229 ms
64 bytes from 192.168.XX.5: seq=23 ttl=64 time=0.381 ms
64 bytes from 192.168.XX.5: seq=24 ttl=64 time=107.587 ms
64 bytes from 192.168.XX.5: seq=25 ttl=64 time=4163.145 ms
64 bytes from 192.168.XX.5: seq=26 ttl=64 time=3163.613 ms
64 bytes from 192.168.XX.5: seq=27 ttl=64 time=2165.128 ms
64 bytes from 192.168.XX.5: seq=28 ttl=64 time=1165.411 ms
64 bytes from 192.168.XX.5: seq=29 ttl=64 time=165.575 ms
64 bytes from 192.168.XX.5: seq=30 ttl=64 time=0.479 ms
64 bytes from 192.168.XX.5: seq=31 ttl=64 time=0.486 ms
64 bytes from 192.168.XX.5: seq=32 ttl=64 time=1981.880 ms
64 bytes from 192.168.XX.5: seq=33 ttl=64 time=982.201 ms
64 bytes from 192.168.XX.5: seq=34 ttl=64 time=0.500 ms
64 bytes from 192.168.XX.5: seq=35 ttl=64 time=55.053 ms
64 bytes from 192.168.XX.5: seq=36 ttl=64 time=54.708 ms
64 bytes from 192.168.XX.5: seq=37 ttl=64 time=129.539 ms
64 bytes from 192.168.XX.5: seq=38 ttl=64 time=672.157 ms
64 bytes from 192.168.XX.5: seq=39 ttl=64 time=270.285 ms
...

umayer · October 8, 2023, 12:30pm

Hey Mark,
on 1:
Already when I tested WED with iperf3 I observed that a second manual invocation back to back did not use WED. Only later (after some time with no traffic?) WED was engaged again on iperf3.
I don't understand WED good enough to tell, but it made me think that some resource is blocked and only later once that resource is released a new WED could engage.
I still kept left WED active at that time.

However, I did observe issues with streaming apps on my TV later that day.
A day later a video conference had issues: Network connection dropped a couple of times for about 10 seconds.
Since then WED is disabled again.
Besides the odd behavior I also saw some of these error messages:
Mon Oct 2 07:52:53 2023 daemon.err hostapd: nl80211: kernel reports: key addition failed

on 2:
I did not observe any latency issues so far. Note that I did not only enable kernel 6.1 but also use a different version of mt76 (fcc2f3d82bc9d4d0a8f00d14e892055847c840a7).
So far it seems rock solid, but I think it is too early for a final verdict.
I did not see key addition failed messages anymore.

Right now I am happy with my image.
On kernel 5.15 I had STAs which showed WiFi connected but were not able to connect (e.g. ping to AP).
I then rebooted said device and all was good again. This happened in particular with many wireless devices. (i.e. It felt reasonably solid if only few persons were at home, but with visitors it happened more frequently)

My setup has VLANs and DAWN (802.11 k, r, v) as well as beamforming active on the APs and the router.
So far only WPA2, but will give WPA3 mixed mode a try soon (guest WiFi has opportunistic encryption enabled, i.e. 802.11w is already used)

andreagdipaolo · October 8, 2023, 12:59pm

Keep in mind that WPA3 (even mixed) and fast roaming don't play nicely with some Apple devices.
They will refuse to connect.

_FailSafe · October 8, 2023, 1:02pm

In my experience this is true, though I've only had it happen on truly old Apple devices (think iPhone 5 & 6). Here's the official Apple doc to indicate WPA3 support:

_FailSafe · October 8, 2023, 1:15pm

I may be able to dedicate some time today to trying my 6.1 build again and figure out what is causing me issues.

FWIW, I am running a very similar sounding configuration, including multiple VLANS+SSIDs, though I'm using Usteer instead of DAWN. On my primary and guest SSIDs, I have switched completely over to WPA3 only as I had some issues with WPA2/WPA3 mixed. In my household of ~20 Apple devices (all newer than iPhone 6 at this point) I have zero WPA3 issues.

My IoT SSID has another ~15 Android/Linux type devices, so for now I have had to leave it at WPA2 until device manufacturers start putting out better firmware updates for WPA3 support. Even WPA2/WPA3 mixed did not make some of these devices happy

andreagdipaolo · October 8, 2023, 1:18pm

I've had problems even with devices running the latest iOS. There are some threads here about this issue.

umayer · October 8, 2023, 2:20pm

@_FailSafe do you have 802.11r (fast transition) active?

odrt · October 8, 2023, 2:28pm

Pure WPA3 is perfectly fine. WPA3-WPA2 mixed mode is often causing problems.

github.com/openwrt/openwrt

FS#3108 - Some devices unable to connect in sae-mixed mode (WPA3/WPA2 Personal)

opened 05:01PM - 18 May 20 UTC

openwrt-bot

flyspray

*Freddicus:* Device: Netgear Nighthawk X4S (R7800) Software version: OpenWrt 1…9.07.2 r10947-65030d81f3 / LuCI openwrt-19.07 branch git-20.134.55291-ba0fb08 Steps to reproduce: - Start with the firmware linked here: https://openwrt.org/toh/netgear/r7800 - Remove wpad-basic - Install wpad-openssl - Configure network to broadcast in WPA3/WPA2 Personal (sae-mixed) Devices previously able to connect to WPA2 mode: - 2012 Macbook Pro on macOS Catalina 10.15.4 - Up-to-date raspberry pi 4 on raspbian buster If I switch the radio to WPA2 Personal only (wpa-psk if not using luci, I believe), they can connect. I switch it back to WPA3/WPA2 Personal (sae-mixed) they cannot connect. Let me know what logs to gather and how to gather them. If it's a client issue, please confirm, but my understanding is that mixed mode should support all WPA2 Personal compatible devices.

This can be avoided with an AP with virtual AP capability: create two SSIDs, one for pure WPA2
for older only WPA2 compatible devices and another with pure WPA3 for recent devices. Avoid WPA3-WPA2 mixed mode.

The Belkin RT3200 aka Linksys E8450 has vap multi SSID support.

Or configure the 2.4 GHz interface to WPA2 and let all WPA3 incompatible stations connect to this SSID.

andreagdipaolo · October 8, 2023, 3:01pm

My current setup is: guest (2.4+5 same name) and my main one (2.4+5 different names), with the exception of a IoT ntework on the 2.4 GHz.
I wanted WPA2/3 or 3 only on all SSIDs but at the moment I chose this configuration:
Fast roaming enabled only on guest and main, on IoT not enabled because it's only coming from a single AP. Guest on WPA2, main on WPA3, IoT on mixed.
I'm quite happy with that.

_FailSafe · October 8, 2023, 5:07pm

Indeed, I do

umayer · October 8, 2023, 8:45pm

even with WED disabled I had a mobile phone in my WiFi which got dropped out of WiFi for a minute or two.
Connectivity got re-established with no manual interaction.
I have grepped logread on one of the interesting MAC addresses:

Sun Oct  8 18:46:26 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:46:27 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:46:27 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:46:28 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:46:29 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session 28021A0C52F7A5B0
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:46:29 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:46:57 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX:17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:46:58 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:46:58 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:46:59 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session 3FA54E91EA1445BD
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:46:59 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:28 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX:17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:47:29 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:47:30 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session EB200FAA26A6B02B
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:47:30 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:30 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:31 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:47:32 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session BB6347E327BD966C
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:47:32 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:58 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX:17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:47:59 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:48:00 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session B4354EF255B4A2D2
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:48:00 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17

This could be related to DAWN.
Obviously it is not clear to me which component is responsible.
@PolynomialDivision this is off of a checkout from 2023-09-30:
OpennWrt SNAPSHOT r24056-86dadeba48 / LuCI Master git-23.272.64820-607e9bb

Would you spot any configuration issues here:

~# cat /etc/config/dawn 

config local
	option loglevel '0'

config network
	option broadcast_ip '191.168.1.255'
	option server_ip '192.168.1.1'
	option broadcast_port '1025'
	option tcp_port '1026'
	option network_option '2'
	option shared_key 'XXXXXXXXXX'
	option iv 'XXXXXXXXXX'
	option use_symm_enc '0'
	option collision_domain '-1'
	option bandwidth '-1'

config hostapd
	option hostapd_dir '/var/run/hostapd'

config times
	option con_timeout '60'
	option update_client '10'
	option remove_client '15'
	option remove_probe '30'
	option remove_ap '460'
	option update_hostapd '10'
	option update_tcp_con '10'
	option update_chan_util '5'
	option update_beacon_reports '20'

config metric 'global'
	option min_probe_count '3'
	option bandwidth_threshold '6'
	option use_station_count '0'
	option max_station_diff '1'
	option eval_probe_req '0'
	option eval_auth_req '0'
	option eval_assoc_req '0'
	option kicking '3'
	option kicking_threshold '20'
	option deny_auth_reason '1'
	option deny_assoc_reason '17'
	option min_number_to_kick '3'
	option chan_util_avg_period '3'
	option set_hostapd_nr '2'
	option duration '10'
	option rrm_mode 'pat'

config metric '802_11g'
	option initial_score '80'
	option ht_support '5'
	option vht_support '5'
	option no_ht_support '0'
	option no_vht_support '0'
	option rssi '15'
	option rssi_val '-60'
	option low_rssi_val '-80'
	option low_rssi '-15'
	option chan_util '0'
	option chan_util_val '140'
	option max_chan_util '-15'
	option max_chan_util_val '170'
	option rssi_weight '0'
	option rssi_center '-70'

config metric '802_11a'
	option initial_score '100'
	option ht_support '5'
	option vht_support '5'
	option no_ht_support '0'
	option no_vht_support '0'
	option rssi '15'
	option rssi_val '-60'
	option low_rssi_val '-71'
	option low_rssi '-15'
	option chan_util '0'
	option chan_util_val '140'
	option max_chan_util '-15'
	option max_chan_util_val '170'
	option rssi_weight '0'
	option rssi_center '-70'

fonix232 · October 9, 2023, 9:42am

Would you mind publishing your configs, incl. the wireless config? I'm having some trouble with both usteer and DAWN and want to see if something is misconfigured on my end.

Specifically I have some trouble with my HomePods continuously disconnecting on a 6 minute cycle (even during use connected to my Apple TV). In fact both usteer and DAWN seems to be kicking off every device from the network on that 6 minute interval...

darekxan · October 10, 2023, 7:31am

I've had severe stability problems with SNAPSHOT build over last couple of months. Every four days or so device was becoming completely unresponsive and required physical restart to get it up and running again.

I tried different radio firmwares and so, but it didn't make any difference regarding this particular problem.

In effort to stay updated I flashed build r23995-ce7209bd21 which crashed on me in the similar way = device just became unresponsive in about three days.

Then I decided to turn off WED after crash and give it a shot.

Percieved Wifi and routing performance didn't change for me at all with WED disabled. I still observe excellent performance:
850Mbps+ iperf3 WIFI <-> Ethernet
around 1Gbit fast.com benchmark.

For the most important part now:
It's been running 12 days stable! No problems in kernel log either.

quarky · October 10, 2023, 8:03am

Yes, WED has stability issue at the moment. My E8450 has been up more than 30 days since I disabled WED. I still have hardware flow-offload enabled tho.

fonix232 · October 10, 2023, 9:50am

I use mine as dumb APs (but with half a dozen VLANs), so WED and HW offloading doesn't do much for me. Same experience though, they're rock stable, and aside from the biweekly update, I never need to touch them.

Gingernut · October 10, 2023, 1:05pm

If anyone would like to test WED and wireless stability with recent updated firmware files. Mediatek uploads them here:

https://git01.mediatek.com/plugins/gitiles/openwrt/feeds/mtk-openwrt-feeds/+/f6c5cede007caf5dd0b9744dbb30f074c3084d6b/autobuild_mac80211_release/package/kernel/mt76/src

You can download them as a .tar.gz and scp them across to the /lib/firmware/mediatek/ directory replacing the ones that are there and then reboot.

You should probably also backup the original firmware files first just in case you need to switch back.

darekxan · October 10, 2023, 6:24pm

This type of issue:
regular crash around 4 day mark which I observe with WED on
would mean in my field of exprertise (which is Kotlin/Swift native mobile development)
that current WED code causes a memory leak in the MCU.

I'm in no way experienced in kernel/driver/mcu development, so it's just a wild guess.

eginnc · October 11, 2023, 12:13am

Same here. My RT3200 has been running over a month on SNAPSHOT as a dumb AP with several WiFi interfaces on several vlans. No WED.

No complaints beyond UNII-1 max txpower still being hobbled to 23 dBm despite the RT3200 having been certified to 27 dBm on UNII-1 by the FCC. Aside from that, it has been rock solid and stable.

FanOfOpenWRT · October 11, 2023, 2:23am

In regards to your complaint about max txpower on the RT3200 —

I thought the last time it was brought up we learned it's to keep the device within the margin of the legal limit?

ka2107 · October 11, 2023, 2:42am

Please see my reply at TX power limitation in OpenWrt - yes or no? - #23 by ka2107