Belkin RT3200/Linksys E8450 WiFi AX discussion

umayer · October 8, 2023, 8:45pm

even with WED disabled I had a mobile phone in my WiFi which got dropped out of WiFi for a minute or two.
Connectivity got re-established with no manual interaction.
I have grepped logread on one of the interesting MAC addresses:

Sun Oct  8 18:46:26 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:46:27 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:46:27 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:46:28 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:46:29 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session 28021A0C52F7A5B0
Sun Oct  8 18:46:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:46:29 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:46:57 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX:17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:46:58 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:46:58 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:46:59 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session 3FA54E91EA1445BD
Sun Oct  8 18:46:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:46:59 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:28 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX:17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:47:29 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:29 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:47:30 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session EB200FAA26A6B02B
Sun Oct  8 18:47:30 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:47:30 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:30 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:31 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:47:32 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session BB6347E327BD966C
Sun Oct  8 18:47:32 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:47:32 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:58 2023 daemon.info dawn: Client 8C:XX:XX:XX:XX:17: Kicking due to low active data transfer: RX rate 2.000000 below 6 limit
Sun Oct  8 18:47:59 2023 daemon.notice hostapd: wl0-ap1: AP-STA-DISCONNECTED 8c:xx:xx:xx:xx:17
Sun Oct  8 18:47:59 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: disassociated due to inactivity
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: authenticated
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 IEEE 802.11: associated (aid 1)
Sun Oct  8 18:48:00 2023 daemon.notice hostapd: wl0-ap1: AP-STA-CONNECTED 8c:xx:xx:xx:xx:17 auth_alg=open
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 RADIUS: starting accounting session B4354EF255B4A2D2
Sun Oct  8 18:48:00 2023 daemon.info hostapd: wl0-ap1: STA 8c:xx:xx:xx:xx:17 WPA: pairwise key handshake completed (RSN)
Sun Oct  8 18:48:00 2023 daemon.notice hostapd: wl0-ap1: EAPOL-4WAY-HS-COMPLETED 8c:xx:xx:xx:xx:17

This could be related to DAWN.
Obviously it is not clear to me which component is responsible.
@PolynomialDivision this is off of a checkout from 2023-09-30:
OpennWrt SNAPSHOT r24056-86dadeba48 / LuCI Master git-23.272.64820-607e9bb

Would you spot any configuration issues here:

~# cat /etc/config/dawn 

config local
	option loglevel '0'

config network
	option broadcast_ip '191.168.1.255'
	option server_ip '192.168.1.1'
	option broadcast_port '1025'
	option tcp_port '1026'
	option network_option '2'
	option shared_key 'XXXXXXXXXX'
	option iv 'XXXXXXXXXX'
	option use_symm_enc '0'
	option collision_domain '-1'
	option bandwidth '-1'

config hostapd
	option hostapd_dir '/var/run/hostapd'

config times
	option con_timeout '60'
	option update_client '10'
	option remove_client '15'
	option remove_probe '30'
	option remove_ap '460'
	option update_hostapd '10'
	option update_tcp_con '10'
	option update_chan_util '5'
	option update_beacon_reports '20'

config metric 'global'
	option min_probe_count '3'
	option bandwidth_threshold '6'
	option use_station_count '0'
	option max_station_diff '1'
	option eval_probe_req '0'
	option eval_auth_req '0'
	option eval_assoc_req '0'
	option kicking '3'
	option kicking_threshold '20'
	option deny_auth_reason '1'
	option deny_assoc_reason '17'
	option min_number_to_kick '3'
	option chan_util_avg_period '3'
	option set_hostapd_nr '2'
	option duration '10'
	option rrm_mode 'pat'

config metric '802_11g'
	option initial_score '80'
	option ht_support '5'
	option vht_support '5'
	option no_ht_support '0'
	option no_vht_support '0'
	option rssi '15'
	option rssi_val '-60'
	option low_rssi_val '-80'
	option low_rssi '-15'
	option chan_util '0'
	option chan_util_val '140'
	option max_chan_util '-15'
	option max_chan_util_val '170'
	option rssi_weight '0'
	option rssi_center '-70'

config metric '802_11a'
	option initial_score '100'
	option ht_support '5'
	option vht_support '5'
	option no_ht_support '0'
	option no_vht_support '0'
	option rssi '15'
	option rssi_val '-60'
	option low_rssi_val '-71'
	option low_rssi '-15'
	option chan_util '0'
	option chan_util_val '140'
	option max_chan_util '-15'
	option max_chan_util_val '170'
	option rssi_weight '0'
	option rssi_center '-70'

fonix232 · October 9, 2023, 9:42am

Would you mind publishing your configs, incl. the wireless config? I'm having some trouble with both usteer and DAWN and want to see if something is misconfigured on my end.

Specifically I have some trouble with my HomePods continuously disconnecting on a 6 minute cycle (even during use connected to my Apple TV). In fact both usteer and DAWN seems to be kicking off every device from the network on that 6 minute interval...

darekxan · October 10, 2023, 7:31am

I've had severe stability problems with SNAPSHOT build over last couple of months. Every four days or so device was becoming completely unresponsive and required physical restart to get it up and running again.

I tried different radio firmwares and so, but it didn't make any difference regarding this particular problem.

In effort to stay updated I flashed build r23995-ce7209bd21 which crashed on me in the similar way = device just became unresponsive in about three days.

Then I decided to turn off WED after crash and give it a shot.

Percieved Wifi and routing performance didn't change for me at all with WED disabled. I still observe excellent performance:
850Mbps+ iperf3 WIFI <-> Ethernet
around 1Gbit fast.com benchmark.

For the most important part now:
It's been running 12 days stable! No problems in kernel log either.

quarky · October 10, 2023, 8:03am

Yes, WED has stability issue at the moment. My E8450 has been up more than 30 days since I disabled WED. I still have hardware flow-offload enabled tho.

fonix232 · October 10, 2023, 9:50am

I use mine as dumb APs (but with half a dozen VLANs), so WED and HW offloading doesn't do much for me. Same experience though, they're rock stable, and aside from the biweekly update, I never need to touch them.

Gingernut · October 10, 2023, 1:05pm

If anyone would like to test WED and wireless stability with recent updated firmware files. Mediatek uploads them here:

https://git01.mediatek.com/plugins/gitiles/openwrt/feeds/mtk-openwrt-feeds/+/f6c5cede007caf5dd0b9744dbb30f074c3084d6b/autobuild_mac80211_release/package/kernel/mt76/src

You can download them as a .tar.gz and scp them across to the /lib/firmware/mediatek/ directory replacing the ones that are there and then reboot.

You should probably also backup the original firmware files first just in case you need to switch back.

darekxan · October 10, 2023, 6:24pm

This type of issue:
regular crash around 4 day mark which I observe with WED on
would mean in my field of exprertise (which is Kotlin/Swift native mobile development)
that current WED code causes a memory leak in the MCU.

I'm in no way experienced in kernel/driver/mcu development, so it's just a wild guess.

eginnc · October 11, 2023, 12:13am

Same here. My RT3200 has been running over a month on SNAPSHOT as a dumb AP with several WiFi interfaces on several vlans. No WED.

No complaints beyond UNII-1 max txpower still being hobbled to 23 dBm despite the RT3200 having been certified to 27 dBm on UNII-1 by the FCC. Aside from that, it has been rock solid and stable.

FanOfOpenWRT · October 11, 2023, 2:23am

In regards to your complaint about max txpower on the RT3200 —

I thought the last time it was brought up we learned it's to keep the device within the margin of the legal limit?

ka2107 · October 11, 2023, 2:42am

Please see my reply at TX power limitation in OpenWrt - yes or no? - #23 by ka2107

_FailSafe · October 11, 2023, 12:59pm

Apologies for the delay! Been a busy week and lost track of time. sigh

Below is my current Usteer config that is running on each of my three RT3200s (AP mode). Couple things to keep in mind as anyone looks at this:

I have done some extensive AP signal/SNR testing (using NetSpot) within my particular home to address dead spots and AP overlap. Just slapping this config into place on your device and calling it a day will do you a disservice. You must take some basic steps to tune your AP radio channels and power to get the right overlap within your space.
We all have our "ecosystems" of choice. My family's is Apple. Yours may be Google or Amazon, or something else. The settings I configured in my Usteer config are tweaked based on Apple's specs for wireless operation (read: roaming) of their devices and software. Heavily Android or you name it else platforms may work better with some of the settings tweaked to their specifications. TLDR; non-Apple mileage may vary.

With my little disclaimer out of the way, here you go:

Here's my wireless config. Keep in mind I have a matching SSID name for primary 2.4Ghz/5Ghz, matching SSID name for guest 2.4Ghz/5Ghz, and an IoT SSID on 2.4Ghz only.

Probably goes without saying, but I'll say it anyway... This is just one of three wireless configs. Each of my RT3200s has a similar config, but the radio configurations have channel, power, and BSS color variations that are unique to each AP.

For anyone interested in some nitty-gritty details, take a look here at some of the interesting discoveries I found around Apple's handling of 802.11k:

Catfriend1 · October 11, 2023, 8:06pm

What does WED mean?

odrt · October 11, 2023, 10:54pm

https://www.google.com/search?q=openwrt+wed
https://forum.openwrt.org/search?q=wed

I think 30 seconds of own research should be the minimum before asking others to explain things that can be easily found.

eginnc · October 12, 2023, 8:25am

All I learned from that thread is that some did not want to take the time to carefully read and understand what I wrote, including reading the FCC report. The RT3200 was testified and certified by the FCC for at least 27 dbM on UNII-1 and UNII-3, including antenna gain. If there were issues with legal limits, the FCC would not have certified it.

fonix232 · October 12, 2023, 3:37pm

Cheers, this is incredibly useful.

Unfortunately I'm in a veeeery mixed household (personal devices are mainly Apple, but I also have a bunch of Google Nest/Home devices, Android TV, and due to my job, a number of Android phones). It's a bummer Netspot is so expensive

Same here I see that our configs mostly match, main differences are that I've set up the NAS ID and mobility domain fields - plus I use local PMK generation.

I'll try your usteer settings though and will report back in a few days if it brought any changes, positive or negative.

rvs513 · October 12, 2023, 6:33pm

The RT3200 has been working perfectly, as long as I stay away from using 'AX' and fast transition.
The 802.11r never seemed to work with mostly Apple devices.
I use the 2.4 and 5 GHz network with the same SSID. I have wed enabled now and WPA2-WPA3 mixed mode.
5 GHz network uses the 80MHz AC mode.
This works very reliable in 22.03.5

The moment I make the switch to 'AX', I see this in the log:

Sun Oct  8 19:22:41 2023 kern.err kernel: [660062.100713] mt7915e 0000:01:00.0: Message 000026ed (seq 13) timeout
Sun Oct  8 19:23:01 2023 kern.err kernel: [660082.580463] mt7915e 0000:01:00.0: Message 00005aed (seq 14) timeout
Sun Oct  8 19:23:22 2023 kern.err kernel: [660103.060206] mt7915e 0000:01:00.0: Message 00005aed (seq 15) timeout
Sun Oct  8 19:23:42 2023 kern.err kernel: [660123.539992] mt7915e 0000:01:00.0: Message 000026ed (seq 1) timeout
Sun Oct  8 19:24:03 2023 kern.err kernel: [660144.019767] mt7915e 0000:01:00.0: Message 00005aed (seq 2) timeout
Sun Oct  8 19:24:23 2023 kern.err kernel: [660164.499538] mt7915e 0000:01:00.0: Message 0000aded (seq 3) timeout
Sun Oct  8 19:24:44 2023 kern.err kernel: [660184.979286] mt7915e 0000:01:00.0: Message 0000aded (seq 4) timeout
Sun Oct  8 19:25:04 2023 kern.err kernel: [660205.459040] mt7915e 0000:01:00.0: Message 000025ed (seq 5) timeout
Sun Oct  8 19:25:25 2023 kern.err kernel: [660225.938815] mt7915e 0000:01:00.0: Message 000025ed (seq 6) timeout

Followed by:
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.014248] ------------[ cut here ]------------
Sun Oct  8 19:41:07 2023 kern.warn kernel: [661168.018966] WARNING: CPU: 1 PID: 21509 at 0xffffffc008ba7af8 [mac80211@0000000080a2274c+0x79000]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.027827] Modules linked in: ksmbd asn1_decoder pppoe ppp_async wireguard pppox ppp_generic nft_redir nft_nat nft_masq nft_flow_offload nft_fib_inet nft_ct nft_chain_nat nf_nat nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet nf_flow_table nf_conntrack_netlink nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 libchacha20poly1305 ipt_REJECT ftdi_sio chacha_neon cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY usbserial slhc sch_cake poly1305_neon nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_quota nft_objref nft_numgen nft_log nft_limit nft_hash nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_counter nf_tables nf_reject_ipv6 nf_reject_ipv4 nf_log_ipv6 nf_log_ipv4 nf_log_common nf_defrag_ipv6 nf_defrag_ipv4 libcurve25519_generic libcrc32c libchacha iptable_mangle iptable_filter ipt_ECN ip_tables hwmon crc_ccitt compat sch_tbf
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.027998]  sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact xt_set x_tables ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ipmac ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ifb ip6_udp_tunnel udp_tunnel oid_registry vfat fat nls_utf8 nls_iso8859_1 nls_cp437 sha512_generic seqiv md5 md4 des_generic libdes uhci_hcd ohci_platform ohci_hcd usb_storage leds_gpio fsl_mph_dr_of ehci_platform ehci_fsl ehci_hcd gpio_button_hotplug [last unloaded: ksmbd]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.180660] CPU: 1 PID: 21509 Comm: kworker/u4:2 Tainted: G S                5.10.176 #0
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.188827] Hardware name: Linksys E8450 (UBI) (DT)
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.193798] Workqueue: phy1 0xffffffc008ba6290 [mac80211@0000000080a2274c+0x79000]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.201450] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.207536] pc : 0xffffffc008ba7af8 [mac80211@0000000080a2274c+0x79000]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.214228] lr : 0xffffffc008ba79a4 [mac80211@0000000080a2274c+0x79000]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.220919] sp : ffffffc012513c90
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.224311] x29: ffffffc012513c90 x28: ffffffc010adf010
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.229703] x27: ffffff8006391000 x26: 0000000000000000
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.235094] x25: ffffffc010adf000 x24: ffffff80027511a0
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.240486] x23: ffffff80099300e8 x22: ffffffc008ba65f8
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.245878] x21: 0000000000000001 x20: ffffff800785b600
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.251270] x19: ffffff8006391000 x18: 00001bf59b0dac5d
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.256661] x17: 0000000000000006 x16: ffffffc0108285d8
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.262052] x15: 0000000000000003 x14: 0000000000000d7e
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.267443] x13: 0000000000000370 x12: 00000000e0ccdeeb
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.272835] x11: 0000000000000000 x10: 000000000000008a
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.278226] x9 : 0000000000000001 x8 : 0000000000000000
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.283618] x7 : 0000000000000004 x6 : 0000000000000000
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.289010] x5 : 000000000000002a x4 : 0000000000000000
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.294402] x3 : ffffff8000cc8b40 x2 : 0000000000000000
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.299793] x1 : ffffff8000cc8b40 x0 : 00000000ffffff92
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.305186] Call trace:
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.307713]  0xffffffc008ba7af8 [mac80211@0000000080a2274c+0x79000]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.314058]  0xffffffc008ba65f8 [mac80211@0000000080a2274c+0x79000]
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.320402]  0xffffffc010045774
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.323620]  0xffffffc010045a88
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.326839]  0xffffffc01004c840
Sun Oct  8 19:41:07 2023 kern.debug kernel: [661168.330058]  0xffffffc010005f68
Sun Oct  8 19:41:07 2023 kern.warn kernel: [661168.333277] ---[ end trace 205fca1d88bcb705 ]---
type or paste code here

odrt · October 12, 2023, 9:38pm

A few caveats to using WED:

Users may experience stability issues as this is a relatively new feature

I would try switching WED off if you have issues with WED enabled.

Don’t configure WPA2-WPA3 mixed mode. I suggest to instead have two SSIDs: one for strictly WPA3 sae for devices with current software and another pure WPA2 SSID for backwards compatibility for older devices that are not WPA3 compatible. You can choose the radio for these SSIDs.

The stability problems with WPA2-WPA3 mixed mode are well known:

github.com/openwrt/openwrt

FS#3108 - Some devices unable to connect in sae-mixed mode (WPA3/WPA2 Personal)

opened 05:01PM - 18 May 20 UTC

openwrt-bot

flyspray

*Freddicus:* Device: Netgear Nighthawk X4S (R7800) Software version: OpenWrt 1…9.07.2 r10947-65030d81f3 / LuCI openwrt-19.07 branch git-20.134.55291-ba0fb08 Steps to reproduce: - Start with the firmware linked here: https://openwrt.org/toh/netgear/r7800 - Remove wpad-basic - Install wpad-openssl - Configure network to broadcast in WPA3/WPA2 Personal (sae-mixed) Devices previously able to connect to WPA2 mode: - 2012 Macbook Pro on macOS Catalina 10.15.4 - Up-to-date raspberry pi 4 on raspbian buster If I switch the radio to WPA2 Personal only (wpa-psk if not using luci, I believe), they can connect. I switch it back to WPA3/WPA2 Personal (sae-mixed) they cannot connect. Let me know what logs to gather and how to gather them. If it's a client issue, please confirm, but my understanding is that mixed mode should support all WPA2 Personal compatible devices.

eginnc · October 12, 2023, 10:10pm

My RT3200 has been OK with 802.11r (the other AP in our home is a Reyee RG E5 with similiar MT7622/7915 hardware) and the same SSID's on 2.4 and 5 GHz connected to common VLANs. Granted only my son has an Apple device (iphone), but there have been no complaints. I do not use WPA2/WPA3 mixed mode or WED, which could be why I don't have issues. As odrt wrote above, hopefully dropping WPA2/WPA3 mixed mode helps.

FWIW, with proper password selection, I think WPA2 (forced AES) is fine, but that is up to you.

_FailSafe · October 13, 2023, 2:27pm

I don't understand exactly what "switch to AX" means in your context, but the mt7915e timeout error you are seeing is suspiciously close to what I (and others) have experienced with the mt76 driver. It is currently believed to be due to a regression in WED code:

github.com/openwrt/mt76

mt7915e: Belkin RT3200 - Timeout waiting for MCU reset state

opened 06:44PM - 27 Feb 23 UTC

Fail-Safe

I just experienced a series of mt7915e crashes. Hopefully the provided stack tra…ces provide enough info to identify the issue. Build info: ``` NAME="OpenWrt" VERSION="SNAPSHOT" ID="openwrt" ID_LIKE="lede openwrt" PRETTY_NAME="OpenWrt SNAPSHOT" VERSION_ID="snapshot" HOME_URL="https://openwrt.org/" BUG_URL="https://bugs.openwrt.org/" SUPPORT_URL="https://forum.openwrt.org/" BUILD_ID="r22133-eaf5c85fd3" OPENWRT_BOARD="mediatek/mt7622" OPENWRT_ARCH="aarch64_cortex-a53" OPENWRT_TAINTS="no-all busybox" OPENWRT_DEVICE_MANUFACTURER="OpenWrt" OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/" OPENWRT_DEVICE_PRODUCT="Generic" OPENWRT_DEVICE_REVISION="v0" OPENWRT_RELEASE="OpenWrt SNAPSHOT r22133-eaf5c85fd3" ``` WED Enabled: ``` root@AP1:~# cat /etc/modules.d/mt7915e mt7915e wed_enable=Y root@AP1:~# cat /sys/module/mt7915e/parameters/wed_enable Y ``` Stack traces: ``` [215557.481510] ------------[ cut here ]------------ [215557.486224] Timeout waiting for MCU reset state 8 [215557.491043] WARNING: CPU: 1 PID: 29183 at mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215557.498888] Modules linked in: nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 cfg80211 nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon compat cls_flower act_vlan cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact cryptodev autofs4 seqiv authencesn authenc leds_gpio gpio_button_hotplug [215557.569268] CPU: 1 PID: 29183 Comm: kworker/u4:1 Tainted: G S 5.15.94 #0 [215557.577351] Hardware name: Linksys E8450 (UBI) (DT) [215557.582308] Workqueue: mt76 mt7915_mac_reset_work [mt7915e] [215557.587968] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [215557.595009] pc : mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215557.600665] lr : mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215557.606321] sp : ffffffc00ee73c50 [215557.609713] x29: ffffffc00ee73c50 x28: ffffff8002864968 x27: ffffff80028625d0 [215557.616931] x26: 0000000000000000 x25: 0000000000000000 x24: ffffff8002862020 [215557.624149] x23: 0000000000000000 x22: ffffff8002866020 x21: 0000000000000008 [215557.631366] x20: ffffff8002869d88 x19: 0000000000000000 x18: ffffffc008aba320 [215557.638583] x17: 0000000000ff1740 x16: 2d1a2d0b01036c60 x15: 0000000000000471 [215557.645800] x14: 000000000000017b x13: ffffffc00ee73978 x12: ffffffc008b12320 [215557.653017] x11: 4d20726f6620676e x10: ffffffc008b12320 x9 : 0000000000000000 [215557.660234] x8 : ffffffc008aba2d0 x7 : ffffffc008aba320 x6 : 0000000000000001 [215557.667451] x5 : ffffff801feb46f8 x4 : 0000000000000000 x3 : 0000000000000027 [215557.674668] x2 : 0000000000000027 x1 : 0000000000000023 x0 : 0000000000000025 [215557.681886] Call trace: [215557.684409] mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215557.689719] mt7915_mac_reset_work+0x218/0xd0c [mt7915e] [215557.695115] process_one_work+0x210/0x3b0 [215557.699209] worker_thread+0x170/0x4d0 [215557.703037] kthread+0x11c/0x130 [215557.706346] ret_from_fork+0x10/0x20 [215557.710002] ---[ end trace 24de3e00bec6051a ]--- [215588.201120] ------------[ cut here ]------------ ``` ``` [215588.201120] ------------[ cut here ]------------ [215588.205834] Timeout waiting for MCU reset state 20 [215588.210740] WARNING: CPU: 1 PID: 29183 at mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215588.218585] Modules linked in: nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 cfg80211 nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon compat cls_flower act_vlan cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact cryptodev autofs4 seqiv authencesn authenc leds_gpio gpio_button_hotplug [215588.288966] CPU: 1 PID: 29183 Comm: kworker/u4:1 Tainted: G S W 5.15.94 #0 [215588.297049] Hardware name: Linksys E8450 (UBI) (DT) [215588.302006] Workqueue: mt76 mt7915_mac_reset_work [mt7915e] [215588.307666] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [215588.314707] pc : mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215588.320363] lr : mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215588.326020] sp : ffffffc00ee73c50 [215588.329411] x29: ffffffc00ee73c50 x28: ffffff8002864968 x27: ffffff80028625d0 [215588.336629] x26: 0000000000000000 x25: 0000000000000000 x24: ffffff8002862020 [215588.343847] x23: 0000000000000000 x22: ffffff8002866020 x21: 0000000000000020 [215588.351064] x20: ffffff8002869d88 x19: 0000000000000000 x18: ffffffc008aba320 [215588.358281] x17: 00000000000034c0 x16: ffffffc008df5000 x15: 00000000000004c8 [215588.365497] x14: 0000000000000198 x13: ffffffc00ee73978 x12: ffffffc008b12320 [215588.372715] x11: 434d20726f662067 x10: ffffffc008b12320 x9 : 0000000000000000 [215588.379931] x8 : ffffffc008aba2d0 x7 : ffffffc008aba320 x6 : 0000000000000001 [215588.387148] x5 : ffffff801feb46f8 x4 : 0000000000000000 x3 : 0000000000000027 [215588.394365] x2 : 0000000000000027 x1 : 0000000000000023 x0 : 0000000000000026 [215588.401582] Call trace: [215588.404106] mt7915_mcu_rf_regval+0x408/0x6f4 [mt7915e] [215588.409415] mt7915_mac_reset_work+0x344/0xd0c [mt7915e] [215588.414811] process_one_work+0x210/0x3b0 [215588.418905] worker_thread+0x170/0x4d0 [215588.422733] kthread+0x11c/0x130 [215588.426041] ret_from_fork+0x10/0x20 [215588.429697] ---[ end trace 24de3e00bec6051b ]--- ``` ``` [220791.197178] ------------[ cut here ]------------ [220791.201890] WARNING: CPU: 0 PID: 1324 at ___ieee80211_stop_tx_ba_session+0x348/0x3ac [mac80211] [220791.210713] Modules linked in: nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 cfg80211 nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon compat cls_flower act_vlan cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact cryptodev autofs4 seqiv authencesn authenc leds_gpio gpio_button_hotplug [220791.281092] CPU: 0 PID: 1324 Comm: hostapd Tainted: G S W 5.15.94 #0 [220791.288654] Hardware name: Linksys E8450 (UBI) (DT) [220791.293609] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [220791.300650] pc : ___ieee80211_stop_tx_ba_session+0x348/0x3ac [mac80211] [220791.307365] lr : ___ieee80211_stop_tx_ba_session+0x210/0x3ac [mac80211] [220791.314091] sp : ffffffc008ef37a0 [220791.317484] x29: ffffffc008ef37a0 x28: ffffff8000326e00 x27: ffffffc008ef3db0 [220791.324703] x26: ffffff8000066880 x25: ffffff8002860880 x24: ffffffc008b7d000 [220791.331922] x23: ffffffc0009a0344 x22: ffffff80066780e8 x21: 0000000000000003 [220791.339139] x20: ffffff800148c100 x19: ffffff800667a000 x18: ffffffc008aba320 [220791.346357] x17: 0000000000000000 x16: 0000000000000000 x15: 0000142a12180207 [220791.353574] x14: ffffffc008bbf220 x13: 0000000000000000 x12: 0000000000000000 [220791.360791] x11: 0000000000000049 x10: 0000000000000840 x9 : 0000000000000000 [220791.368008] x8 : ffffff800166124a x7 : 0000000000000000 x6 : 0000000000000000 [220791.375225] x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffffff8000326e00 [220791.382442] x2 : 0000000000000000 x1 : ffffff8000326e00 x0 : 00000000fffffff4 [220791.389660] Call trace: [220791.392183] ___ieee80211_stop_tx_ba_session+0x348/0x3ac [mac80211] [220791.398550] ieee80211_sta_tear_down_BA_sessions+0x74/0x130 [mac80211] [220791.405176] ieee80211_find_sta_by_link_addrs+0x370/0x540 [mac80211] [220791.411628] sta_info_destroy_addr_bss+0x38/0x70 [mac80211] [220791.417298] ieee80211_color_change_finish+0x1278/0x1500 [mac80211] [220791.423664] cfg80211_check_station_change+0x1384/0x4720 [cfg80211] [220791.430027] genl_family_rcv_msg_doit+0xb4/0x110 [220791.434730] genl_rcv_msg+0xd0/0x1c0 [220791.438386] netlink_rcv_skb+0x58/0x120 [220791.442303] genl_rcv+0x34/0x50 [220791.445523] netlink_unicast+0x1f0/0x2ec [220791.449526] netlink_sendmsg+0x19c/0x3d0 [220791.453529] ____sys_sendmsg+0x258/0x2a0 [220791.457535] ___sys_sendmsg+0x78/0xc0 [220791.461276] __sys_sendmsg+0x54/0xb0 [220791.464929] __arm64_sys_sendmsg+0x20/0x30 [220791.469103] invoke_syscall+0x44/0x110 [220791.472934] el0_svc_common.constprop.0+0x48/0xf0 [220791.477717] do_el0_svc+0x18/0x20 [220791.481111] el0_svc+0x14/0x50 [220791.484246] el0t_64_sync_handler+0xe0/0x110 [220791.488594] el0t_64_sync+0x158/0x15c [220791.492336] ---[ end trace 24de3e00bec6051c ]--- ``` ``` [220798.366402] ------------[ cut here ]------------ [220798.371112] WARNING: CPU: 1 PID: 1324 at ___ieee80211_stop_tx_ba_session+0x348/0x3ac [mac80211] [220798.379934] Modules linked in: nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 cfg80211 nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c hwmon compat cls_flower act_vlan cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact cryptodev autofs4 seqiv authencesn authenc leds_gpio gpio_button_hotplug [220798.450316] CPU: 1 PID: 1324 Comm: hostapd Tainted: G S W 5.15.94 #0 [220798.457879] Hardware name: Linksys E8450 (UBI) (DT) [220798.462833] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [220798.469875] pc : ___ieee80211_stop_tx_ba_session+0x348/0x3ac [mac80211] [220798.476594] lr : ___ieee80211_stop_tx_ba_session+0x210/0x3ac [mac80211] [220798.483306] sp : ffffffc008ef37a0 [220798.486698] x29: ffffffc008ef37a0 x28: ffffff8000326e00 x27: ffffffc008ef3db0 [220798.493915] x26: ffffff8000066880 x25: ffffff8002860880 x24: ffffffc008b7d000 [220798.501132] x23: ffffffc0009a0344 x22: ffffff800644e508 x21: 0000000000000003 [220798.508350] x20: ffffff8003becb00 x19: ffffff800644a000 x18: 0000000000000000 [220798.515566] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [220798.522782] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000062467 [220798.530000] x11: 00000000000001f0 x10: 0000000000000840 x9 : 0000000000000000 [220798.537216] x8 : ffffff800673124a x7 : 0000000000000000 x6 : 0000000000000000 [220798.544433] x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffffff8000326e00 [220798.551650] x2 : 0000000000000000 x1 : ffffff8000326e00 x0 : 00000000fffffff4 [220798.558867] Call trace: [220798.561391] ___ieee80211_stop_tx_ba_session+0x348/0x3ac [mac80211] [220798.567762] ieee80211_sta_tear_down_BA_sessions+0x74/0x130 [mac80211] [220798.574387] ieee80211_find_sta_by_link_addrs+0x370/0x540 [mac80211] [220798.580839] sta_info_destroy_addr_bss+0x38/0x70 [mac80211] [220798.586510] ieee80211_color_change_finish+0x1278/0x1500 [mac80211] [220798.592876] cfg80211_check_station_change+0x1384/0x4720 [cfg80211] [220798.599238] genl_family_rcv_msg_doit+0xb4/0x110 [220798.603940] genl_rcv_msg+0xd0/0x1c0 [220798.607596] netlink_rcv_skb+0x58/0x120 [220798.611512] genl_rcv+0x34/0x50 [220798.614733] netlink_unicast+0x1f0/0x2ec [220798.618735] netlink_sendmsg+0x19c/0x3d0 [220798.622738] ____sys_sendmsg+0x258/0x2a0 [220798.626744] ___sys_sendmsg+0x78/0xc0 [220798.630485] __sys_sendmsg+0x54/0xb0 [220798.634139] __arm64_sys_sendmsg+0x20/0x30 [220798.638313] invoke_syscall+0x44/0x110 [220798.642143] el0_svc_common.constprop.0+0x48/0xf0 [220798.646926] do_el0_svc+0x18/0x20 [220798.650319] el0_svc+0x14/0x50 [220798.653454] el0t_64_sync_handler+0xe0/0x110 [220798.657802] el0t_64_sync+0x158/0x15c [220798.661545] ---[ end trace 24de3e00bec60528 ]--- ```

I have been running my RT3200s in 5Ghz@HE80 mode for two years now without any issue with AX or 802.11r, specifically. But all three of them have the MCU reset timeout crash after several hours of uptime when WED is enabled.

adworacz · October 13, 2023, 3:53pm

Anyone else seeing reduced speeds on the new official 23.05 release?

I'm on AX, channel 100, with 160Mhz, and I'm seeing a noticeable drop in speeds.

On 23.05-rc3, I was able to get pretty insane speeds (~930 Mbps, both directions) over wifi, but with this release, I'm seeing a max of around 860Mbps down and ~375Mbps up, measured using the same device (Intel AX 211) that I used to measure ~930Mbps with, from the same distance. Interestingly, my iPhone is getting ~630Mbps both directions.

The slowdown happens even when using the performance CPU governor. It's the same with ondemand as well.

I measured the speeds using OpenSpeedTest running on a NAS (connected via gigabit ethernet), as well as iperf3 (on the same NAS), and I'm getting consistent results.

Edit: This is with WED disabled, in both 23.05-RC3 and the official 23.05.