Wifi interfaces down on speedtest upload

jmanuelmri · May 30, 2024, 12:00pm

I'm having one weird issue.

I have 2 dumb AP, one is Asus TUF AX6000, the other one Asus TUF AX4200.
On both I have same issue, I've on both one wifi SSID in 2.4ghz which works just fine. Also have on both one SSID with multiple vlans (filtering vlan by client password). I did remove wpad-basic-mbedtls (which was the one installed on my router by default) and installed wpad-mbedtls in order to do the vlan filter by password.

Everything works, but when I'm connected to the 5ghz SSID on any of the routers, whenever I perform a speedtest, download test is fine, but on upload test wifi kicks my device out, and both wifi interfaces (2.4ghz and 5) in the router restart, so wifi dissapear for like 10 seconds.

I've tested turning off wifi in one of the dump ap, just in case there was any conflict with SSID, still same issue.

If I use one SSID for each band and only vlan, everything works as expected.

Here is what I catch in the log:

Thu May 30 13:49:09 2024 daemon.info hostapd: phy0-ap0: STA 44:3d:54:01:xx:xx IEEE 802.11: authenticated
Thu May 30 13:49:09 2024 daemon.info hostapd: phy0-ap0: STA 44:3d:54:01:xx:xx IEEE 802.11: associated (aid 1)
Thu May 30 13:49:09 2024 daemon.notice hostapd: phy0-ap0: AP-STA-CONNECTED 44:3d:54:01xx:xx auth_alg=open
Thu May 30 13:49:09 2024 daemon.info hostapd: phy0-ap0: STA 44:3d:54:01:xx:xx RADIUS: starting accounting session D94752AE56061EB7
Thu May 30 13:49:09 2024 daemon.info hostapd: phy0-ap0: STA 44:3d:54:01:xx:xx WPA: pairwise key handshake completed (RSN)
Thu May 30 13:49:09 2024 daemon.notice hostapd: phy0-ap0: EAPOL-4WAY-HS-COMPLETED 44:3d:54:01:xx:xx
Thu May 30 13:49:52 2024 kern.err kernel: [ 1995.245309] mt798x-wmac 18000000.wifi: Message 000026ed (seq 14) timeout
Thu May 30 13:50:12 2024 kern.err kernel: [ 2015.703323] mt798x-wmac 18000000.wifi: Message 00005aed (seq 15) timeout
Thu May 30 13:50:33 2024 kern.err kernel: [ 2036.161134] mt798x-wmac 18000000.wifi: Message 000026ed (seq 1) timeout
Thu May 30 13:50:33 2024 kern.err kernel: [ 2036.167760] mt798x-wmac 18000000.wifi: Message 000026ed (seq 2) timeout

Any ideas?

brada4 · May 30, 2024, 1:30pm

And what is that client you are using that does not ack packets under load?

jmanuelmri · May 30, 2024, 1:42pm

I tested with different clients, mainly iPhone15, Macbook pro OSX, and Windows 10 laptop, it happens on all of them. Should there be ack packets in that log?

brada4 · May 30, 2024, 1:49pm

Seems related to https://github.com/openwrt/mt76/issues/690 ?

brada4 · May 30, 2024, 3:55pm

Do you have WED or offloads enabled?

jmanuelmri · May 31, 2024, 9:35am

I don't know if the issue is the same, I search on the forum before posting, the only post with exactly same issue I found is this one: BPI-R3 multi_ap WDS issue (device crash)

but Im not even using wds.

I don't have WED enabled. I have hardware offloading enabled on main router, but I have the problem on the dump aps attached to that router, not on the router itself, since I'm not doing multi vlan SSID on main router.

brada4 · May 31, 2024, 9:49am

Please post
/etc/config/wireless
and
ubus call system board

wds is not good with offloads or wed,

jmanuelmri · June 1, 2024, 8:20am

Here is the config:

/etc/config/wireless
config wifi-device 'radio0'
	option type 'mac80211'
	option path 'platform/soc/18000000.wifi'
	option channel '8'
	option band '2g'
	option htmode 'HE20'
	option country 'ES'
	option cell_density '0'
	option txpower '19'

config wifi-iface 'default_radio0'
	option device 'radio0'
	option network 'IoT'
	option mode 'ap'
	option ssid 'ssidIOT'
	option encryption 'psk2'
	option key 'censored'

config wifi-device 'radio1'
	option type 'mac80211'
	option path 'platform/soc/18000000.wifi+1'
	option channel '120'
	option band '5g'
	option htmode 'HE80'
	option country 'ES'
	option cell_density '0'
	option txpower '20'

config wifi-iface 'default_radio1'
	option device 'radio1'
	option mode 'ap'
	option ssid 'ssidvl30'
	option encryption 'psk2'
	option key 'censored'
	option network 'GUEST'

config wifi-vlan
	option name 'vl5'
	option network 'LOCAL'
	option vid '5'
	option iface 'default_radio1'

config wifi-station
	option key 'censored'
	option vid '5'
	option iface 'default_radio1'

config wifi-vlan
	option name 'vl30'
	option network 'GUEST'
	option vid '30'
	option iface 'default_radio1'

config wifi-station
	option key 'censored'
	option vid '30'
	option iface 'default_radio1'

config wifi-vlan
	option name 'vl9'
	option network 'MAIN'
	option vid '9'
	option iface 'default_radio1'

config wifi-station
	option key 'censored'
	option vid '9'
	option iface 'default_radio1'

ubus call system board


        "kernel": "5.15.150",
        "hostname": "OpenWrt-Salon",
        "system": "ARMv8 Processor rev 4",
        "model": "ASUS TUF-AX6000",
        "board_name": "asus,tuf-ax6000",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "23.05.3",
                "revision": "r23809-234f1a2efa",
                "target": "mediatek/filogic",
                "description": "OpenWrt 23.05.3 r23809-234f1a2efa"
        }
}

As I said, wds and wed are disabled in router and in dumb aps (no firewall on dumb aps). The config posted is from one of the dump ap, but this happens on both.

HW offloading is enabled in main router. Everything works, just wifi fails on upload speedtest in dumb aps, I can not reproduce it if I don't saturate wifi network.

brada4 · June 1, 2024, 8:26am

It looks like issue 690, check that packet steering is enabled and actually yields almost same load on all CPU cores. Maybe irqbalance can help offload CPU0 if only that tops the lines.

jmanuelmri · June 1, 2024, 8:42am

Enable packet steering on dumb ap's you mean? It's disabled atm in all, router and dumb ap's.

brada4 · June 1, 2024, 8:43am

You could buy cheaper router for that
Examine with htop.

jmanuelmri · June 1, 2024, 9:03am

I checked with htop, even with packet steering disabled, CPU cores doesnt even hit 15%, all cores are around 0-10% when im speed testing, so I don't think it's an overload problem in CPU0.

LilRedDog · June 1, 2024, 9:12am

Show us the results.

jmanuelmri · June 2, 2024, 9:53am

Here are the results:

Packet steering disabled:

Packet steering enabled:

I think it's pretty much the same. Would say it crashed even faster with packet steering enabled.

brada4 · June 2, 2024, 10:03am

Try snapshot -> backup conf -> flash snapshot -> check
If no improvement -> sysupgrade back with conf reset, restore initial conf.

In htop press f2->disable hide kernel threads, enable cpu details. Should be minimal pink band representing softirq on the right side of currently red "kernel" cpu usage.

jmanuelmri · June 3, 2024, 6:47pm

Okay, did everything.
Tried with snapshop, the problem persist, about the same results in htop.

I did sysupgrade back and then restore with the config, disabled hide kernel threads, and "enable cpu details" was not an option, I enabled Detailed cpu time (which I assume it is what u mean).

Here are the results while speedtesting:

brada4 · June 3, 2024, 7:10pm

Interrupts and other kernel work is well balanced. nothing to add there. Somebody wiser should say what parameters to change to make driver emit more debug data in logs.

jmanuelmri · June 4, 2024, 1:25pm

Thanks for your help, we tried. Maybe someone can tell us more about this thing.