AQL and the ath10k is *lovely*

So I built and loaded the ath10k driver commenting out the line
wiphy_ext_feature_set(ar->hw->wiphy, NL80211_EXT_FEATURE_AQL);
in mac.c

Running with that, there are no aql files in
/sys/kernel/debug/ieee80211/phy0/netdev\:wlan0/stations/*/aql when stations are connected to phy0; however,
cat /sys/kernel/debug/ieee80211/phy0/netdev\:wlan0/stations/*/airtime
still returns an output.

Repeating the tests above "sans aql" gives similar results so it seems my testing/observations are more airtime fairness related than aql.

What makes a bigger difference is client/AP location (AQL enabled in the ath10k driver this case). Airtime fairness seems to behave more like I'd expect when all clients have a line of sight to the AP. i.e. the faster client stay fast while slower clients take a hit. Even then, I'm still surprised at how much of a bandwidth hit the slow clients take when I start simultaneous netperf sessions from 2-3 clients. It still seems like some clients come to a complete stop briefly. This may happen more frequently with increased router uptime - I'll need to test over an extended period.

As an aside, most testing I've done is with 80 MHz width channels on the 5 GHz band (channel 36). I tried with 40 MHz width and generally see similar behavior just not as pronounced.

Lastly, I'd really like to adjust the airtime station weights with a command like

r7500v2 # iw dev wlan0 station set <MAC address> airtime_weight 26
command failed: Not supported (-95)

but I only get the output above. Perhaps this hostapd commit will enable that (but I'm likely just misunderstanding).

There hasn't been much video conferencing going on in my household since the lastest mac80211 updates so I'll need to give this some time to see if it's really improved now or not. That and I'd still like to try a non-ct firmware just to see if makes a difference.

If you'd like to compile ath10k without airtime fairness at all try commenting out the following:

NL80211_EXT_FEATURE_AIRTIME_FAIRNESS

I think it's lines 9260-9263 in mac.c in the ath10k driver for Linux/Backports version 5.8.

Thanks, I probably will. I assume doing so returns the airtime scheduler to what it was before (round robin) vs the new thing (virtual time-based).

It looks like there is a hostap-ct that has some form of "airtime config" option. I haven't looked into that yet - porting a package over to openwrt likely would take some effort.

It will completely disable the airtime fairness scheduler. The virtual time-based solution was never adopted and merged.

You will get this:

Rather than this:

Development of the virtual time solution stopped at v5 of the patch as seen here: https://patchwork.kernel.org/patch/11307545/

Looking into the author's history it was also the last thing they've submitted: https://patchwork.kernel.org/project/linux-wireless/list/?submitter=184429&state=*&archive=both

2 Likes

Was it effectively abandoned? Or will it be in the OpenWrt kernel for the 2020 release?

1 Like

hmm, I'm missing something...

after

        /* if (ath10k_peer_stats_enabled(ar) || */                                
        /*     test_bit(WMI_SERVICE_REPORT_AIRTIME, ar->wmi.svc_map)) */          
        /*      wiphy_ext_feature_set(ar->hw->wiphy, */                           
        /*                            NL80211_EXT_FEATURE_AIRTIME_FAIRNESS); */   

in ath10k-5.4/mac.c, compiling, moving the resulting ath10k_core.ko
to /lib/modules/5.4.63/ath10k_core.ko and rebooting I still see an output from
cat /sys/kernel/debug/ieee80211/phy0/netdev\:wlan0/stations/*/airtime
and I can still reproduce the symptoms above.

I think I need to start a new thread specific to my device as this does not seem "AQL" related. Also I may have made some progress finding more up to date non-ct ath10k firmware to try (but I'll likely have to use the ath10k-ct driver).

2 Likes

it looks like your search ended in 2019. It looks like work has continued into 2020 https://lore.kernel.org/linux-wireless/?q=Toke+Høiland-Jørgensen

Has anyone else observed this same issue (https://lists.bufferbloat.net/pipermail/make-wifi-fast/2020-September/002933.html) where Airtime Fairness doesn't play well with AQL?

2 Likes

I note that I kind of dropped out of openwrt dev for the past year. What's the status of the AQL patches in mainline presently? And this patch?

1 Like

@tohojo Hi! What's the latest news on this?

I did rebase the patch fairly recently and sent it off to Felix for testing before posting on the list. He hasn't gotten back to me, but I assume it's on his backlog.

I do plan to do a bit more testing myself, but need to set up an environment for that (I'm awfully short on clients here). If you wanna volunteer for testing I can send you the patch, otherwise y'all will just have to wait :wink:

1 Like

Sure, I can probably try it out, and I imagine others here on the forum would like to as well. Could you post the patch here?

What kind of tests would you recommend doing?

2 Likes

I'd also like to try this out.

https://github.com/tohojo/openwrt/blob/virtual-airtime/package/kernel/mac80211/patches/subsys/346-mac80211-Switch-to-a-virtual-time-based-airtime-sche.patch

Only compile-tested on OpenWrt

Airtime fairness-related ones? :wink:

Generally, testing if fairness is worse/better than without the patch, and checking that it doesn't break anything (such as stations stalling entirely, that sort of thing)

3 Likes

@tohojo Just to double check, is this the right way to do it: I copied the patch to /package/kernel/mac80211/patches/subsys/ and compiled an image.

Running an 8-stream TCP download with flent totally stalled out my laptop and my smartphone on the network. Here's the graph from the test:

I am also seeing the same thing on an access point with equipped with a QCA 9880 (AC Wave 1) chipset.

Right, that was the kind of failure mode I was afraid of (and why I haven't posted the patch again yet). Will try to get some stations setup and do some testing myself...

1 Like

This conversation is kind of about 3 different things. ATF, reducing the fq_codel target more generally, and interactions with various bits of ath10k firmware. I'd like to get cracking on this again in the coming month, would like it if we standardized on a 10ms target universally, to start with. Then I don't know what ath10k firmware is most common, my take on it is the smallbuffers one is too small for higher rates. What does the -htt one do?

As for ATF, goferit toke... :slight_smile:

Longer term I do wish we had a good way of rate limiting multicast and even more importantly, limiting retries, on this firmware at lower rates. Any clues to be had here?

The HTT-MGT firmware transports management frames over the
normal HTT tx path instead of over wmi.

What is the tradeoffs of that? What do other platforms like ath9k or mt76 do?