AQL and the ath10k is *lovely*

ParanoidZoid · 2020-04-04T09:17:47.564Z

There must be a client connected to the 5Ghz/ath10k controlled phy. The command also assumes that phy0/wlan0 is the 5Ghz/ath10k interface

I'm running this on an Archer A7v5 (equivalent hardware to the Archer C7v5, with just a different flash layout).

shaarkys · 2020-04-04T12:42:32.954Z

Thank you, you were right, first client connected and it's ok now.

cat /sys/kernel/debug/ieee80211/phy0/netdev:wlan0/stations/*/aql

Q depth: VO: 0 us VI: 0 us BE: 0 us BK: 0 us
Q limit[low/high]: VO: 5000/12000 VI: 5000/12000 BE: 5000/12000 BK: 5000/12000

SteveNewcomb · 2020-04-04T14:28:56.000Z

I'd love to know how to provide them to you; please feel free to suggest
relevant readings.

What I can tell you for sure is that throughput on our meshes varies
wildly, constantly and fairly nonsensically, I assume because of
poorly-managed bufferbloat. Moreover, unless nothing else is going on
on the mesh, Google Voice telephony is pretty frustrating for users, I
assume because its protocol simply drops hoards of packets that have
been delayed too long.

(I think the "classic" ath10k driver causes other annoying events
probably unrelated to bufferbloat. The Candela Technologies version
(ath10k-ct) is described as having fixed certain problems that plausibly
explain at least some of these annoying events, so I'm eager to try
ath10k-ct whenever I learn how to make it work.)

dana44 · 2020-04-04T15:03:25.099Z

Where have you found information on the 20 release time line? I'm not even sure which kernel it'll use. It looks like 4.19 instead of 5.4 to produce a build quicker?

I'm very excited to jump in and help test aql too.

Pepe · 2020-04-04T16:15:34.458Z

dana44:

Where have you found information on the 20 release time line? I'm not even sure which kernel it'll us

These details all on the mailing list openwrt-adm.
http://lists.infradead.org/pipermail/openwrt-adm/2020-February/001316.html and here is the link where it all started: http://lists.infradead.org/pipermail/openwrt-adm/2020-January/001253.html

dtaht · 2020-04-04T17:02:14.440Z

I had basically given up on the ath10k before now. I look forward to hearing your results with batman. The test I use is flent.org's rrul_be test primarily, but you can get more information about the characteristics of your existing and under test links with a test like:

flent -H some_server_on_the_other_side -t sometitle --te=upload_streams=4 --socket-stats tcp_nup

While flent is available for most linuxes nowadays, the most current version can be found on github. The client depends on python3, the server only needs netperf (but irtt is helpful also).

SteveNewcomb · 2020-04-06T18:52:13.000Z

Looking into Flent. Fascinating and very promising. I keep seeing
dthat's name on relevant papers, along with that of Toke
Høiland-Jørgensen. I must say that the graphics in flent dot org slash
flent-the-flexible-network-tester dot pdf are exceedingly compelling
with respect to fq-codel. Wow.

I think I've made Flent begin to work now. First come are some notes
about installation, in case they save anybody any time, then my question.

I'm running Debian Buster (current "stable") on my personal workstation
(laptop). It has flent:

# apt install flent

but Debian Buster evidently doesn't offer netperf, even in "non-free".
Go figure. At Debian's no-such-command advice, I installed netperfmeter
but it didn't help:

# flent rrul -H rpc160
Started Flent 1.2.2 using Python 3.7.3.
Starting rrul test. Expected run time: 70 seconds.
ERROR: Runner TCP upload BE failed check: No netperf binary found in PATH.

I should have followed dthat's advice to begin with. Now I did so,
having read the wry non-Gnu-license statement attached to the relevant
github repository:

# apt install iperf texinfo irtt

# git clone https://github.com/HewlettPackard/netperf.git
# cd netperf
# ./autogen.sh
# ./configure --enable-demo
# make
# make-install

(The make failed without the texinfo package)

On the routers that are serving as batman-adv nodes, I installed:

# opkg install netperf flent-tools

...which seemed to provide and daemonize netserver as well.

Then I got this:

# flent rrul -H rpc160
Started Flent 1.2.2 using Python 3.7.3.
Starting rrul test. Expected run time: 70 seconds.
WARNING: Program exited non-zero.
Runner class: NetperfDemoRunner
Command: /usr/local/bin/netperf -P 0 -v 0 -D -0.20 -4 -Y EF,EF -H rpc160 
-p 12865 -t UDP_RR -l 70 -F /dev/urandom    -- -e 2  -H rpc160 -k 
THROUGHPUT,LOCAL_CONG_CONTROL,REMOTE_CONG_CONT\
ROL,TRANSPORT_MSS,LOCAL_TRANSPORT_RETRANS,REMOTE_TRANSPORT_RETRANS,LOCAL_SOCKET_TOS,REMOTE_SOCKET_TOS,DIRECTION,ELAPSED_TIME,PROTOCOL,LOCAL_SEND_SIZE,LOCAL_RECV_SIZE,REMOTE_SEND_SIZE,RE\
MOTE_RECV_SIZE
Return code: 255
Stdout: establish control: are you sure there is a netserver listening 
on rpc160 at port 12865?
establish_control could not establish the control connection from 
0.0.0.0 port 0 address family AF_INET to rpc160 port 12865 address 
family AF_INET

...which was a bit baffling, at least for a moment there, because

# ssh rpc160
@rpc160:/root# ps w | grep netserver
  6307 root      1104 S    /usr/bin/netserver
  6428 root      1216 S    grep netserver
@rpc160:/root# netstat -nap | grep 12865
tcp        0      0 :::12865                :::* LISTEN      6307/netserver

...but it turned out to be a firewall-hole-punching issue. To
/etc/config/firewall I added:

## netserver
config rule
         option name             netserver-flent
         option src              wan
         option proto            udp
         option dest_port        12865
         option target           ACCEPT
         option family           ipv4

...restarted the firewall, and it seemed to work:

# flent rrul -H rpc160
Started Flent 1.2.2 using Python 3.7.3.
Starting rrul test. Expected run time: 70 seconds.
Data file written to ./rrul-2020-04-06T142915.863247.flent.gz.
Summary of rrul test run at 2020-04-06 18:29:15.863247:

                              avg       median          # data pts
  Ping (ms) ICMP   :      6105.98      4307.50 ms              114
  Ping (ms) UDP BE :       134.91         1.72 ms               71
  Ping (ms) UDP BK :        60.59         2.07 ms               51
  Ping (ms) UDP EF :        86.49       146.41 ms              169
  Ping (ms) avg    :      1596.99       379.45 ms              312
  TCP download BE  :         0.11         0.11 Mbits/s          76
  TCP download BK  :         0.03         0.05 Mbits/s          34
  TCP download CS5 :        31.53        31.49 Mbits/s         291
  TCP download EF  :        30.95        31.00 Mbits/s         289
  TCP download avg :        15.65        21.13 Mbits/s         300
  TCP download sum :        62.62        62.72 Mbits/s         300
  TCP totals       :        63.72        62.96 Mbits/s         302
  TCP upload BE    :         0.02         0.03 Mbits/s           2
  TCP upload BK    :         0.01         0.01 Mbits/s           2
  TCP upload CS5   :         0.53         0.35 Mbits/s          25
  TCP upload EF    :         0.54         0.35 Mbits/s          25
  TCP upload avg   :         0.28         0.34 Mbits/s         196
  TCP upload sum   :         1.10         0.34 Mbits/s         196

This was a testbed mesh consisting of 2 nodes both running in the same
room, running

# uname -a
Linux rpc160 4.19.108 #0 Sat Apr 4 10:28:18 2020 mips GNU/Linux

which was compiled from OpenWRT trunk two days ago.

But I have two batman-adv meshes in production (see rosepark dot us hash
map). I believe they have serious bufferbloat problems. Dave, you want
batman-adv data. What would you like me to do now?

Steve Newcomb

dtaht · 2020-04-06T21:11:53.220Z

hoo, boy is that miserable!

the rrul_be test should be mildly better because it only fills the best effort hw queue.

yer gonna love aql. even then I'd suggest totally disabling wifi 802.11e either with the qos map or remarking most packets to diffserv 0 or both.

dtaht · 2020-04-06T21:48:48.468Z

I like to look at the output flent.gz files. The flent-gui lets you do comparison plots in particular.

muddyfeet · 2020-04-07T13:16:56.795Z

Have just tried the latest snapshot on two archer C7 v2 access points. The change is indeed lovely! Latency has come down when doing a dslreports speedtest from 100-200ms to around 25ms on the ath10k radio.

http://www.dslreports.com/speedtest/61820776

Thank you to all who made this happen.

dtaht · 2020-04-07T18:27:06.267Z

yea!

I'd really like more folk to try one tiny tweak for 5ghz: Reducing the codel target to 6ms. IMHO Really should "just work". requires a one line patch.

Also aql_threshold can be fiddled with in sysfs.

baxobox · 2020-04-07T19:34:07.595Z

What is the last snapshot? Or did they compile themselves?

alexatkinuk · 2020-04-08T05:14:44.385Z

Does this apply to all ath10k devices and do you need to explicitly install the -ct variant?

I've not really been paying attention to the new drivers so have no idea what its all about, but I have a C7 and a Fritzbox 7530, both of which could benefit from this.

Gingernut · 2020-04-08T07:31:42.434Z

AFAIK all ath10k devices should be using by default the -ct variant.

JonP · 2020-04-08T20:34:36.709Z

I too am seeing that kind of improvement, on my C7 v3 (US v3, has no 2.4ghz int antennas) on v2 April 4 snapshot.

Prior performance on 5ghz was 20-30ms only with very strong signal and highest data rates, but with a bit of signal drop 100-200ms was the normal. As opposed to the ath9k 2.4ghz being in the 20's almost till loss of connection.

Now, am happily seeing 20-40ms on the ath10k 5ghz radio, far across the house... that is, inbetween times of my cable provider ISP itself falling apart under heavy stay at home Netflix binge primetime loading.... was seeing 10% packet loss to 8.8.8.8 the other day!

Dave... could you or someone publish a dummies tweeking guide, so this dummy and others could more easily try different settings?

dtaht · 2020-04-10T20:48:23.900Z

There's very very little to set or twiddle with. It mostly just works (in combination with other really complicated things) I've long recommend folk make other mods to their wifi on top of this for years, however those are generic enough to apply to any fq_codeled chip, not just this one, and are subject to my biases for extremely low latency vs bulk throughput. (use a qos_map to just use BE, reduce txop size to 3ms or less, in both the beacon and driver, reduce hw retries, disable b rates) and I keep hoping someone will believe me when I say this makes wifi massively better for gaming, web traffic, in the face of multiple devices, interference, and bad links.

Specific to aql on the ath10k, for this thread.

If you are running at low rates (HT20 in my case, 28Mbits typica; throughput), reducing the aql_threshold to 6k is helping. I'd actually like this to autoconfigure.... but need field data.

If you are low on cpu and operating at high rates, you can twiddle the aqm quantum. 300 is a good number below 200mbit.

Gingernut · 2020-04-11T07:05:30.701Z

package / kernel / mac80211 / patches / subsys / 310-mac80211-Implement-Airtime-based-Queue-Limit-AQL.patch

You mean to edit the values at build time here?

Router · 2020-04-11T07:21:22.519Z

Gingernut:

AFAIK all ath10k devices should be using by default the -ct variant.

I get better much throughput with the non-ct variant on my C7v2.

Gingernut · 2020-04-11T07:51:06.066Z

Have you tested latest snapshots since airtime fairness patches were ported to ath10k-ct also?

dtaht · 2020-04-11T21:29:35.187Z

That's kind of a confusing statement. I found the ath10k essentially unusable at low speeds, but I have not got around to trying for more lab-like high speed results as yet...

I got scared by the 5.4 kernel upgrade that happened mere days after this wonderful first test. I have limited time in life to debug general kernel issues (has anyone tried the 5.4 kernel on the ubnt and archer platforms yet?).