Just e-mailed several files to you. Thanks!
I meant to include some setup & environment info with my previous post.
I am running these Flent tests from a 2019 MacBook Pro, so AC 3x3 (5GHZ/VHT80). My tests are run from my office which is approx 60-70 feet from my AP and there are several walls and a floor between. My RSSI (from my MBP) is -60 to -61 dBm and noise approx -92 dBm.
I'll be happy to test closer to the AP with a clear line of sight at any point. 
EDIT I should have been more clear on the distance, I'm sitting 30-35 linear feet from the AP (60-70 feet for round-trip distance). That's what I get for multi-tasking...
Thats an amazing result. Go outside? go on the roof? somewhere where there's some interference and even more distance? Put it into a microwave? (don't turn it on!)
1 Like
Is this more along the lines of the data you were hoping to see? If so, I can switch my r7800 back over to the ath10k-ct and produce the same output:
tcp_nup test from an ubuntu VM running with a bridged adapter on my MacBook (if this isn't a valid test case, I will abort future tests from this VM):
Summary of tcp_nup test run from 2020-04-24 20:38:57.402624
Title: 'owrt_r13051_k-5.4.34_r7800_ath10k-ct-smallbuffers_run5'
avg median # data pts
Ping (ms) ICMP : 45.93 41.90 ms 1743
TCP upload avg : 91.88 N/A Mbits/s 1743
TCP upload sum : 367.50 N/A Mbits/s 1743
TCP upload::1 : 108.71 116.19 Mbits/s 1743
TCP upload::1::tcp_cwnd : 712.92 731.00 1107
TCP upload::1::tcp_delivery_rate : 82.87 77.53 1106
TCP upload::1::tcp_pacing_rate : 230.56 210.53 1106
TCP upload::1::tcp_rtt : 54.24 47.20 1104
TCP upload::1::tcp_rtt_var : 2.43 0.94 1104
TCP upload::2 : 72.29 74.34 Mbits/s 1743
TCP upload::2::tcp_cwnd : 446.15 450.00 1106
TCP upload::2::tcp_delivery_rate : 56.79 53.15 1106
TCP upload::2::tcp_pacing_rate : 147.64 136.69 1106
TCP upload::2::tcp_rtt : 50.14 46.11 1102
TCP upload::2::tcp_rtt_var : 3.26 2.01 1102
TCP upload::3 : 79.10 80.05 Mbits/s 1743
TCP upload::3::tcp_cwnd : 497.76 507.00 1106
TCP upload::3::tcp_delivery_rate : 60.78 56.64 1106
TCP upload::3::tcp_pacing_rate : 164.61 148.83 1106
TCP upload::3::tcp_rtt : 51.37 46.32 1102
TCP upload::3::tcp_rtt_var : 2.84 1.51 1102
TCP upload::4 : 107.40 114.78 Mbits/s 1743
TCP upload::4::tcp_cwnd : 696.94 711.00 1106
TCP upload::4::tcp_delivery_rate : 82.15 76.24 1105
TCP upload::4::tcp_pacing_rate : 227.36 205.70 1105
TCP upload::4::tcp_rtt : 53.77 47.29 1102
TCP upload::4::tcp_rtt_var : 2.36 0.94 1102
Summary of tcp_nup test run from 2020-04-24 20:41:16.758382
Title: 'owrt_r13051_k-5.4.34_r7800_ath10k-ct-smallbuffers_run6'
avg median # data pts
Ping (ms) ICMP : 48.09 46.30 ms 1747
TCP upload avg : 94.42 N/A Mbits/s 1747
TCP upload sum : 377.69 N/A Mbits/s 1747
TCP upload::1 : 62.24 72.22 Mbits/s 1747
TCP upload::1::tcp_cwnd : 393.63 428.00 1107
TCP upload::1::tcp_delivery_rate : 49.71 45.17 1107
TCP upload::1::tcp_pacing_rate : 127.74 111.72 1107
TCP upload::1::tcp_rtt : 49.74 46.05 1106
TCP upload::1::tcp_rtt_var : 4.11 2.87 1106
TCP upload::2 : 117.69 120.77 Mbits/s 1747
TCP upload::2::tcp_cwnd : 786.71 566.00 1105
TCP upload::2::tcp_delivery_rate : 88.69 79.37 1105
TCP upload::2::tcp_pacing_rate : 248.59 185.79 1105
TCP upload::2::tcp_rtt : 53.49 47.07 1103
TCP upload::2::tcp_rtt_var : 2.66 1.04 1103
TCP upload::3 : 120.01 133.35 Mbits/s 1747
TCP upload::3::tcp_cwnd : 814.10 922.00 1106
TCP upload::3::tcp_delivery_rate : 93.39 86.99 1106
TCP upload::3::tcp_pacing_rate : 254.06 199.27 1106
TCP upload::3::tcp_rtt : 54.23 47.45 1103
TCP upload::3::tcp_rtt_var : 2.52 0.97 1103
TCP upload::4 : 77.75 79.04 Mbits/s 1747
TCP upload::4::tcp_cwnd : 496.95 479.00 1105
TCP upload::4::tcp_delivery_rate : 61.56 56.38 1105
TCP upload::4::tcp_pacing_rate : 159.95 143.18 1105
TCP upload::4::tcp_rtt : 50.54 46.34 1103
TCP upload::4::tcp_rtt_var : 3.14 1.73 1103
rrul and rrul_be tests from my MacBook:
Summary of rrul test run from 2020-04-24 21:00:55.721746
Title: 'owrt_r13051_k-5.4.34_r7800_ath10k-ct-smallbuffers_run7'
avg median # data pts
Ping (ms) ICMP : 75.10 82.90 ms 1775
Ping (ms) UDP BE : 14.53 77.04 ms 1775
Ping (ms) UDP BK : 15.45 114.81 ms 1775
Ping (ms) UDP EF : 13.50 68.54 ms 1775
Ping (ms) avg : 29.65 N/A ms 1775
TCP download BE : 6.70 5.95 Mbits/s 1775
TCP download BK : 4.27 3.14 Mbits/s 1775
TCP download CS5 : 6.65 4.81 Mbits/s 1775
TCP download EF : 7.62 5.67 Mbits/s 1775
TCP download avg : 6.31 N/A Mbits/s 1775
TCP download sum : 25.24 N/A Mbits/s 1775
TCP totals : 213.46 N/A Mbits/s 1775
TCP upload BE : 74.40 74.15 Mbits/s 1775
TCP upload BK : 40.19 41.28 Mbits/s 1775
TCP upload CS5 : 36.80 37.70 Mbits/s 1775
TCP upload EF : 36.83 37.67 Mbits/s 1775
TCP upload avg : 47.05 N/A Mbits/s 1775
TCP upload sum : 188.22 N/A Mbits/s 1775
Summary of rrul_be test run from 2020-04-24 21:07:51.263921
Title: 'owrt_r13051_k-5.4.34_r7800_ath10k-ct-smallbuffers_run8'
avg median # data pts
Ping (ms) ICMP : 27.00 26.50 ms 1775
Ping (ms) UDP BE1 : 11.99 34.98 ms 1775
Ping (ms) UDP BE2 : 12.13 34.88 ms 1775
Ping (ms) UDP BE3 : 12.12 35.09 ms 1775
Ping (ms) avg : 15.81 N/A ms 1775
TCP download BE : 1.92 1.90 Mbits/s 1775
TCP download BE2 : 1.90 1.90 Mbits/s 1775
TCP download BE3 : 1.92 1.89 Mbits/s 1775
TCP download BE4 : 1.88 1.86 Mbits/s 1775
TCP download avg : 1.90 N/A Mbits/s 1775
TCP download sum : 7.62 N/A Mbits/s 1775
TCP totals : 409.22 N/A Mbits/s 1775
TCP upload BE : 99.04 100.08 Mbits/s 1775
TCP upload BE2 : 104.67 103.49 Mbits/s 1775
TCP upload BE3 : 94.96 95.34 Mbits/s 1775
TCP upload BE4 : 102.93 102.67 Mbits/s 1775
TCP upload avg : 100.40 N/A Mbits/s 1775
TCP upload sum : 401.60 N/A Mbits/s 1775
rrul and rrul_be tests from an ubuntu VM running with a bridged adapter on my MacBook (if this isn't a valid test case, I will abort future tests from this VM):
Summary of rrul test run from 2020-04-24 20:53:06.730024
Title: 'owrt_r13051_k-5.4.34_r7800_ath10k-ct-smallbuffers_run7'
avg median # data pts
Ping (ms) ICMP : 52.29 47.30 ms 1749
Ping (ms) UDP BE : 48.29 42.92 ms 1749
Ping (ms) UDP BK : 57.87 4.18 ms 1749
Ping (ms) UDP EF : 48.92 44.66 ms 1749
Ping (ms) avg : 51.84 N/A ms 1749
TCP download BE : 8.12 10.49 Mbits/s 1749
TCP download BK : 2.59 5.90 Mbits/s 1749
TCP download CS5 : 5.26 7.39 Mbits/s 1749
TCP download EF : 5.85 7.17 Mbits/s 1749
TCP download avg : 5.46 N/A Mbits/s 1749
TCP download sum : 21.82 N/A Mbits/s 1749
TCP totals : 236.64 N/A Mbits/s 1749
TCP upload BE : 38.15 59.00 Mbits/s 1749
TCP upload BE::tcp_cwnd : 358.24 389.00 722
TCP upload BE::tcp_delivery_rate : 49.91 46.48 721
TCP upload BE::tcp_pacing_rate : 102.11 89.86 722
TCP upload BE::tcp_rtt : 70.16 49.92 721
TCP upload BE::tcp_rtt_var : 3.70 1.75 721
TCP upload BK : 32.56 50.73 Mbits/s 1749
TCP upload BK::tcp_cwnd : 550.94 524.00 721
TCP upload BK::tcp_delivery_rate : 40.26 36.39 721
TCP upload BK::tcp_pacing_rate : 99.68 86.00 721
TCP upload BK::tcp_rtt : 117.90 88.63 719
TCP upload BK::tcp_rtt_var : 11.53 1.78 719
TCP upload CS5 : 72.34 113.42 Mbits/s 1749
TCP upload CS5::tcp_cwnd : 616.32 579.00 721
TCP upload CS5::tcp_delivery_rate : 89.96 89.25 720
TCP upload CS5::tcp_pacing_rate : 187.66 161.62 721
TCP upload CS5::tcp_rtt : 79.95 48.86 721
TCP upload CS5::tcp_rtt_var : 3.36 0.88 721
TCP upload EF : 71.77 111.81 Mbits/s 1749
TCP upload EF::tcp_cwnd : 620.60 605.00 720
TCP upload EF::tcp_delivery_rate : 90.52 88.98 719
TCP upload EF::tcp_pacing_rate : 189.32 165.37 719
TCP upload EF::tcp_rtt : 80.86 48.79 718
TCP upload EF::tcp_rtt_var : 1.91 0.90 718
TCP upload avg : 53.70 N/A Mbits/s 1749
TCP upload sum : 214.82 N/A Mbits/s 1749
Summary of rrul_be test run from 2020-04-24 21:00:13.179883
Title: 'owrt_r13051_k-5.4.34_r7800_ath10k-ct-smallbuffers_run7'
avg median # data pts
Ping (ms) ICMP : 58.38 51.20 ms 1750
Ping (ms) UDP BE1 : 72.15 5.31 ms 1750
Ping (ms) UDP BE2 : 72.25 5.39 ms 1750
Ping (ms) UDP BE3 : 72.78 5.51 ms 1750
Ping (ms) avg : 68.89 N/A ms 1750
TCP download BE : 1.92 1.91 Mbits/s 1750
TCP download BE2 : 1.90 1.88 Mbits/s 1750
TCP download BE3 : 2.05 2.27 Mbits/s 1750
TCP download BE4 : 2.38 2.21 Mbits/s 1750
TCP download avg : 2.06 N/A Mbits/s 1750
TCP download sum : 8.25 N/A Mbits/s 1750
TCP totals : 204.43 N/A Mbits/s 1750
TCP upload BE : 54.91 68.09 Mbits/s 1750
TCP upload BE2 : 43.88 56.95 Mbits/s 1750
TCP upload BE2::tcp_cwnd : 424.90 402.00 876
TCP upload BE2::tcp_delivery_rate : 38.85 36.37 875
TCP upload BE2::tcp_pacing_rate : 114.73 100.19 876
TCP upload BE2::tcp_rtt : 69.74 54.18 873
TCP upload BE2::tcp_rtt_var : 7.88 3.56 873
TCP upload BE3 : 49.60 62.40 Mbits/s 1750
TCP upload BE3::tcp_cwnd : 482.58 479.00 876
TCP upload BE3::tcp_delivery_rate : 43.96 41.99 875
TCP upload BE3::tcp_pacing_rate : 132.06 119.56 876
TCP upload BE3::tcp_rtt : 69.15 54.89 875
TCP upload BE3::tcp_rtt_var : 7.72 2.40 875
TCP upload BE4 : 47.79 61.11 Mbits/s 1750
TCP upload BE4::tcp_cwnd : 462.72 449.00 875
TCP upload BE4::tcp_delivery_rate : 42.89 40.26 875
TCP upload BE4::tcp_pacing_rate : 126.11 112.77 875
TCP upload BE4::tcp_rtt : 69.89 54.88 873
TCP upload BE4::tcp_rtt_var : 7.41 2.77 873
TCP upload BE::tcp_cwnd : 552.71 484.00 865
TCP upload BE::tcp_delivery_rate : 48.05 43.81 865
TCP upload BE::tcp_pacing_rate : 145.65 125.18 865
TCP upload BE::tcp_rtt : 71.97 55.18 863
TCP upload BE::tcp_rtt_var : 7.14 2.36 863
TCP upload avg : 49.05 N/A Mbits/s 1750
TCP upload sum : 196.18 N/A Mbits/s 1750
I will work on trying to apply your kernel patch this weekend and run some more numbers. I'm a little rusty on all the building & patching, so I am trying to get back up to speed on it at the same time.
the rrul_be bidir starvation really bothers me. 21Mbits vs 400. There's a couple potential reasons for that.
aside from that this is a valid scheme, and if you could send along these flent files I'd appreciate it.
1 Like
Is there some sort of rate limiter internally for these WiFi fixes? Why can’t be do this automagically for Ethernet? I actually went back to using my Archer C7 v4 with 19.07.2 After I unbricked think I it was done and holly crap this unit is running like champ so far. I noticed a much better difference in streaming and gaming with just piece of cake and no diffserv markings. WiFi is waaayy better now. Very pleased.
Forgive my grammar, my laptop is updating so I’m on my phone. Not a big texted lol.
ethernet, with a BQL enabled driver (most of them these days), will automatically do the right fq_codel thing at 10Mbit, 100Mbit 1Gbit, 10Gbit, 40 Gbit, etc. No shaping required, and the cpu cost is low.
the fq_codel for wifi stuff did the right thing at all rates ( this was a good day, 2016: http://blog.cerowrt.org/post/real_results/ ) on the ath9k and mt76, but we had to invent "AQL" to make it work on the ath10k chip (and iwl one day). Which is what we are testing here.
cpu expensive sqm style shaping is only required when your "line rate" from your isp is weird - like 40Mbit down, 5Mbit up.
the ath10k had too much buffering internally. AQL (and BQL) provide backpressure so fq_codel can do smart things. I gave a detailed example about how broken linux backpressure before bql in 2011 over here, i really should write it down somewhere else:
I’m simply amazed that a company has never contracted you guys to make the ultimate product. Last time I messed with my C7 was over a year ago, and I see it’s received lots of love! Makes my 3200acm seem puny now.
It's genuinely apalling that a dinky and cheap Archer C7/A7 is able to beat the Linksys mwlwifi devices in terms of WiFi stability/quality of service.
Ever since I've been running a 2x Archer A7 setup and I just removed the antennas and disabled the WiFi on my 1900ACS (pure router/firewall at this point), everyone's been happier in my household.
Video conference calls without having to tell others to stop watching youtube/netflix or downloading used to be a pipe dream.
Thanks to everyone on the bufferbloat team again. @dtaht @tohojo @ldir and https://github.com/chromi and the guys in Google's chromium team where AQL was first written.
Speaks to the power of getting drivers upstreamed and having a dedicated open source team for the drivers (even if they don't communicate that well esp. when it comes to firmware matters).
4 Likes
cherrypicking a bit, it looks like my 5ms codel
for 5gz... won.
However as to why it won kind of implies we ran out of some other major resource in the kernel with the original target 20 setting. Packet captures and some better logging are in order.
Still.... dancing:
3 Likes
They did. 8 months before we were done, gfiber laid us off. Been limping around ever since. But we were handed a really lousy cpu to start with (comcerto c2000) and it got worse from there.
Next time someone asks me to build the ultimate product, I get to pick the chipset up front, and get full source code to the whole stack.
What gfiber ships today is actually pretty good, but, oh, do I have ptsd from that project.....
3 Likes
I've seen this bql monitor before, looks interesting
bqlmon is useful. I wish gro didn't affect BQL so much and that BQL itself, revised to scale to multiple hw queues better. But not in this context for wifi. I'm thinking I need to look over the NAPI implementation next.....
1 Like
` Returning to the subject of this thread, "AQL and the ath10k is
lovely", I'm 100% convinced that it's a fact. It is lovely,
including with Batman.
For reasons I do not understand, I am unable to make satisfactory
tests with Flent/netperf. Instead of test results, here's an
anecdote based on visual observation.
On a pre-AQL build and on a post-AQL build of OpenWRT, I ran the
same home-brew test, which was to view in real time a ping job to
google.com and, after watching it a while, to begin simultaneously
downloading a large file into the same host that was doing the
pinging. Although the two setups were not identical, the
performance contrast was satisfactorily demonstrated, I believe.
On the pre-AQL setup, the pings were *fairly* consistently around
20ms until the download started. Then they became much slower and
much less consistent, typically running around 80ms and as high as
1300ms. The download job was an rsync with the --progress flag so
I could see the consistency of the download speed. The download
speed was very inconsistent, varying between 100 and 700
kilobytes/sec. (I have been dismayed by this kind of performance
for a long time now.)
On the post-AQL batman setup, the pings were consistently around
17ms, regardless of whether the download job was running. The
progress of the download job was also consistent, staying around
550 kilobytes/sec.
It was a visually stunning contrast that made me involuntarily
shout "Wow". My wife came in to watch, and was similarly
impressed.
`
1 Like
A lot more than just aql changed between these benchmarks,
and one of the causes of improved latency is we're starving the txop grant process a bit and thus hurting bandwidth a bit in the bidir scenario especially.
I'm happy, it's good enough to backport to stable (inc the batman flow_dissect patch), but we can do better.
But that might take more time than I have.
2 Likes
Is this something that may actually happen for a 19.07 release? I'd love to take advantage of AQL on my home network without the risk of upsetting family that comes with running a nightly build.
we can certainly say the ath10k's prior behavior was a massive bug.
This fix...
The reduced codel target is helping kill another bug but only the first 1/3 of the patch should be used. The righter number for the 5ghz wifi codel target is probably around 6-8ms, but (I think and could be wrong!) the behavior we had at target 20 was proably due to running out of queue in general, as is the throughput problem on a 3x3 chip...
For better batman, also need that flow dissector patch. I'd really like more of the qca firmware were tested. I dunno what the schedule for 19.07.3 is, nor 20. @nbd - alive? @jow ?
before I quit for the day I seem to remember someone a few years back, attempting to add NAPI to the ath10k, and I'm afraid to look
if its in there or not. Dumb idea for wifi...