AQL and the ath10k is *lovely*

@dtaht Is netperf-east.bufferbloat.net alive? I tried running Flent test to it from "my" setup here in USA (not the Macbook sitting in India).

Flent log for netperf-east.bufferbloat.net : https://gist.github.com/KA2107/87dac2b00a882db431f4c86dcffea01a

Router/AP: Belkin RT3200 running OpenWrt 22.03.0-rc5
ISP: Comcast Xfinity (USA), DOCSIS 3.1, 50/10 Mbps Plan, IPv4 + IPv6
Location: (near) Chicago, Illinois, USA

SQM on WAN: CAKE + layer_cake.qos, Download 50150, Upload 11134, Ingress options "ingress docsis besteffort nat dual-dsthost", Egress options "egress docsis diffserv4 nat dual-srchost ack-filter", no other overhead options defined

WIFI Settings: 5 GHz, Band 165, HE20, WPA3-SAE H2E, 802.11w mandatory, Beacon Interval 100, DTIM Interval 5

Client: Lenovo Thinkpad X1 Yoga
OS: Arch Linux x86_64
WIRED: Intel Ethernet I219-V
WIFI: Intel Wireless AC-9560 160 MHz (only upto 802.11ac / WiFi 5, does not support 802.11ax / WiFi 6)

Client and router are in the same room.

rrul_be test WIRED result for flent-newark.bufferbloat.net:

rrul_be-2022-07-10T221802.426248.flent_

rrul_be test WIFI result for flent-newark.bufferbloat.net:

rrul_be-2022-07-10T223716.028943.flent_

Correct. But, I'd like to insist rc5 minus patches 334 and 335. This is something to keep in consideration.

It's the only Linux box I can stand up. The idea is to run Linux bridging the WiFi interface. This is just because you are sharing your weariness of macOS.
Apologies, sometimes I forget people don't know what I'm thinking of.

1 Like

I am very happy we can actually test via OSX. I drowned my 2013 air over a year ago, and disliked the m1 replacement I got so much as to give it back. Happy also that flent works again. happy also to see solid if less than perfect long term results and no crashes and having y'all be so interested in helping.

3 Likes

you get a faster download from wifi than wired? :boggle: Don't know how that is possible. hmm...

I'm not responsible for netperf-east. atlanta, dallas, fremont, toronto (.starlink.taht.net) I maintain. I'll ping rich..

what's the ethernet driver on the yoga?

# lspci -v

00:14.3 Network controller: Intel Corporation Comet Lake PCH-LP CNVi WiFi
	Subsystem: Intel Corporation Device 0030
	Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 7
	Memory at ea238000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [c8] Power Management version 3
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [80] MSI-X: Enable+ Count=16 Masked-
	Capabilities: [100] Latency Tolerance Reporting
	Capabilities: [164] Vendor Specific Information: ID=0010 Rev=0 Len=014 <?>
	Kernel driver in use: iwlwifi
	Kernel modules: iwlwifi

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (10) I219-V
	Subsystem: Lenovo Device 2292
	Flags: bus master, fast devsel, latency 0, IRQ 167, IOMMU group 12
	Memory at ea200000 (32-bit, non-prefetchable) [size=128K]
	Capabilities: [c8] Power Management version 3
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Kernel driver in use: e1000e
	Kernel modules: e1000e

I ran the RRUL BE tests against 3 servers. The test was repeated twice for each server. The router R7800 runs 22.03.0 RC5 image with NSS acceleration and without SQM enabled. ISP D/L speed ~ 650 Mbps / 220 Mbps. Router and client are about 50ft away from each other, with 3 dry walls in between.

SPEEDTEST (same client location as when running flent)

speedtest


SERVER: locally running on the router (R7800, performance governor)



SERVER: netperf-west.bufferbloat.net



SERVER: flent-fremont.bufferbloat.net



2 Likes

thx @vochong - wow - gang's all here! too many variables under test. Any chance you have a seperate server you can hook up to the lan or wan port you can put netperf on, so we can just test the wifi? Or can you put the flent client on the lan port so we can just test the isp link?

The period in your second local test is suggestive...

but the rest is so darn noisy.

1 Like

Wow! Looks like today I'm in a roll...

21.02.1 baseline:

Summary of tcp_ndown test run from 2022-07-11 05:12:50.762651
  Title: 'flent_tcp_ndown_60s_openwrt_v21.02.1_mt76'

                             avg       median          # data pts
 Ping (ms) ICMP   :        19.80        21.50 ms             1400
 TCP download avg :       141.10          N/A Mbits/s        1400
 TCP download sum :       564.41          N/A Mbits/s        1400
 TCP download::1  :       157.18       151.26 Mbits/s        1400
 TCP download::2  :       144.03       144.43 Mbits/s        1400
 TCP download::3  :       124.73       129.23 Mbits/s        1400
 TCP download::4  :       138.47       139.84 Mbits/s        1400

flent tcp_ndown 4x streams 21.02.1 (click me to download)

22.03.0-rc (minus patches 334 and 335):

Summary of tcp_ndown test run from 2022-07-11 05:23:29.328235
  Title: 'flent_tcp_ndown_60s_openwrt_v22.03.0-rc5_mt76'

                             avg       median          # data pts
 Ping (ms) ICMP   :        22.30        23.85 ms             1400
 TCP download avg :       113.68          N/A Mbits/s        1400
 TCP download sum :       454.72          N/A Mbits/s        1400
 TCP download::1  :       114.57       115.62 Mbits/s        1400
 TCP download::2  :       108.56       111.56 Mbits/s        1400
 TCP download::3  :       118.48       118.65 Mbits/s        1400
 TCP download::4  :       113.11       114.43 Mbits/s        1400

flent tcp_ndown 4x streams 22.03.0-rc5 (minus patches 334 and 335) (click me to download)

Flent client was connected (1000Mb/s, full duplex) to a LAN port on the router R7800. The test was repeated twice against the server flent-fremont.bufferbloat.net.

Somehow I don't fully trust these public Flent servers. The graphs only look nice for people having relatively slow D/U speeds. I will connect a local Flent server on the WAN port and then LAN port and run the tests tomorrow.

Here are the wired tests against local server running on the router R7800 itself. Compare them to the similar tests (same local server) for wireless client, we just have to think of WIFI like a living beating heart and wired Ethernet like a dead flat-lined electrocardiogram. Make WIFI a livingly "lovely" tsunami, not deadly "homely" flat-liner! J/k :slight_smile:

@amteza are you removing patches 334/335 because it actually makes a difference, or just because I recommended it for a specific test a while back?

@vochong - I don't trust those specific servers at these speeds either. It's my hope that the new cloud is better.

Moving to your wired lan result with induced latency ~12ms, the current record-holder is cake on the APU2: AQL and the ath10k is *lovely* - #621 by sebastian_de

Which clocks in at 1.5ms or so. I have always been kind of curious (is this a nss enabled build? Validating NSS fq_codel's correctness - #58 by KONG) what happens on this test with just

tc qdisc replace dev your_lan_interface root nssfq_codel limit 4096 flows 1024 quantum 1514 target 5ms interval 100ms set_default

Going to the wifi result... yuck! How distant from the AP are you?

@amteza Thx for showing a 110Mbit loss of throughput on this simper test comparing the reference release and your patchset.

Going back to the rpm test, I see it is using 20! download streams in your 800 RPM case, and 12 in the 3600 case.

Could you repeat the ndown test with 20 flows? (working hypothesis: we are scaling badly as we add flows)

Do you have two boxes setup for this or are you reflashing the same one? Even a distance of a millimeter could be accounting for the throughput difference.

1 Like

@dtaht I have setup local netperf and irtt server in my Belkin RT3200 router (thanks to @anon98444528 AQL and the ath10k is *lovely* - #608 by nmrh for the instructions). I also installed iperf and irtt in my Arch Linux setup. Those packages were not installed during the previous tests AQL and the ath10k is *lovely* - #674 by ka2107 (netperf was installed though) and ran both WIRED and WIFI rrul_be tests:

Router (also AP) and Client setup: AQL and the ath10k is *lovely* - #674 by ka2107

I did not monitor router CPU usage during the tests.

rrul_be WIRED test to local router/AP:

rrul_be-2022-07-11T114042.151627.flent_

rrul_be WIFI test to local router/AP:

Client and router are in the same room. No wall in between.
"iwd" showed RSSI as -4300 which I believe translates to -43 dBm.
My Google Pixel 6 phone sitting next to the laptop showed same WiFi SSID RSSI as -38 dBm.

rrul_be-2022-07-11T114655.740175.flent_

There's a couple different RT3200s out there... the belkin wifi 6 router? The client is on what wifi chip? Update - sorry, did not read the link to the pointer to the device...

You should be getting 100s of mbits out of this setup, but I'm confused - is HE20 a single channel, like HT20?

As much as I might advocate for minimal channel use, few folk testing ac and ax gear run below VHT80. If it's really a single channel then that's kind of near what we would expect from a wireless-n device.

But: The green BE download should not have run away with all the bandwidth, nor the ratio between down and up so poor.

What form of WPA is enabled? Update - still precoffee here - ok, WPA3. Try WPA2?

As for the wired result, pretty good, was there other traffic on the link to account for the upload loss at T+18 and T+49-T+52? (a 5 minute test is helpful for periodic checks)

the yoga's Intel Wireless AC-9560 should be fq_codel enabled in linux releases after 5.4, I think. Does it have noqueue as a qdisc and an aqm file in /sys/kernel/debug/iee*/phy*/aqm?

I think there is something else/more going on other than just a noqueue qdisc and lack of aqm on the linux client. I suspect HE20 (an 802.11ax throughput mode) might be part of it, but I don't think this explains the green BE download ending up with all the bandwidth. I do suspect other clients on the network are contributing to that.

@ka2107: simplest thing to do is retest a few times and see if the results reproduce (if possible try this with only one client on this band and then again with a few connected).

For reference, my linux kernel 5.4 laptop with an intel 7260 wifi card (noqueue qdisc, aqm file present but I cant output from it) connected to my r7500v2 on 5 GHz, channel 36, VHT80:

Multiple clients are connected to 5GHz but only the laptop is active during this test. Durring an earlier test, all the BE download streams went to near 0 Mbps at about 40s into the test for about 5 s and then recovered. This I suspect is the result of other 5GHz wifi clients network activity.

In short, @ka2107's set up should be "running circles around mine while laughing and singing happily"

EDIT: "the earlier test"

FWIW, how AQL/ATF behaves with a "one" clinet flent test is useful to establish it's working as intended, nothing crashes, latency bugs are fixed, etc.; however, most use cases are with 6 to dozens of clients connected. I do still have wifi issues and I think they are related to how my AP behaves with multiple clients using it. I just don't know yet how to identify, demonstrate, or report it in a way that is "actionable" for a dev to fix it. I still suspect AQL/ATF contributing to that, but I also know its not all of it.

If there is still "fuel left in the tank" after the immediate bugs are sufficiently resolved, then maybe we could spend time on that - but I'm in no rush so take Aug. to ? off.

HE20 is 20 MHz 802.11ax (WiFi 6). In my Apt, channel 165 is the only channel that is completely free of any SSIDs. However Channel 165 is a 20 MHz only (no adjacent channels for 40 MHz or 80 MHz).

I thought with my plan speed being only 50/10 Mbps, I can get away with using only a single 20 MHz channel. My Google Pixel 6 connect to the WIFI in HE20 (802.11ax) mode and my Yoga laptop (Intel 9560) connects in VHT20 (802.11ac) mode.

Do you think WPA2 will be faster than WPA3?

Ref: AQL and the ath10k is *lovely* - #688 by ka2107

# tc -s qdisc show

## Ethernet
qdisc fq_codel 0: dev enp0s31f6 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 7361821001 bytes 5452300 pkt (dropped 0, overlimits 0 requeues 801) 
 backlog 0b 0p requeues 801
  maxpacket 68130 drop_overlimit 0 new_flow_count 202074 ecn_mark 0
  new_flows_len 0 old_flows_len 0

## WiFi
qdisc noqueue 0: dev wlan0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
# cat /sys/kernel/debug/ieee80211/phy0/aqm
access name value
R fq_flows_cnt 4096
R fq_backlog 0
R fq_overlimit 0
R fq_overmemory 0
R fq_collisions 0
R fq_memory_usage 0
RW fq_memory_limit 16777216
RW fq_limit 8192
RW fq_quantum 300
# uname -a
Linux  5.18.10-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 07 Jul 2022 17:18:13 +0000 x86_64 GNU/Linux