FWIW a quick search on the web and the tx_queue_data2_burst value is found many times with a value of 0.
Yes, you are correct, that's the default value for AC_BE, not sure why OpenWrt has it at 2.0. Just wondering if someone know why.
1 Like
@dtaht, I was able to perform some test modifying these parameters. Looks like mt76 chipset is able to use reduced parameters and it helps with the balance between upload download differences.
It clearly has an impact, see test below with above parameters:
To note: up vs down bandwidth utilisation almost swaped, reduction on ≈5 ms average latency.
Next test includes previous one plus wmm_ac_be_txop_limit=94:
I can confirm new TXOP limit is pushed to clients:
1 Like
long-standing mystery solved!!! assuming that bump was from the (previously unknown) openwrt txop limit of 2 to 3. Still total throughput of ~600Mbits! It still might not be the "right thing" - goal (for me at least) is to reduce latencies for the multi-station case....
Anyway, another test would be wmm_ac_be_txop_limit=32: which cuts the TU to 1ms. I'm now unsure how to cut the AP's TU to the same! I no longer remember what the burst value is supposed to configure, but it's not the same as the txop limit.
I guess you refer to tx_queue_data2_burst not TXOP limit, right?
See below, my last one for today. 
Parameters tested:
tx_queue_data2_aifs=1
tx_queue_data2_cwmin=7
tx_queue_data2_cwmax=15
tx_queue_data2_burst=3.0
wmm_ac_be_txop_limit=32
up/downs?
It would be good to see if these changes make any difference on the ath10k, too.
Has to be by tomorrow early AEST, sorry.
No worries. I'm in PDT, typically working 7-7 with a nap in the middle.
On a down test...
tx_queue_data2_burst=5.0
might make your inner bandwidth junkie happy.
tx_queue_data2_burst=.5 # (hopefully this will round up to TU properly)
Might make the dream of playing a twitch game over wifi less distant. The problem is, I think there is easily another 10-15ms of latency lurking somewhere else in the stack, be it NAPI (ethernet or wifi), AQL, BQL, rx rings, the cpu scheduler, reorder buffer, or what have you...
@Gingernut No, but I do hope we fool with the ax equivalents. What hardware do you have? Is it possible for you to repeat some of the tests we've been using?
Found when and why, see below Felix's patch:
From 8650201f10afe83387fd6cde00b08172172eeba3 Mon Sep 17 00:00:00 2001
From: Felix Fietkau <nbd@nbd.name>
Date: Wed, 19 Jun 2019 12:32:20 +0200
Subject: [PATCH] mac80211: add config tweak for tx bursting when using VHT
By default, set BE tx queue TXOP limit to 1.0 in the hostapd config
Many vendor drivers are doing similar things to boost throughput.
On MT7612 under ideal conditions, it improves tx throughput from 470 Mbit/s
to about 570 Mbit/s.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
---
.../kernel/mac80211/files/lib/netifd/wireless/mac80211.sh | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/package/kernel/mac80211/files/lib/netifd/wireless/mac80211.sh b/package/kernel/mac80211/files/lib/netifd/wireless/mac80211.sh
index 0426cb60f7..6dc4e5bf5f 100644
--- a/package/kernel/mac80211/files/lib/netifd/wireless/mac80211.sh
+++ b/package/kernel/mac80211/files/lib/netifd/wireless/mac80211.sh
@@ -23,6 +23,7 @@ drv_mac80211_init_device_config() {
config_add_string path phy 'macaddr:macaddr'
config_add_string hwmode
+ config_add_string tx_burst
config_add_int beacon_int chanbw frag rts
config_add_int rxantenna txantenna antenna_gain txpower distance
config_add_boolean noscan ht_coex
@@ -97,9 +98,10 @@ mac80211_hostapd_setup_base() {
[ "$auto_channel" -gt 0 ] && json_get_values channel_list channels
json_get_vars noscan ht_coex
- json_get_values ht_capab_list ht_capab
+ json_get_values ht_capab_list ht_capab tx_burst
[ -n "$noscan" -a "$noscan" -gt 0 ] && hostapd_noscan=1
+ [ "$tx_burst" = 0 ] && tx_burst=
ieee80211n=1
ht_capab=
@@ -229,6 +231,7 @@ mac80211_hostapd_setup_base() {
vht_link_adapt:3 \
vht160:2
+ set_default tx_burst 2.0
append base_cfg "ieee80211ac=1" "$N"
vht_cap=0
for cap in $(iw phy "$phy" info | awk -F "[()]" '/VHT Capabilities/ { print $2 }'); do
@@ -310,6 +313,7 @@ mac80211_hostapd_setup_base() {
${channel:+channel=$channel}
${channel_list:+chanlist=$channel_list}
${hostapd_noscan:+noscan=1}
+${tx_burst:+tx_queue_data2_burst=$tx_burst}
$base_cfg
EOF
--
2.30.2
At least this means that adding in /etc/config/wireless under config wifi-device something like option tx_burst '5.0' will work. No need for patching anything if this is the only configuration parameter to change. I hope this helps testing it.
Update: There is a bug in mac80211.sh and the reading of option tx_burst, the next line in the patch
+ json_get_values ht_capab_list ht_capab tx_burst
should be
json_get_values ht_capab_list ht_capab
+ json_get_vars tx_burst
1 Like
At the moment I have mt76 and ath10k, stock OpenWRT ath10k-ct.
If I get some spare time I will get some tests done on the ath10k platform, no promises though.
P.D This is focusing on the 5GHz band correct?
Nice find.
What I don't get is it states to set BE tx queue TXOP limit to 1.0 but then explicitly sets it to 2.0.
Or at least it looks like that.
Clearly a typo, the 1 key and 2 key are very close.
1 Like
Did you see this PR patch:
https://patchwork.ozlabs.org/project/openwrt/patch/20220722063631.9903-1-sultan@kerneltoast.com/
In terms of other random latency reducing stuff, I've always wanted to revisit NAPI_POLL_WEIGHT and make it configurable. Arm devices with short ins pipelines like the A53 can context switch rapidly (especially with a nearly dedicated core), and have relatively small caches, so doing less work, more often, might be a win. The mt76 defaults to 64, where, who knows? 8 might be doable.
See also: https://patchwork.ozlabs.org/project/netdev/patch/1362535042.15793.144.camel@edumazet-glaptop/
A few more tests. With below configured parameters in hostapd:
tx_queue_data2_aifs=1
tx_queue_data2_cwmin=7
tx_queue_data2_cwmax=15
tx_queue_data2_burst=3.0
wmm_ac_be_txop_limit=94
Following up/down tests were ran in macOS:
T="macos v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods"; for i in 1 2 4 8 16; do flent -l 30 --socket-stats -x --step-size=.05 --te=upload_streams=$i -H openwrt.lan -t tcp_nup-$i-threads-$T tcp_nup; flent -l 30 --socket-stats -x --step-size=.05 --te=download_streams=$i -H openwrt.lan -t tcp_ndown-$i-threads-$T tcp_ndown; done
I'm not sure how useful this will be as the SsRunner cannot be executed on macOS, no ss command. Did anyone compile it for macOS?
And now up/down tests ran in Linux:
T="linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods"; for i in 1 2 4 8 16; do flent -l 30 --socket-stats -x --step-size=.05 --te=upload_streams=$i -H openwrt.lan -t tcp_nup-$i-threads-$T tcp_nup; flent -l 30 --socket-stats -x --step-size=.05 --te=download_streams=$i -H openwrt.lan -t tcp_ndown-$i-threads-$T tcp_ndown; done
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_nup test. Expected run time: 40 seconds.
Data file written to ./tcp_nup-2022-08-04T180236.448340.tcp_nup-1-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_nup test run from 2022-08-04 18:02:36.448340
Title: 'tcp_nup-1-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 20.20 17.05 ms 800
TCP upload avg : 581.35 N/A Mbits/s 800
TCP upload sum : 581.35 N/A Mbits/s 800
TCP upload::1 : 581.35 592.17 Mbits/s 800
TCP upload::1::tcp_cwnd : 1662.25 1367.00 509
TCP upload::1::tcp_delivery_rate : 543.21 539.70 509
TCP upload::1::tcp_pacing_rate : 829.72 797.32 509
TCP upload::1::tcp_rtt : 28.74 28.16 509
TCP upload::1::tcp_rtt_var : 0.22 0.07 509
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T180321.824282.tcp_ndown-1-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:03:21.824282
Title: 'tcp_ndown-1-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 18.50 17.45 ms 800
TCP download avg : 459.34 N/A Mbits/s 800
TCP download sum : 459.34 N/A Mbits/s 800
TCP download::1 : 459.34 467.75 Mbits/s 800
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_nup test. Expected run time: 40 seconds.
Data file written to ./tcp_nup-2022-08-04T180407.176189.tcp_nup-2-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_nup test run from 2022-08-04 18:04:07.176189
Title: 'tcp_nup-2-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 26.87 28.40 ms 798
TCP upload avg : 295.78 N/A Mbits/s 800
TCP upload sum : 591.57 N/A Mbits/s 800
TCP upload::1 : 374.50 386.94 Mbits/s 800
TCP upload::1::tcp_cwnd : 1157.21 1114.00 507
TCP upload::1::tcp_delivery_rate : 348.52 346.27 507
TCP upload::1::tcp_pacing_rate : 548.71 518.97 507
TCP upload::1::tcp_rtt : 31.21 29.82 507
TCP upload::1::tcp_rtt_var : 0.47 0.20 507
TCP upload::2 : 217.07 216.53 Mbits/s 800
TCP upload::2::tcp_cwnd : 675.62 630.00 507
TCP upload::2::tcp_delivery_rate : 202.02 202.34 507
TCP upload::2::tcp_pacing_rate : 315.48 305.86 507
TCP upload::2::tcp_rtt : 31.44 30.07 507
TCP upload::2::tcp_rtt_var : 0.57 0.30 507
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T180452.627249.tcp_ndown-2-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:04:52.627249
Title: 'tcp_ndown-2-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 23.00 24.90 ms 799
TCP download avg : 241.73 N/A Mbits/s 799
TCP download sum : 483.46 N/A Mbits/s 799
TCP download::1 : 241.30 244.83 Mbits/s 799
TCP download::2 : 242.16 246.33 Mbits/s 799
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_nup test. Expected run time: 40 seconds.
Data file written to ./tcp_nup-2022-08-04T180538.069757.tcp_nup-4-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_nup test run from 2022-08-04 18:05:38.069757
Title: 'tcp_nup-4-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 34.05 31.95 ms 798
TCP upload avg : 151.00 N/A Mbits/s 800
TCP upload sum : 603.98 N/A Mbits/s 800
TCP upload::1 : 212.48 217.38 Mbits/s 800
TCP upload::1::tcp_cwnd : 692.34 703.00 503
TCP upload::1::tcp_delivery_rate : 203.72 203.59 503
TCP upload::1::tcp_pacing_rate : 313.53 296.97 503
TCP upload::1::tcp_rtt : 32.79 32.90 503
TCP upload::1::tcp_rtt_var : 0.69 0.31 503
TCP upload::2 : 125.89 130.91 Mbits/s 800
TCP upload::2::tcp_cwnd : 410.25 426.00 504
TCP upload::2::tcp_delivery_rate : 120.98 124.14 504
TCP upload::2::tcp_pacing_rate : 183.69 175.98 504
TCP upload::2::tcp_rtt : 32.92 32.96 504
TCP upload::2::tcp_rtt_var : 0.99 0.56 504
TCP upload::3 : 134.12 140.65 Mbits/s 800
TCP upload::3::tcp_cwnd : 437.49 455.50 504
TCP upload::3::tcp_delivery_rate : 129.47 132.38 504
TCP upload::3::tcp_pacing_rate : 196.25 186.38 504
TCP upload::3::tcp_rtt : 32.93 32.88 502
TCP upload::3::tcp_rtt_var : 0.84 0.50 502
TCP upload::4 : 131.49 136.40 Mbits/s 800
TCP upload::4::tcp_cwnd : 433.67 454.00 505
TCP upload::4::tcp_delivery_rate : 126.53 129.99 505
TCP upload::4::tcp_pacing_rate : 191.57 183.41 505
TCP upload::4::tcp_rtt : 33.37 33.26 503
TCP upload::4::tcp_rtt_var : 0.94 0.52 503
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T180623.742220.tcp_ndown-4-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:06:23.742220
Title: 'tcp_ndown-4-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 25.30 24.45 ms 800
TCP download avg : 124.37 N/A Mbits/s 800
TCP download sum : 497.48 N/A Mbits/s 800
TCP download::1 : 108.32 120.79 Mbits/s 800
TCP download::2 : 115.44 124.77 Mbits/s 800
TCP download::3 : 192.98 144.30 Mbits/s 800
TCP download::4 : 80.74 90.15 Mbits/s 800
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_nup test. Expected run time: 40 seconds.
Data file written to ./tcp_nup-2022-08-04T180709.395616.tcp_nup-8-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_nup test run from 2022-08-04 18:07:09.395616
Title: 'tcp_nup-8-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 30.73 34.50 ms 797
TCP upload avg : 71.40 N/A Mbits/s 800
TCP upload sum : 571.23 N/A Mbits/s 800
TCP upload::1 : 78.05 80.18 Mbits/s 800
TCP upload::1::tcp_cwnd : 271.46 267.00 509
TCP upload::1::tcp_delivery_rate : 73.87 73.49 509
TCP upload::1::tcp_pacing_rate : 110.31 104.17 509
TCP upload::1::tcp_rtt : 36.23 35.73 509
TCP upload::1::tcp_rtt_var : 1.09 0.79 509
TCP upload::2 : 64.25 65.90 Mbits/s 800
TCP upload::2::tcp_cwnd : 223.21 225.00 509
TCP upload::2::tcp_delivery_rate : 61.26 61.78 509
TCP upload::2::tcp_pacing_rate : 90.25 85.85 509
TCP upload::2::tcp_rtt : 36.29 35.63 509
TCP upload::2::tcp_rtt_var : 1.20 0.90 509
TCP upload::3 : 56.55 58.10 Mbits/s 800
TCP upload::3::tcp_cwnd : 196.84 201.00 508
TCP upload::3::tcp_delivery_rate : 53.64 54.25 508
TCP upload::3::tcp_pacing_rate : 78.96 76.05 508
TCP upload::3::tcp_rtt : 36.49 35.70 506
TCP upload::3::tcp_rtt_var : 1.41 1.10 506
TCP upload::4 : 44.52 47.15 Mbits/s 800
TCP upload::4::tcp_cwnd : 155.95 160.00 507
TCP upload::4::tcp_delivery_rate : 42.11 43.04 507
TCP upload::4::tcp_pacing_rate : 61.87 60.95 507
TCP upload::4::tcp_rtt : 37.11 36.27 507
TCP upload::4::tcp_rtt_var : 1.70 1.35 507
TCP upload::5 : 151.87 155.40 Mbits/s 800
TCP upload::5::tcp_cwnd : 535.59 519.00 508
TCP upload::5::tcp_delivery_rate : 144.29 135.61 508
TCP upload::5::tcp_pacing_rate : 220.29 203.96 508
TCP upload::5::tcp_rtt : 36.00 35.02 508
TCP upload::5::tcp_rtt_var : 0.72 0.44 508
TCP upload::6 : 49.82 52.06 Mbits/s 800
TCP upload::6::tcp_cwnd : 175.54 179.50 508
TCP upload::6::tcp_delivery_rate : 47.22 48.45 508
TCP upload::6::tcp_pacing_rate : 68.95 67.17 508
TCP upload::6::tcp_rtt : 37.14 36.50 507
TCP upload::6::tcp_rtt_var : 1.48 1.22 507
TCP upload::7 : 55.95 57.09 Mbits/s 800
TCP upload::7::tcp_cwnd : 195.55 200.00 508
TCP upload::7::tcp_delivery_rate : 52.88 53.68 508
TCP upload::7::tcp_pacing_rate : 78.04 74.91 508
TCP upload::7::tcp_rtt : 36.72 36.05 508
TCP upload::7::tcp_rtt_var : 1.36 1.04 508
TCP upload::8 : 70.22 72.13 Mbits/s 800
TCP upload::8::tcp_cwnd : 244.83 244.00 509
TCP upload::8::tcp_delivery_rate : 66.24 66.35 509
TCP upload::8::tcp_pacing_rate : 99.16 93.99 509
TCP upload::8::tcp_rtt : 36.25 35.59 507
TCP upload::8::tcp_rtt_var : 1.17 0.86 507
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T180755.494117.tcp_ndown-8-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:07:55.494117
Title: 'tcp_ndown-8-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 24.10 25.95 ms 799
TCP download avg : 57.60 N/A Mbits/s 799
TCP download sum : 460.79 N/A Mbits/s 799
TCP download::1 : 57.69 59.53 Mbits/s 799
TCP download::2 : 51.42 53.64 Mbits/s 799
TCP download::3 : 55.95 57.42 Mbits/s 799
TCP download::4 : 60.80 60.40 Mbits/s 799
TCP download::5 : 66.62 64.65 Mbits/s 799
TCP download::6 : 61.32 60.70 Mbits/s 799
TCP download::7 : 52.60 53.90 Mbits/s 799
TCP download::8 : 54.39 56.09 Mbits/s 799
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_nup test. Expected run time: 40 seconds.
Data file written to ./tcp_nup-2022-08-04T180841.586505.tcp_nup-16-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_nup test run from 2022-08-04 18:08:41.586505
Title: 'tcp_nup-16-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 38.21 35.40 ms 795
TCP upload avg : 35.10 N/A Mbits/s 799
TCP upload sum : 561.67 N/A Mbits/s 799
TCP upload::1 : 32.48 33.12 Mbits/s 799
TCP upload::10 : 39.66 40.25 Mbits/s 799
TCP upload::10::tcp_cwnd : 140.93 142.00 506
TCP upload::10::tcp_delivery_rate : 37.47 37.87 506
TCP upload::10::tcp_pacing_rate : 55.34 52.94 506
TCP upload::10::tcp_rtt : 37.09 37.27 505
TCP upload::10::tcp_rtt_var : 1.94 1.47 505
TCP upload::11 : 32.97 33.55 Mbits/s 799
TCP upload::11::tcp_cwnd : 117.74 120.00 506
TCP upload::11::tcp_delivery_rate : 30.96 31.83 506
TCP upload::11::tcp_pacing_rate : 45.45 43.70 506
TCP upload::11::tcp_rtt : 37.47 37.66 506
TCP upload::11::tcp_rtt_var : 2.18 1.66 506
TCP upload::12 : 40.62 40.88 Mbits/s 799
TCP upload::12::tcp_cwnd : 144.84 145.00 505
TCP upload::12::tcp_delivery_rate : 38.25 38.23 505
TCP upload::12::tcp_pacing_rate : 56.78 54.58 505
TCP upload::12::tcp_rtt : 37.14 37.30 503
TCP upload::12::tcp_rtt_var : 1.96 1.54 503
TCP upload::13 : 28.55 29.21 Mbits/s 799
TCP upload::13::tcp_cwnd : 103.92 105.00 505
TCP upload::13::tcp_delivery_rate : 26.79 26.78 505
TCP upload::13::tcp_pacing_rate : 39.18 38.09 505
TCP upload::13::tcp_rtt : 38.36 38.39 505
TCP upload::13::tcp_rtt_var : 2.45 2.03 505
TCP upload::14 : 42.47 42.84 Mbits/s 799
TCP upload::14::tcp_cwnd : 150.95 151.00 506
TCP upload::14::tcp_delivery_rate : 40.14 39.93 506
TCP upload::14::tcp_pacing_rate : 59.18 56.63 506
TCP upload::14::tcp_rtt : 37.12 37.25 506
TCP upload::14::tcp_rtt_var : 1.94 1.51 506
TCP upload::15 : 32.01 32.05 Mbits/s 799
TCP upload::15::tcp_cwnd : 116.16 119.00 505
TCP upload::15::tcp_delivery_rate : 30.11 30.77 505
TCP upload::15::tcp_pacing_rate : 44.11 42.68 505
TCP upload::15::tcp_rtt : 38.12 38.12 505
TCP upload::15::tcp_rtt_var : 2.28 1.76 505
TCP upload::16 : 33.45 33.88 Mbits/s 799
TCP upload::16::tcp_cwnd : 119.32 121.50 506
TCP upload::16::tcp_delivery_rate : 31.48 32.32 506
TCP upload::16::tcp_pacing_rate : 46.36 44.44 506
TCP upload::16::tcp_rtt : 37.36 37.51 504
TCP upload::16::tcp_rtt_var : 2.16 1.63 504
TCP upload::1::tcp_cwnd : 117.89 120.00 505
TCP upload::1::tcp_delivery_rate : 30.62 31.50 505
TCP upload::1::tcp_pacing_rate : 45.11 43.19 505
TCP upload::1::tcp_rtt : 38.06 38.10 505
TCP upload::1::tcp_rtt_var : 2.22 1.73 505
TCP upload::2 : 33.08 33.66 Mbits/s 799
TCP upload::2::tcp_cwnd : 118.41 121.00 505
TCP upload::2::tcp_delivery_rate : 31.16 31.97 505
TCP upload::2::tcp_pacing_rate : 45.73 44.21 505
TCP upload::2::tcp_rtt : 37.53 37.62 503
TCP upload::2::tcp_rtt_var : 2.19 1.72 503
TCP upload::3 : 33.75 33.74 Mbits/s 799
TCP upload::3::tcp_cwnd : 120.57 123.00 506
TCP upload::3::tcp_delivery_rate : 31.71 32.39 506
TCP upload::3::tcp_pacing_rate : 46.65 44.92 506
TCP upload::3::tcp_rtt : 37.53 37.47 505
TCP upload::3::tcp_rtt_var : 2.22 1.72 505
TCP upload::4 : 35.19 35.79 Mbits/s 799
TCP upload::4::tcp_cwnd : 125.44 128.00 506
TCP upload::4::tcp_delivery_rate : 33.34 34.01 506
TCP upload::4::tcp_pacing_rate : 48.69 46.86 506
TCP upload::4::tcp_rtt : 37.46 37.53 504
TCP upload::4::tcp_rtt_var : 2.11 1.65 504
TCP upload::5 : 32.67 33.00 Mbits/s 799
TCP upload::5::tcp_cwnd : 117.05 119.00 505
TCP upload::5::tcp_delivery_rate : 30.64 31.48 505
TCP upload::5::tcp_pacing_rate : 45.11 43.57 505
TCP upload::5::tcp_rtt : 37.56 37.64 505
TCP upload::5::tcp_rtt_var : 2.21 1.71 505
TCP upload::6 : 33.12 33.86 Mbits/s 799
TCP upload::6::tcp_cwnd : 118.54 121.00 505
TCP upload::6::tcp_delivery_rate : 31.18 32.01 505
TCP upload::6::tcp_pacing_rate : 45.74 44.03 505
TCP upload::6::tcp_rtt : 37.57 37.50 502
TCP upload::6::tcp_rtt_var : 2.18 1.76 502
TCP upload::7 : 48.73 48.23 Mbits/s 799
TCP upload::7::tcp_cwnd : 171.96 165.00 506
TCP upload::7::tcp_delivery_rate : 46.09 43.93 506
TCP upload::7::tcp_pacing_rate : 67.60 63.12 506
TCP upload::7::tcp_rtt : 37.12 37.38 506
TCP upload::7::tcp_rtt_var : 1.80 1.41 506
TCP upload::8 : 29.93 30.14 Mbits/s 799
TCP upload::8::tcp_cwnd : 107.38 108.00 505
TCP upload::8::tcp_delivery_rate : 28.20 28.26 505
TCP upload::8::tcp_pacing_rate : 41.19 40.17 505
TCP upload::8::tcp_rtt : 37.73 37.80 505
TCP upload::8::tcp_rtt_var : 2.30 1.82 505
TCP upload::9 : 32.99 33.55 Mbits/s 799
TCP upload::9::tcp_cwnd : 117.65 120.00 505
TCP upload::9::tcp_delivery_rate : 30.98 31.64 505
TCP upload::9::tcp_pacing_rate : 45.50 43.84 505
TCP upload::9::tcp_rtt : 37.43 37.48 505
TCP upload::9::tcp_rtt_var : 2.19 1.82 505
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T180928.507790.tcp_ndown-16-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_wmm_tx_data2_mods.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:09:28.507790
Title: 'tcp_ndown-16-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off wmm tx_data2 mods'
avg median # data pts
Ping (ms) ICMP : 24.80 27.35 ms 800
TCP download avg : 29.97 N/A Mbits/s 800
TCP download sum : 479.56 N/A Mbits/s 800
TCP download::1 : 32.16 32.03 Mbits/s 800
TCP download::10 : 31.41 31.47 Mbits/s 800
TCP download::11 : 28.51 29.52 Mbits/s 800
TCP download::12 : 27.98 28.80 Mbits/s 800
TCP download::13 : 30.22 30.53 Mbits/s 800
TCP download::14 : 30.19 30.56 Mbits/s 800
TCP download::15 : 27.65 28.49 Mbits/s 800
TCP download::16 : 30.66 30.89 Mbits/s 800
TCP download::2 : 32.20 31.56 Mbits/s 800
TCP download::3 : 30.68 30.84 Mbits/s 800
TCP download::4 : 28.23 29.31 Mbits/s 800
TCP download::5 : 29.25 29.80 Mbits/s 800
TCP download::6 : 28.08 29.09 Mbits/s 800
TCP download::7 : 30.43 30.52 Mbits/s 800
TCP download::8 : 32.65 31.83 Mbits/s 800
TCP download::9 : 29.26 29.97 Mbits/s 800
Last round of requested DOWNLOAD tests.
Test parameters (only):
tx_queue_data2_burst=5.0
I executed them in Linux and macOS. Under each output log below you'll find links to a folder with all the tests, filenames help identify the ones for each operating system.
Linux output:
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T181756.063666.tcp_ndown-1-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:17:56.063666
Title: 'tcp_ndown-1-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 17.80 16.25 ms 797
TCP download avg : 474.14 N/A Mbits/s 797
TCP download sum : 474.14 N/A Mbits/s 797
TCP download::1 : 474.14 488.75 Mbits/s 797
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T181841.410507.tcp_ndown-2-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:18:41.410507
Title: 'tcp_ndown-2-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 23.70 24.60 ms 800
TCP download avg : 233.18 N/A Mbits/s 800
TCP download sum : 466.36 N/A Mbits/s 800
TCP download::1 : 246.81 244.62 Mbits/s 800
TCP download::2 : 219.55 226.04 Mbits/s 800
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T181926.854477.tcp_ndown-4-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:19:26.854477
Title: 'tcp_ndown-4-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 20.60 20.70 ms 799
TCP download avg : 114.89 N/A Mbits/s 799
TCP download sum : 459.54 N/A Mbits/s 799
TCP download::1 : 106.50 110.47 Mbits/s 799
TCP download::2 : 109.41 114.40 Mbits/s 799
TCP download::3 : 115.42 119.53 Mbits/s 799
TCP download::4 : 128.21 130.73 Mbits/s 799
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T182012.520083.tcp_ndown-8-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:20:12.520083
Title: 'tcp_ndown-8-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 22.30 23.40 ms 799
TCP download avg : 58.05 N/A Mbits/s 799
TCP download sum : 464.36 N/A Mbits/s 799
TCP download::1 : 59.76 60.11 Mbits/s 799
TCP download::2 : 61.47 62.12 Mbits/s 799
TCP download::3 : 58.82 59.64 Mbits/s 799
TCP download::4 : 65.04 64.83 Mbits/s 799
TCP download::5 : 62.41 62.42 Mbits/s 799
TCP download::6 : 50.70 51.66 Mbits/s 799
TCP download::7 : 55.60 56.76 Mbits/s 799
TCP download::8 : 50.56 52.27 Mbits/s 799
Starting Flent 2.0.1 using Python 3.10.4.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-04T182058.593037.tcp_ndown-16-threads-linux_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:20:58.593037
Title: 'tcp_ndown-16-threads-linux v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 30.60 32.70 ms 800
TCP download avg : 29.02 N/A Mbits/s 800
TCP download sum : 464.36 N/A Mbits/s 800
TCP download::1 : 30.32 31.33 Mbits/s 800
TCP download::10 : 28.92 30.10 Mbits/s 800
TCP download::11 : 27.43 29.05 Mbits/s 800
TCP download::12 : 28.99 30.30 Mbits/s 800
TCP download::13 : 29.34 30.47 Mbits/s 800
TCP download::14 : 27.88 29.27 Mbits/s 800
TCP download::15 : 30.98 31.56 Mbits/s 800
TCP download::16 : 29.73 30.86 Mbits/s 800
TCP download::2 : 27.58 29.13 Mbits/s 800
TCP download::3 : 27.80 29.46 Mbits/s 800
TCP download::4 : 29.71 30.45 Mbits/s 800
TCP download::5 : 28.48 29.67 Mbits/s 800
TCP download::6 : 29.51 30.47 Mbits/s 800
TCP download::7 : 29.18 30.37 Mbits/s 800
TCP download::8 : 27.62 29.51 Mbits/s 800
TCP download::9 : 30.89 31.35 Mbits/s 800
macOS output:
Starting Flent 2.0.1 using Python 3.9.13.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-05T042917.037349.tcp_ndown-1-threads-macOS_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:29:17.037349
Title: 'tcp_ndown-1-threads-macOS v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 20.80 23.60 ms 799
TCP download avg : 471.16 N/A Mbits/s 799
TCP download sum : 471.16 N/A Mbits/s 799
TCP download::1 : 471.16 475.47 Mbits/s 799
Starting Flent 2.0.1 using Python 3.9.13.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-05T043002.520946.tcp_ndown-2-threads-macOS_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:30:02.520946
Title: 'tcp_ndown-2-threads-macOS v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 24.10 26.30 ms 799
TCP download avg : 217.68 N/A Mbits/s 799
TCP download sum : 435.36 N/A Mbits/s 799
TCP download::1 : 203.92 210.31 Mbits/s 799
TCP download::2 : 231.44 229.42 Mbits/s 799
Starting Flent 2.0.1 using Python 3.9.13.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-05T043048.130109.tcp_ndown-4-threads-macOS_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:30:48.130109
Title: 'tcp_ndown-4-threads-macOS v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 26.40 28.60 ms 799
TCP download avg : 106.34 N/A Mbits/s 799
TCP download sum : 425.36 N/A Mbits/s 799
TCP download::1 : 105.41 106.02 Mbits/s 799
TCP download::2 : 107.53 106.54 Mbits/s 799
TCP download::3 : 103.04 102.44 Mbits/s 799
TCP download::4 : 109.38 107.00 Mbits/s 799
Starting Flent 2.0.1 using Python 3.9.13.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-05T043133.985191.tcp_ndown-8-threads-macOS_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:31:33.985191
Title: 'tcp_ndown-8-threads-macOS v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 25.30 28.30 ms 799
TCP download avg : 53.02 N/A Mbits/s 799
TCP download sum : 424.17 N/A Mbits/s 799
TCP download::1 : 56.93 55.93 Mbits/s 799
TCP download::2 : 50.91 51.62 Mbits/s 799
TCP download::3 : 52.50 52.52 Mbits/s 799
TCP download::4 : 53.09 52.97 Mbits/s 799
TCP download::5 : 50.79 51.04 Mbits/s 799
TCP download::6 : 50.71 51.22 Mbits/s 799
TCP download::7 : 55.26 54.67 Mbits/s 799
TCP download::8 : 53.98 53.07 Mbits/s 799
Starting Flent 2.0.1 using Python 3.9.13.
Starting tcp_ndown test. Expected run time: 40 seconds.
Data file written to ./tcp_ndown-2022-08-05T043220.294767.tcp_ndown-16-threads-macOS_v22_03-rc6_mt76_WLAN_ECN-on_loc_servs_off_tx_queue_data2_burst_5_0.flent.gz
Summary of tcp_ndown test run from 2022-08-04 18:32:20.294767
Title: 'tcp_ndown-16-threads-macOS v22.03-rc6 mt76 WLAN ECN-on loc servs off tx_queue_data2_burst=5.0'
avg median # data pts
Ping (ms) ICMP : 25.20 28.20 ms 799
TCP download avg : 26.91 N/A Mbits/s 799
TCP download sum : 430.54 N/A Mbits/s 799
TCP download::1 : 28.01 27.12 Mbits/s 799
TCP download::10 : 26.28 26.16 Mbits/s 799
TCP download::11 : 25.94 26.21 Mbits/s 799
TCP download::12 : 27.29 27.15 Mbits/s 799
TCP download::13 : 27.16 26.69 Mbits/s 799
TCP download::14 : 28.37 28.13 Mbits/s 799
TCP download::15 : 25.29 25.24 Mbits/s 799
TCP download::16 : 25.82 25.88 Mbits/s 799
TCP download::2 : 28.57 27.34 Mbits/s 799
TCP download::3 : 26.88 26.77 Mbits/s 799
TCP download::4 : 26.97 27.14 Mbits/s 799
TCP download::5 : 28.36 27.28 Mbits/s 799
TCP download::6 : 25.37 25.52 Mbits/s 799
TCP download::7 : 26.26 26.27 Mbits/s 799
TCP download::8 : 26.54 26.52 Mbits/s 799
TCP download::9 : 27.43 26.98 Mbits/s 799
Bonus round (free rrul_be test):
- As usual, get your data clicking here
And, finally for the gamers in the family (does not look very good, tho'):
tx_queue_data2_burst=0.5
- Download the data clicking me
Okay, these, I hope, were a couple of hours well spent. Now time for a good and healthy training and back to business.
1 Like
Thx for the miserable results. They are always useful. This was one of my all time favorite rants, at SIGCOMM:
Nobody's invited me back.
I would have hoped that by increasing
tx_queue_data2_burst=3.0
to 5, we would have seen the 570Mbit download that @nbd claimed in his commit. It could be that the VI-queue style
tx_queue_data2_aifs=1
tx_queue_data2_cwmin=7
tx_queue_data2_cwmax=15
could be getting in the way here. I'd only intended btw to test that part briefly, and then actually test the VI queue by itself and vs the BE queue. We have a lot of people in the IETF that really want to use up the VI queue for a new kind of traffic, and it concerns me. Similarly Qosify is trying to land stuff there also.
Wow 6 reviewers required... not like the old days. Nagging @nbd or @jow to do two 1 liners like the above PRs via irc used to work for me...
This is only active in the first round, in last round with tx_queue_data2_burst=5.0 and 0.5 those 3 parameters are left as they are by default in BE queue, so tests are supposed to be valid.
To test BE vs VI, I knew I was forgetting something, I'll see what I can do.