Flent rrul tests WLAN.HK.2.7.0.1-01744 and WLAN.HK.2.9.0.1-01890

Hello. I'm trying to understand the wifi performance of the wlan.hk firmwares and the iperf3 --bidir performance. Please note, that I'm not an experienced developer or any degree in this stuff.
So my tests might be not very good and may give false data.
Anyway, I just wanna share the data from doing some Flent and iperf3 stuff.
I have noticed that the --bidir perfomance has been like this a few months. Different snapshots and wlan.hk firmwares give me the same results.

Lets start with some basic information about the setup.

Flent server = airprint (RPI 4 2GB version) wired
-----------------------------------------------------------------
AP = WAX630 IPQ8074
     Channel 60@40 
     WPA2
     Basic config with only 802.11k enabled
-----------------------------------------------------------------
Flent client = Macbook Pro 13 2016 3x3 | RSSI -52@600 - Noise -98

All test are done with

OpenWrt SNAPSHOT, r24222-eacc885816

A simple speedtest

Idle Latency:     4.68 ms   (jitter: 0.13ms, low: 4.56ms, high: 4.78ms)
    Download:   140.53 Mbps (data used: 63.3 MB)
                  6.24 ms   (jitter: 0.99ms, low: 4.45ms, high: 9.48ms)
      Upload:   143.36 Mbps (data used: 68.6 MB)
                  5.38 ms   (jitter: 0.70ms, low: 4.32ms, high: 8.37ms)
 Packet Loss:     0.0%

Some pings to the flent server doing nothing (they are similar for both hk.2.7 and hk.2.9)

64 bytes from airprint (192.168.100.50): icmp_seq=1 ttl=64 time=2.79 ms
64 bytes from airprint (192.168.100.50): icmp_seq=2 ttl=64 time=3.54 ms
64 bytes from airprint (192.168.100.50): icmp_seq=3 ttl=64 time=3.35 ms
64 bytes from airprint (192.168.100.50): icmp_seq=4 ttl=64 time=4.38 ms
64 bytes from airprint (192.168.100.50): icmp_seq=5 ttl=64 time=4.13 ms
64 bytes from airprint (192.168.100.50): icmp_seq=6 ttl=64 time=2.50 ms
# iperf3 --bidir
# WLAN.HK.2.7.0.1-01744-QCAHKSWPL_SILICONZ-1
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][RX-S]   0.00-1.00   sec  27.3 MBytes   228 Mbits/sec
[  8][TX-S]   0.00-1.00   sec  31.2 MBytes   262 Mbits/sec    0   1.67 MBytes
[  5][RX-S]   1.00-2.00   sec  25.6 MBytes   215 Mbits/sec
[  8][TX-S]   1.00-2.00   sec  33.8 MBytes   283 Mbits/sec    0   3.13 MBytes
[  5][RX-S]   2.00-3.00   sec  27.3 MBytes   229 Mbits/sec
[  8][TX-S]   2.00-3.00   sec  31.2 MBytes   262 Mbits/sec    0   3.13 MBytes
[  5][RX-S]   3.00-4.00   sec  26.2 MBytes   220 Mbits/sec
[  8][TX-S]   3.00-4.00   sec  33.8 MBytes   283 Mbits/sec    0   3.13 MBytes
[  5][RX-S]   4.00-5.00   sec  27.8 MBytes   234 Mbits/sec
[  8][TX-S]   4.00-5.00   sec  31.2 MBytes   262 Mbits/sec    0   3.13 MBytes
[  5][RX-S]   5.00-6.00   sec  32.5 MBytes   272 Mbits/sec
[  8][TX-S]   5.00-6.00   sec  25.0 MBytes   210 Mbits/sec  168   1.58 MBytes
[  5][RX-S]   6.00-7.00   sec  46.7 MBytes   392 Mbits/sec
[  8][TX-S]   6.00-7.00   sec  12.5 MBytes   105 Mbits/sec  492   31.1 KBytes
[  5][RX-S]   7.00-8.00   sec  53.9 MBytes   452 Mbits/sec
[  8][TX-S]   7.00-8.00   sec  5.00 MBytes  41.9 Mbits/sec  135    119 KBytes
[  5][RX-S]   8.00-9.00   sec  54.0 MBytes   453 Mbits/sec
[  8][TX-S]   8.00-9.00   sec  3.75 MBytes  31.5 Mbits/sec   13    113 KBytes
[  5][RX-S]   9.00-10.00  sec  53.8 MBytes   451 Mbits/sec
[  8][TX-S]   9.00-10.00  sec  5.00 MBytes  41.9 Mbits/sec    0    143 KBytes
[  5][RX-S]  10.00-11.00  sec  52.6 MBytes   441 Mbits/sec
[  8][TX-S]  10.00-11.00  sec  6.25 MBytes  52.4 Mbits/sec    0    173 KBytes
[  5][RX-S]  11.00-12.00  sec  52.5 MBytes   440 Mbits/sec
[  8][TX-S]  11.00-12.00  sec  5.00 MBytes  41.9 Mbits/sec   27    103 KBytes
[  5][RX-S]  12.00-13.00  sec  53.5 MBytes   449 Mbits/sec
[  8][TX-S]  12.00-13.00  sec  3.75 MBytes  31.5 Mbits/sec   57   97.6 KBytes
[  5][RX-S]  13.00-14.00  sec  52.6 MBytes   441 Mbits/sec
[  8][TX-S]  13.00-14.00  sec  5.00 MBytes  41.9 Mbits/sec    0    130 KBytes

And some Flent images from the rrul test with WLAN.HK.2.7.0.1-01744.

This is from testing WLAN.HK.2.9.0.1-01890.

# Iperf3 --bidir
# WLAN.HK.2.9.0.1-01890
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][RX-S]   0.00-1.00   sec  10.3 MBytes  86.8 Mbits/sec
[  8][TX-S]   0.00-1.00   sec  19.2 MBytes   161 Mbits/sec    0    841 KBytes
[  5][RX-S]   1.00-2.00   sec  13.6 MBytes   114 Mbits/sec
[  8][TX-S]   1.00-2.00   sec  25.0 MBytes   210 Mbits/sec    0   2.07 MBytes
[  5][RX-S]   2.00-3.00   sec  17.5 MBytes   147 Mbits/sec
[  8][TX-S]   2.00-3.00   sec  26.2 MBytes   220 Mbits/sec  257   1.53 MBytes
[  5][RX-S]   3.00-4.00   sec  23.8 MBytes   200 Mbits/sec
[  8][TX-S]   3.00-4.00   sec  25.0 MBytes   210 Mbits/sec    0   1.63 MBytes
[  5][RX-S]   4.00-5.00   sec  23.9 MBytes   201 Mbits/sec
[  8][TX-S]   4.00-5.00   sec  22.5 MBytes   189 Mbits/sec    0   1.71 MBytes
[  5][RX-S]   5.00-6.00   sec  23.9 MBytes   201 Mbits/sec
[  8][TX-S]   5.00-6.00   sec  21.2 MBytes   178 Mbits/sec    0   1.77 MBytes
[  5][RX-S]   6.00-7.00   sec  23.5 MBytes   197 Mbits/sec
[  8][TX-S]   6.00-7.00   sec  21.2 MBytes   178 Mbits/sec    0   1.81 MBytes
[  5][RX-S]   7.00-8.00   sec  30.2 MBytes   254 Mbits/sec
[  8][TX-S]   7.00-8.00   sec  26.2 MBytes   220 Mbits/sec    0   1.84 MBytes
[  5][RX-S]   8.00-9.00   sec  33.1 MBytes   278 Mbits/sec
[  8][TX-S]   8.00-9.00   sec  26.2 MBytes   220 Mbits/sec    0   1.86 MBytes
[  5][RX-S]   9.00-10.00  sec  33.8 MBytes   284 Mbits/sec
[  8][TX-S]   9.00-10.00  sec  27.5 MBytes   231 Mbits/sec  122   1.37 MBytes
[  5][RX-S]  10.00-11.00  sec  30.2 MBytes   253 Mbits/sec
[  8][TX-S]  10.00-11.00  sec  20.0 MBytes   168 Mbits/sec   27   1.02 MBytes
[  5][RX-S]  11.00-12.00  sec  20.7 MBytes   174 Mbits/sec
[  8][TX-S]  11.00-12.00  sec  16.2 MBytes   136 Mbits/sec    0   1.09 MBytes
[  5][RX-S]  12.00-13.00  sec  30.1 MBytes   253 Mbits/sec
[  8][TX-S]  12.00-13.00  sec  20.0 MBytes   168 Mbits/sec   39    810 KBytes
[  5][RX-S]  13.00-14.00  sec  43.9 MBytes   368 Mbits/sec
[  8][TX-S]  13.00-14.00  sec  13.8 MBytes   115 Mbits/sec  432    144 KBytes
[  5][RX-S]  14.00-15.00  sec  53.2 MBytes   446 Mbits/sec
[  8][TX-S]  14.00-15.00  sec  5.00 MBytes  42.0 Mbits/sec   72    122 KBytes
[  5][RX-S]  15.00-16.00  sec  54.3 MBytes   455 Mbits/sec
[  8][TX-S]  15.00-16.00  sec  1.25 MBytes  10.5 Mbits/sec  137   39.6 KBytes
[  5][RX-S]  16.00-17.00  sec  55.2 MBytes   463 Mbits/sec
[  8][TX-S]  16.00-17.00  sec  1.25 MBytes  10.5 Mbits/sec   16   55.1 KBytes
[  5][RX-S]  17.00-18.00  sec  55.1 MBytes   463 Mbits/sec
[  8][TX-S]  17.00-18.00  sec  3.75 MBytes  31.5 Mbits/sec    0   86.3 KBytes
[  5][RX-S]  18.00-19.00  sec  52.6 MBytes   441 Mbits/sec
[  8][TX-S]  18.00-19.00  sec  3.75 MBytes  31.5 Mbits/sec    0    123 KBytes
[  5][RX-S]  19.00-20.00  sec  51.3 MBytes   430 Mbits/sec
[  8][TX-S]  19.00-20.00  sec  6.25 MBytes  52.4 Mbits/sec    0    156 KBytes
[  5][RX-S]  20.00-21.00  sec  52.3 MBytes   439 Mbits/sec
[  8][TX-S]  20.00-21.00  sec  6.25 MBytes  52.4 Mbits/sec    0    184 KBytes
[  5][RX-S]  21.00-22.00  sec  52.2 MBytes   438 Mbits/sec
[  8][TX-S]  21.00-22.00  sec  6.25 MBytes  52.5 Mbits/sec    0    211 KBytes
[  5][RX-S]  22.00-23.00  sec  51.2 MBytes   430 Mbits/sec
[  8][TX-S]  22.00-23.00  sec  8.75 MBytes  73.4 Mbits/sec    0    235 KBytes
[  5][RX-S]  22.00-23.00  sec  51.2 MBytes   430 Mbits/sec
[  8][TX-S]  22.00-23.00  sec  8.75 MBytes  73.4 Mbits/sec    0    235 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][RX-S]   0.00-23.00  sec   873 MBytes   318 Mbits/sec                  receiver
[  8][TX-S]   0.00-23.00  sec   358 MBytes   131 Mbits/sec  1102             sender

And for just for testing, this is a reverse --bidir test

# Reverse --bidir 
# WLAN.HK.2.9.0.1-01890
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  50.8 MBytes   427 Mbits/sec    0   2.47 MBytes
[  5]   1.00-2.00   sec  55.0 MBytes   461 Mbits/sec   95   1.33 MBytes
[  5]   2.00-3.00   sec  52.5 MBytes   440 Mbits/sec    0   1.40 MBytes
[  5]   3.00-4.00   sec  51.2 MBytes   430 Mbits/sec   32   1.05 MBytes
[  5]   4.00-5.00   sec  46.2 MBytes   388 Mbits/sec  147    817 KBytes
[  5]   5.00-6.00   sec  43.8 MBytes   367 Mbits/sec  207    624 KBytes
[  5]   6.00-7.00   sec  45.0 MBytes   377 Mbits/sec    0    679 KBytes
[  5]   7.00-8.00   sec  45.0 MBytes   377 Mbits/sec    0    727 KBytes
[  5]   8.00-9.00   sec  45.0 MBytes   377 Mbits/sec    0    772 KBytes
[  5]   9.00-10.00  sec  46.2 MBytes   388 Mbits/sec    0    819 KBytes
[  5]  10.00-11.00  sec  48.8 MBytes   409 Mbits/sec    0    863 KBytes
[  5]  11.00-12.00  sec  46.2 MBytes   388 Mbits/sec    0    904 KBytes
[  5]  12.00-13.00  sec  46.2 MBytes   388 Mbits/sec    0    943 KBytes
[  5]  13.00-14.00  sec  47.5 MBytes   398 Mbits/sec    0    980 KBytes
[  5]  14.00-15.00  sec  46.2 MBytes   388 Mbits/sec    0   1015 KBytes
[  5]  15.00-16.00  sec  47.5 MBytes   398 Mbits/sec    0   1.03 MBytes
[  5]  16.00-17.00  sec  50.0 MBytes   419 Mbits/sec    0   1.06 MBytes
[  5]  17.00-18.00  sec  50.0 MBytes   419 Mbits/sec    0   1.10 MBytes
[  5]  18.00-19.00  sec  50.0 MBytes   419 Mbits/sec    0   1.13 MBytes
[  5]  19.00-20.00  sec  46.2 MBytes   388 Mbits/sec    0   1.16 MBytes
[  5]  20.00-21.00  sec  50.0 MBytes   419 Mbits/sec    0   1.22 MBytes
[  5]  21.00-22.00  sec  51.2 MBytes   430 Mbits/sec    0   1.40 MBytes
[  5]  22.00-23.00  sec  50.0 MBytes   419 Mbits/sec    0   1.62 MBytes

My question is why am I seeing the drops in performace for [TX-S] ? Is that the tcp_congestion_control or aql/airtime fairness? Should both drop and stay balanced?

Hi @dtaht , I'm just curious what Flent test is good to run to check for performance?
Is there any config/settings I should tune on the server or the client?

Edit: IPQ8074 does not support AQL or AT.

both runs are odd. rrul_be please?

Here you go. Did a reboot of my rpi to be sure and AP is running wlan.hk.2.7.

# wlan.hk.2.7 - cubic
 Summary of rrul_be test run from 2023-11-01 11:56:08.460212

                              avg       median       99th %          # data pts
 Ping (ms) ICMP    :       195.13       229.00       317.20 ms              349
 Ping (ms) UDP BE1 :        10.41        25.82       100.27 ms              350
 Ping (ms) UDP BE2 :        11.73        26.17        94.58 ms              350
 Ping (ms) UDP BE3 :        11.30        26.06       100.84 ms              350
 Ping (ms) avg     :        11.15          N/A          N/A ms              350
 TCP download BE   :        41.75        41.02       105.79 Mbits/s         350
 TCP download BE2  :        92.21        98.39       154.29 Mbits/s         350
 TCP download BE3  :        49.68        52.31       125.88 Mbits/s         350
 TCP download BE4  :       106.90       110.24       165.26 Mbits/s         350
 TCP download avg  :        72.64          N/A          N/A Mbits/s         350
 TCP download sum  :       290.54          N/A          N/A Mbits/s         350
 TCP totals        :       486.61          N/A          N/A Mbits/s         350
 TCP upload BE     :        35.43        30.52       238.65 Mbits/s         350
 TCP upload BE2    :        49.10        36.78       122.08 Mbits/s         350
 TCP upload BE3    :        60.49        64.55       179.03 Mbits/s         350
 TCP upload BE4    :        51.05        50.23       109.31 Mbits/s         350
 TCP upload avg    :        49.02          N/A          N/A Mbits/s         350
 TCP upload sum    :       196.07          N/A          N/A Mbits/s         350

Edit: Updated test.

Edit again: I believe there is a major problem somewhere that cause my upload to be very very bad compared to download.
I tested with 2 other AP's and the results was the same. One AP is ipq8074 and the second is IPQ6018 on a stock firmware. At first I thought it could be the RPI, but doing a simple iperf3 test from a Nuc to a RPI show 970 -ish something.

If other people with ipq8074 like to post your iperf3 --bidir result or rrul_be please do.

But don't waste any more of your time on this @dtaht :slight_smile: I have to figure out what the problem might be first, then we can solve the other stuff :+1:

Edit: Just a follow up... it's not the best, but I believe it's related... iperf3 on the AP itself. It should handle bidir right? I've seen balanced results on the IPQ8074 from other posts.

[  8][TX-S]  18.00-19.00  sec  5.78 MBytes  48.4 Mbits/sec    0   1.05 MBytes
[  5][RX-S]  19.00-20.00  sec  31.5 MBytes   264 Mbits/sec
[  8][TX-S]  19.00-20.00  sec  4.78 MBytes  40.1 Mbits/sec    0   1.17 MBytes
[  5][RX-S]  20.00-21.01  sec  30.1 MBytes   252 Mbits/sec
[  8][TX-S]  20.00-21.01  sec  5.00 MBytes  41.7 Mbits/sec    0   1.17 MBytes
[  5][RX-S]  21.01-22.00  sec  31.8 MBytes   267 Mbits/sec
[  8][TX-S]  21.01-22.00  sec  5.00 MBytes  42.1 Mbits/sec    0   1.17 MBytes
[  5][RX-S]  22.00-23.01  sec  32.0 MBytes   267 Mbits/sec
[  8][TX-S]  22.00-23.01  sec  5.00 MBytes  41.7 Mbits/sec    0   1.17 MBytes
[  5][RX-S]  23.01-24.00  sec  29.0 MBytes   245 Mbits/sec
[  8][TX-S]  23.01-24.00  sec  5.00 MBytes  42.3 Mbits/sec    0   1.67 MBytes
[  5][RX-S]  24.00-25.00  sec  30.8 MBytes   258 Mbits/sec
[  8][TX-S]  24.00-25.00  sec  3.75 MBytes  31.4 Mbits/sec    0   1.67 MBytes

And LAN (my RPI) to LAN (IPQ807x) in use and with traffic...

[  5][TX-C]   3.00-4.00   sec  73.8 MBytes   619 Mbits/sec    0    461 KBytes
[  7][RX-C]   3.00-4.00   sec  99.4 MBytes   834 Mbits/sec
[  5][TX-C]   4.00-5.00   sec  76.8 MBytes   645 Mbits/sec    0    505 KBytes
[  7][RX-C]   4.00-5.00   sec  98.8 MBytes   829 Mbits/sec
[  5][TX-C]   5.00-6.00   sec  76.8 MBytes   644 Mbits/sec    0    535 KBytes
[  7][RX-C]   5.00-6.00   sec  95.9 MBytes   805 Mbits/sec
[  5][TX-C]   6.00-7.01   sec  96.5 MBytes   801 Mbits/sec    0    568 KBytes
[  7][RX-C]   6.00-7.01   sec  59.4 MBytes   493 Mbits/sec
[  5][TX-C]   7.01-8.00   sec  92.4 MBytes   782 Mbits/sec    0    604 KBytes

Hi @robimarko. Since you are familiar with ipq807x, I thought I should ask if you could give me a hint/tip to look for when my WLAN to LAN performance is not so good.

In my last edit you can see that LAN to LAN is capable of more speed, but from WLAN to LAN, as the edit above, I get very low TX-S speed. Same for both my ipq807x AP's.

My wifi settings are basic, I've tried to set a more balanced IRQ, I used ethtool to enable/disable offloads. I have tried several WLAN.HK firmwares too, but no luck.
But none of those do any good. So I'm a little stuck.

So my next thing was to try the latest commit from both qca-ssdk & nss-dp. nss-dp failed on 0006-nss_dp_main-Use-a-phy-handle-property-to-connect-to-.patch.

I also had a look at my AP's DTS files, but I can't see anything missing comparing other DTS files.

Could it be the switch or wlan.hk that is limiting the performance?

Thanks in advance.

Well, I suspect that is it due to the basic ethernet driver which offers no offloading at all and is coupled with a bad switch driver.

Alright. Thanks :+1:

do not do iperf on the ap?

How the h#!% are you able to get so low ping latency? 20ms... :champagne:
ipq8074 is a different chapter i guess.

I have applied your patches MS2TIME stuff + NAPI_POLL_WEIGHT, and doing some testing. So far...

This is WiFi AC@40MHz NSS3 over 8-ish meters with a chair and a wall blocking.
After patches...

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  7][TX-C]   0.00-30.01  sec   914 MBytes   255 Mbits/sec                  sender
[  7][TX-C]   0.00-30.02  sec   914 MBytes   255 Mbits/sec                  receiver
[  9][RX-C]   0.00-30.01  sec   150 MBytes  42.0 Mbits/sec  678             sender
[  9][RX-C]   0.00-30.02  sec   147 MBytes  41.1 Mbits/sec                  receiver

to this with some more adjustments

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  7][TX-C]   0.00-30.00  sec   491 MBytes   137 Mbits/sec                  sender
[  7][TX-C]   0.00-30.02  sec   489 MBytes   137 Mbits/sec                  receiver
[  9][RX-C]   0.00-30.00  sec   664 MBytes   186 Mbits/sec    0             sender
[  9][RX-C]   0.00-30.02  sec   661 MBytes   185 Mbits/sec                  receiver

I know it's iperf3 and probably not great, but it shows some data, and a difference :slight_smile:

But still struggling with 300+ on icmp during a rrul_be tests. And when you show your data (the cerowrt post), I feel a little sad inside. :upside_down_face:

Just a little follow up on this, and --bidir etc... I started to notice that my iPhone mini and Macbook Air with AX, had lower performance doing simple speedtests. It was very clear that the latency was all over the place doing a test compared to my MBP with AC.

Anyway, I'm posting some simple test results from a Macbook Air, when I have AX enabled vs disabled.

AX enabled  = HE40  (AX)
AX disabled = VHT40 (AC)

Note: This did not do anything in terms of performance.
option vht160 '0'
option short_gi_160 '0'

Below is speedtest results

Below is from waveform

And last, a little test with iperf3 --bidir to see if it improves with AC only (AX disabled). Hmm... that's better.

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  7][TX-C]   0.00-10.00  sec   173 MBytes   145 Mbits/sec    0             sender
[  7][TX-C]   0.00-10.03  sec   172 MBytes   144 Mbits/sec                  receiver
[  9][RX-C]   0.00-10.00  sec   169 MBytes   142 Mbits/sec    0             sender
[  9][RX-C]   0.00-10.03  sec   165 MBytes   138 Mbits/sec                  receiver

So there you go. Now I'm running AC mode, and just plain browsing feels quicker too.
To be clear. I have tested wlan.hk.2.6 up to 2.9 and no difference. I did test NSS for IPQ807x too but no improvement. The idea of testing this came from this thread: