R7500v2 (ath10k 9980) netperf observations, firmware/board-2.bin files, and other wifi issues


I'm getting ready to build and test an image from master (I'm going to pull in the patch from greearb for dma_burst/fwcfg and your patch in the post above assuming its ready for a test). If your patch for the pre-cal is ready, I can pull that in as well... but no rush. I'll likely have another chance in a week or two.

the patch just put task that do openwrt directly in the firmware so if you want you can skip them... they don't really give any benefits

1 Like

poor wifi quality continues...

  • Using the non-ct driver and firmware on 19.07.4: no change, same symptoms.

  • Eliminating multiple (2) SSID's (home & guest networks) on the same band: same symptoms. (I do use VLANs and I did leave 2.4 with only one SSID on the guest network).

  • moving back to master and ath10k-ct driver/firmware, setting dma_burst to 0: same symptoms.

  • AP/client location seem less likely now: here are netperf and dsl reports results from a client ~3 m from the AP with a clear line of site (5 clients total on 5 GHz, light traffic if any from other clients, no clients on 2.4):

Only one client running netperf over 5 GHz wifi (single SSID, dma_burst=0) to a wired client connected to the AP:

(p383.mkl) [6] $ netperf -l 60 -D 1s -H
MIGRATED TCP STREAM TEST from ( port 0 AF_INET to () port 0 AF_INET : demo
Interim result:  169.92 10^6bits/s over 1.051 seconds ending at 1600602418.733
Interim result:  166.38 10^6bits/s over 1.021 seconds ending at 1600602419.754
Interim result:  155.83 10^6bits/s over 1.068 seconds ending at 1600602420.822
Interim result:  169.03 10^6bits/s over 1.036 seconds ending at 1600602421.858
Interim result:  162.64 10^6bits/s over 1.040 seconds ending at 1600602422.898
Interim result:  151.42 10^6bits/s over 1.073 seconds ending at 1600602423.971
Interim result:  154.90 10^6bits/s over 1.048 seconds ending at 1600602425.020
Interim result:  159.24 10^6bits/s over 1.041 seconds ending at 1600602426.061
Interim result:  132.61 10^6bits/s over 1.201 seconds ending at 1600602427.262
Interim result:   79.60 10^6bits/s over 1.666 seconds ending at 1600602428.928
Interim result:   73.97 10^6bits/s over 1.077 seconds ending at 1600602430.005
Interim result:   87.02 10^6bits/s over 1.155 seconds ending at 1600602431.161
Interim result:  130.95 10^6bits/s over 1.064 seconds ending at 1600602432.225
Interim result:  186.55 10^6bits/s over 1.029 seconds ending at 1600602433.253
Interim result:  176.75 10^6bits/s over 1.055 seconds ending at 1600602434.308
Interim result:  174.47 10^6bits/s over 1.041 seconds ending at 1600602435.349
Interim result:  177.40 10^6bits/s over 1.043 seconds ending at 1600602436.391
Interim result:  176.05 10^6bits/s over 1.060 seconds ending at 1600602437.452
Interim result:  183.83 10^6bits/s over 1.012 seconds ending at 1600602438.464
Interim result:  171.61 10^6bits/s over 1.072 seconds ending at 1600602439.536
Interim result:  184.32 10^6bits/s over 1.001 seconds ending at 1600602440.536
Interim result:  176.31 10^6bits/s over 1.046 seconds ending at 1600602441.582
Interim result:  175.99 10^6bits/s over 1.031 seconds ending at 1600602442.613
Interim result:  175.99 10^6bits/s over 1.047 seconds ending at 1600602443.660
Interim result:  180.15 10^6bits/s over 1.040 seconds ending at 1600602444.701
Interim result:  183.64 10^6bits/s over 1.025 seconds ending at 1600602445.726
Interim result:  176.83 10^6bits/s over 1.038 seconds ending at 1600602446.764
Interim result:  177.46 10^6bits/s over 1.054 seconds ending at 1600602447.818
Interim result:  158.57 10^6bits/s over 1.199 seconds ending at 1600602449.017
Interim result:   87.76 10^6bits/s over 1.806 seconds ending at 1600602450.823
Interim result:   80.01 10^6bits/s over 1.098 seconds ending at 1600602451.920
Interim result:   89.63 10^6bits/s over 1.139 seconds ending at 1600602453.060
Interim result:   91.52 10^6bits/s over 1.318 seconds ending at 1600602454.377
Interim result:   91.15 10^6bits/s over 1.005 seconds ending at 1600602455.383
Interim result:   83.94 10^6bits/s over 1.085 seconds ending at 1600602456.468
Interim result:   91.69 10^6bits/s over 1.152 seconds ending at 1600602457.620
Interim result:  101.04 10^6bits/s over 1.047 seconds ending at 1600602458.667
Interim result:  181.07 10^6bits/s over 1.056 seconds ending at 1600602459.723
Interim result:  175.92 10^6bits/s over 1.054 seconds ending at 1600602460.777
Interim result:  181.32 10^6bits/s over 1.037 seconds ending at 1600602461.814
Interim result:  165.14 10^6bits/s over 1.122 seconds ending at 1600602462.935
Interim result:   89.10 10^6bits/s over 1.853 seconds ending at 1600602464.789
Interim result:   96.78 10^6bits/s over 1.039 seconds ending at 1600602465.828
Interim result:   81.39 10^6bits/s over 1.188 seconds ending at 1600602467.016
Interim result:   94.18 10^6bits/s over 1.058 seconds ending at 1600602468.074
Interim result:   79.43 10^6bits/s over 1.186 seconds ending at 1600602469.260
Interim result:   91.72 10^6bits/s over 1.145 seconds ending at 1600602470.405
Interim result:   85.36 10^6bits/s over 1.073 seconds ending at 1600602471.478
Interim result:   85.96 10^6bits/s over 1.081 seconds ending at 1600602472.559
Interim result:  178.41 10^6bits/s over 1.064 seconds ending at 1600602473.623
Interim result:  180.90 10^6bits/s over 1.027 seconds ending at 1600602474.650
Interim result:  177.20 10^6bits/s over 1.043 seconds ending at 1600602475.693
Interim result:  180.33 10^6bits/s over 1.024 seconds ending at 1600602476.717
Interim result:  180.96 10^6bits/s over 0.964 seconds ending at 1600602477.682

dsl reports, same client on 5 GHz wifi, one SSID, dma_burst=0, followed by a test on ethernet (same client but its wifi turned off):

Connection Time                 Down 	Up 	    Ping 	B	Q	S	Link	ISP	Streams
ethernet   2020-09-20 07:50:22	24.19	2.51	22  	A	A	A	Cable	16 / 16 	
5 GHz wifi 2020-09-20 07:48:48	16.72	2.47	24	    B	C	B	Cable	16 / 16 	

Note the wifi results started out looking like the ethernet results in the first part of the test but then the rate dropped.

@slh, @Ansuel apologies for bothering but have you seen results like this? I use to get results like the ethernet results above on wifi all the time - even with multiple SSIDs and more clients on wifi.

I still need to look into wifi interference from neighbors...

Problem is that I can't do lots of test since the router is in the same room of my device and my home is very small :frowning:

1 Like

Router: ZyXEL nbg6817 ()
Client: Intel Pentium G2020, Intel AX200 802.11ax card
netperf server: Intel i7-3770, RTL8168f/8111f 1 GBit/s ethernet, directly connected to the nbg6817

[    8.178564] iwlwifi 0000:02:00.0: enabling device (0100 -> 0102)
[    8.263497] iwlwifi 0000:02:00.0: Direct firmware load for iwlwifi-cc-a0-56.ucode failed with error -2
[    8.263500] iwlwifi 0000:02:00.0: Falling back to sysfs fallback for: iwlwifi-cc-a0-56.ucode
[    8.285924] usbcore: registered new interface driver btusb
[    8.291531] Bluetooth: hci0: Firmware revision 0.0 build 188 week 26 2020
[    8.365955] iwlwifi 0000:02:00.0: api flags index 2 larger than supported by driver
[    8.365964] iwlwifi 0000:02:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version:
[    8.365967] iwlwifi 0000:02:00.0: Found debug destination: EXTERNAL_DRAM
[    8.365968] iwlwifi 0000:02:00.0: Found debug configuration: 0
[    8.366189] iwlwifi 0000:02:00.0: loaded firmware version 55.d9698065.0 cc-a0-55.ucode op_mode iwlmvm
[    8.421761] iwlwifi 0000:02:00.0: Direct firmware load for iwl-debug-yoyo.bin failed with error -2
[    8.421763] iwlwifi 0000:02:00.0: Falling back to sysfs fallback for: iwl-debug-yoyo.bin
[    8.742331] checking generic (e0000000 300000) vs hw (f7800000 400000)
[    8.742333] checking generic (e0000000 300000) vs hw (e0000000 10000000)

# wpa_cli -i wlp2s0 status | grep -v -e bssid -e ssid -e address -e uuid
[   22.434504] ath10k_pci 0000:01:00.0: enabling device (0140 -> 0142)
[   22.434582] ath10k_pci 0000:01:00.0: enabling bus mastering
[   22.435161] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[   27.894598] ath10k_pci 0000:01:00.0: qca9984/qca9994 hw1.0 target 0x01000000 chip_id 0x00000000 sub 168c:cafe
[   27.894646] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   27.910214] ath10k_pci 0000:01:00.0: firmware ver 10.4-3.10-00047 api 5 features no-p2p,mfp,peer-flow-ctrl,btcoex-param,allows-mesh-bcast,no-ps crc32 19ca6df2
[   30.179642] ath10k_pci 0000:01:00.0: board_file api 2 bmi_id 0:1 crc32 85498734
[   33.932002] ath10k_pci 0000:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 512 raw 0 hwcrypto 1
[   33.995054] ath: EEPROM regdomain sanitized
[   33.995070] ath: EEPROM regdomain: 0x64
[   33.995082] ath: EEPROM indicates we should expect a direct regpair map
[   33.995108] ath: Country alpha2 being used: 00
[   33.995117] ath: Regpair used: 0x64

root@nbg6817:~# iwinfo wlan1 assoc
No station connected
root@nbg6817:~# iwinfo wlan0 assoc
94:E6:F7:XX:XX:XX  -67 dBm / -104 dBm (SNR 37)  4 ms ago
        RX: 650.0 MBit/s, VHT-MCS 7, 80MHz, VHT-NSS 2   3091244 Pkts.
        TX: 650.0 MBit/s, VHT-MCS 7, 80MHz, VHT-NSS 2   1618468 Pkts.
        expected throughput: unknown
$ netperf -l 60 -D 1s -H
MIGRATED TCP STREAM TEST from ( port 0 AF_INET to () port 0 AF_INET : demo
Interim result:  343.31 10^6bits/s over 1.019 seconds ending at 1600628250.666
Interim result:  488.19 10^6bits/s over 1.025 seconds ending at 1600628251.691
Interim result:  462.06 10^6bits/s over 1.057 seconds ending at 1600628252.748
Interim result:  483.65 10^6bits/s over 1.002 seconds ending at 1600628253.750
Interim result:  471.53 10^6bits/s over 1.026 seconds ending at 1600628254.775
Interim result:  473.47 10^6bits/s over 1.038 seconds ending at 1600628255.814
Interim result:  448.87 10^6bits/s over 1.055 seconds ending at 1600628256.868
Interim result:  447.37 10^6bits/s over 1.025 seconds ending at 1600628257.894
Interim result:  445.49 10^6bits/s over 1.004 seconds ending at 1600628258.898
Interim result:  477.26 10^6bits/s over 1.021 seconds ending at 1600628259.919
Interim result:  476.78 10^6bits/s over 1.001 seconds ending at 1600628260.920
Interim result:  483.33 10^6bits/s over 1.005 seconds ending at 1600628261.925
Interim result:  471.66 10^6bits/s over 1.025 seconds ending at 1600628262.950
Interim result:  466.82 10^6bits/s over 1.010 seconds ending at 1600628263.960
Interim result:  486.79 10^6bits/s over 1.011 seconds ending at 1600628264.971
Interim result:  484.27 10^6bits/s over 1.034 seconds ending at 1600628266.005
Interim result:  481.88 10^6bits/s over 1.005 seconds ending at 1600628267.010
Interim result:  478.46 10^6bits/s over 1.022 seconds ending at 1600628268.032
Interim result:  477.87 10^6bits/s over 1.001 seconds ending at 1600628269.033
Interim result:  477.30 10^6bits/s over 1.001 seconds ending at 1600628270.035
Interim result:  474.25 10^6bits/s over 1.013 seconds ending at 1600628271.048
Interim result:  477.19 10^6bits/s over 1.015 seconds ending at 1600628272.063
Interim result:  479.29 10^6bits/s over 1.021 seconds ending at 1600628273.084
Interim result:  462.94 10^6bits/s over 1.035 seconds ending at 1600628274.119
Interim result:  471.07 10^6bits/s over 1.004 seconds ending at 1600628275.124
Interim result:  454.71 10^6bits/s over 1.036 seconds ending at 1600628276.160
Interim result:  469.39 10^6bits/s over 1.012 seconds ending at 1600628277.172
Interim result:  485.00 10^6bits/s over 1.036 seconds ending at 1600628278.208
^CInterim result:  506.74 10^6bits/s over 0.416 seconds ending at 1600628278.624
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

131072  16384  16384    29.02     467.66
1 Like

Another "just for fun" worked example showing how to enable/disable mu-mimo aka multi-user beamforming on the ath10k-ct driver/firmware by using the ath10k-ct fwcfg api. It's "just for fun" as mu-mimo enabled/disabled does not affect my netperf observations with the clients I test with - perhaps this might help others.

Note that it may be possible to accomplish the same result via hostapd but that is currently not as straight forward on openwrt. More about hostapd tweaks later.

There is a ddwrt forum post here recommending to disable mu-mimo support apparently based on this article. Apparently, some broadcom wifi cards (commonly used in apple devices) are incompatible with certain (non-broadcom) mu-mimo implementations. The issue seems to be with the broadcom card (i.e. it is not a ath10k-ct bug) and the result is poor wifi performance if/when mu-mimo is used.

One of the mac osx clients I'm testing with does use a broadcom wifi card (a bcm4355, for which I have trouble finding clear specs) - but this card does not seem to support beamforming. See below for how to check the AP's view of it's connected client vht capabilities.

Looking over the ath10k-ct driver core.c it looks there is support to enable/disable "beamforming" (both mu and su) in the ct-firmware by using the ct fwcfg api.

Hostapd's vht_cpapb value can be used to show if mu-mimo (aka MU-BEAMFORME[E|R]) is enabled/disabled via:

r7500v2 # cat /var/run/hostapd-phy0.conf | grep BEAM

To disable mu-mimo:

r7500v2 # cat /sys/kernel/debug/ieee80211/phy1/ath10k/firmware_info | grep fwcfg
fwcfg: fwcfg-pci-0001:01:00.0.txt
r7500v2 # cat /sys/kernel/debug/ieee80211/phy0/ath10k/firmware_info | grep fwcfg
fwcfg:     fwcfg-pci-0000:01:00.0.txt
echo "nobeamform_mu = 1" >> /lib/firmware/ath10k/fwcfg-pci-0000:01:00.0.txt
echo "nobeamform_mu = 1" >> /lib/firmware/ath10k/fwcfg-pci-0001:01:00.0.txt

and reboot the AP.

After the reboot:

r7500v2 # dmesg | grep fwcfg
[   16.531378] ath10k_pci 0000:01:00.0: fwcfg key: nobeamform_mu  val: 1
[   18.795420] ath10k_pci 0001:01:00.0: fwcfg key: nobeamform_mu  val: 1
r7500v2 # cat /var/run/hostapd-phy0.conf | grep vht_capab

So it looks like the ath10k-ct driver picks up the fwcfg setting. Given that the hostapd vht_capab parameter no longer contains any MU-BEAMFORME[E|R] values, I'm guessing it is now disabled.

I'm not sure how to see if mu-mimo is actually being used when it is enabled. It is interesting to see the ct dev asking similar questions 5 yr ago here - I don't see "BF-ANTENNA-4" or "SOUNDING-DIMENSION-4" in my /var/run/hostapd-phy0.conf (ref: ath10k config wiki). See this post if you want to try enabling these via hostapd.

One can check what the AP's thinks its client vht capabilities are via

r7500v2 # cat /sys/kernel/debug/ieee80211/phy0/netdev:wlan0/stations/*/vht_capa
VHT supported
cap: 0x0f815832
                BEAMFORMEE-STS: 0x2
                SOUNDING-DIMENSIONS: 0x1
                MPDU-LENGTH-EXPONENT: 0x7
                LINK-ADAPTATION-VHT-MRQ-MFB: 0x3
RX MCS: fffa
TX MCS: fffa
VHT supported
cap: 0x039179b1
                BEAMFORMEE-STS: 0x3
                SOUNDING-DIMENSIONS: 0x1
                MPDU-LENGTH-EXPONENT: 0x7
                LINK-ADAPTATION-VHT-MRQ-MFB: 0x0
RX MCS: fffa
TX MCS: fffa

I have an intel wifi client that does report support for mu-mimo but this is the only one and I think I need at least two (non-broadcom) mu-mimo devices to test.

@slh and/or @ansuel, if either of you have a minute and have a device running a recent master build with the ath10k-ct driver and firmware, could you do iw list and post the output below "Supported extended features:" for the 5GHz phy?

I'm looking for a line like:

[ AIRTIME_FAIRNESS ]: airtime fairness scheduling

For example, the output from a r7800 shown here does output such a line.

The output from the r7500v2 (built Nov 8 with kernel 5.10.78, ath10k-firmware-qca99x0-ct-full-htt - 2020-11-08-1 and kmod-ath10k-ct - 5.10.78+2021-09-22-e6a7d5b5-1)

       Supported extended features:
                * [ VHT_IBSS ]: VHT-IBSS
                * [ RRM ]: RRM
                * [ SET_SCAN_DWELL ]: scan dwell setting
                * [ FILS_STA ]: STA FILS (Fast Initial Link Setup)
                * [ CQM_RSSI_LIST ]: multiple CQM_RSSI_THOLD records
                * [ CONTROL_PORT_OVER_NL80211 ]: control port over nl80211
                * [ TXQS ]: FQ-CoDel-enabled intermediate TXQs
                * [ AQL ]: Airtime Queue Limits (AQL)
                * [ CONTROL_PORT_NO_PREAUTH ]: disable pre-auth over nl80211 control port support
                * [ DEL_IBSS_STA ]: deletion of IBSS station support
                * [ SCAN_FREQ_KHZ ]: scan on kHz frequency support
                * [ CONTROL_PORT_OVER_NL80211_TX_STATUS ]: tx status for nl80211 control port support

i.e. there is no indication of airtime fairness support.

Thank you.

Can you check the hostpad config? I think it's not enabled there.

1 Like

What should i look for?

presumably i should see something in /var/run/hostapd-phy0.conf

yes that should be the location for the config

ya, but i have no idea what should be there for airtime fariness to be enabled.

my /var/run/hostapd-phy0.conf (some values redacted)



wpa_key_mgmt=WPA-PSK WPA-PSK-SHA256

i assume it should be added to vht or ht capab you should check how airtime fairness should be enabled from hostapd

did you check Controlling airtime fairness

also i can see this

+	} else if (os_strcmp(buf, "airtime_mode") == 0) {
+		conf->airtime_mode = atoi(pos);
+	} else if (os_strcmp(buf, "airtime_update_interval") == 0) {
+		conf->airtime_update_interval = atoi(pos);
+	} else if (os_strcmp(buf, "airtime_bss_weight") == 0) {
+		bss->airtime_weight = atoi(pos);
+	} else if (os_strcmp(buf, "airtime_bss_limit") == 0) {
+		bss->airtime_limit = atoi(pos);
+	} else if (os_strcmp(buf, "airtime_sta_weight") == 0) {
+		if (add_airtime_weight(bss, pos) < 0) {
+			wpa_printf(MSG_DEBUG, "Line %d: Invalid airtime weight '%s'",
+				   line, pos);
+			return 1;
+		}


+##### Airtime policy configuration ###########################################
+# Set the airtime policy operating mode:
+# 0 = disabled (default)
+# 1 = static config
+# 2 = per-BSS dynamic config
+# 3 = per-BSS limit mode
+# Interval (in milliseconds) to poll the kernel for updated station activity in
+# dynamic and limit modes
+# Static configuration of station weights (when airtime_mode=1). Kernel default
+# weight is 256; set higher for larger airtime share, lower for smaller share.
+# Each entry is a MAC address followed by a weight.
+#airtime_sta_weight=02:01:02:03:04:05 256
+#airtime_sta_weight=02:01:02:03:04:06 512
+# Per-BSS airtime weight. In multi-bss mode, set for each BSS and hostapd will
+# configure station weights to enforce the correct ratio between BSS weights
+# depending on the number of active stations. The *ratios* between different
+# BSSes is what's important, not the absolute numbers.
+# Must be set for all BSSes if airtime_mode=2 or 3, has no effect otherwise.
+# Whether the current BSS should be limited (when airtime_mode=3).
+# If set, the BSS weight ratio will be applied in the case where the current BSS
+# would exceed the share defined by the BSS weight ratio. E.g., if two BSSes are
+# set to the same weights, and one is set to limited, the limited BSS will get
+# no more than half the available airtime, but if the non-limited BSS has more
+# stations active, that *will* be allowed to exceed its half of the available
+# airtime.

think you should tweak the hostpad.sh to add these extra feature (hardcode them to quickly test and check if the value appear in the supported values)

yes, i've been there and i've looked for some config variable to put in the hostapd config but so far its not clear to me what should be there.

Is it possible for you to check if your iw list output has an airtime fairness line and, if so, what might be in your hostapd config file that is not in mine?

Note i have built and tested the ath10k-ct driver with and without commenting out support for "airtime fairness" here.

There is no difference for me either way.

This in combination with my iw list output makes me suspect "airtime fairness" is not enabled in the ath10k-ct 99x0 firmware. Before I ping @greearb, I want to confirm that it is working for others.

I'll try playing with the config variables you posted above and see if shows up.

1 Like

@nmrh check the updated post ahahha i posted the missing feature and in fact it looks to be disabled by default...

if you notice it does work I can think of pushing a patch to add support for it and add support to make it configurable on openwrt. (or also consider setting it to a default sane value)

1 Like

again just to make sure... hostpad.conf should not be directly changed so you should tweak the shell script that generates that and restart wifi

i can edit the /var/run/hostapd-phy0.conf file and

kill -HUP $(pidof hostapd) # reload hostapd config

which i believe picks up the changes

oh ok using that won't regen the config...

wifi down
wifi up does tho

I think you should edit hostapd.sh or mac80211.sh as they are what generates the /var/run/hostapd-phy*.conf.


i just tried adding airtime_mode=0 and restarting hostapd with no change in iw list output.

/lib/netifd/hostapd.sh does not seem to have anything related to airtime fairness commented out (i.e. i think it should work there assuming it's enabled in the firmware).

I'll look at /lib/netifd/wireless/mac80211.sh now...

but it would be really nice if someone could answer my question "does it show up in your iw list output?" and if so "what is in your hostapd-phy0.conf file related to airtime?"

and why you set it to 0 ? it's the default option aka disabled....
try 2 pls