TP-Link Archer C7 v2 AC1750 ath10 firmware crashes (2)

Previous topic closed for replies >

TP-Link Archer C7 v2 AC1750 ath10 firmware crashes

It turned out I still got crashes with the non-ct firmware.

I've upgraded to 21.02.1 and reverted to the -ct firmware, and I'm still getting (infrequent) kernel warnings.

For example:

# cat /proc/cpuinfo 
system type		: Qualcomm Atheros QCA9558 ver 1 rev 0
machine			: TP-Link Archer C7 v2
processor		: 0
cpu model		: MIPS 74Kc V5.0
BogoMIPS		: 359.42
wait instruction	: yes
microsecond timers	: yes
tlb_entries		: 32
extra interrupt vector	: yes
hardware watchpoint	: yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb]
isa			: mips1 mips2 mips32r1 mips32r2
ASEs implemented	: mips16 dsp dsp2
Options implemented	: tlb 4kex 4k_cache prefetch mcheck ejtag llsc dc_aliases perf_cntr_intr_bit cdmm contextconfig perf
shadow register sets	: 1
kscratch registers	: 0
package			: 0
core			: 0
VCED exceptions		: not available
VCEI exceptions		: not available

# cat /etc/openwrt_release 
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='21.02.1'
DISTRIB_REVISION='r16325-88151b8303'
DISTRIB_TARGET='ath79/generic'
DISTRIB_ARCH='mips_24kc'
DISTRIB_DESCRIPTION='OpenWrt 21.02.1 r16325-88151b8303'
DISTRIB_TAINTS=''

# uptime
 16:46:48 up 14 days, 21:49,  load average: 0.10, 0.03, 0.00

dmesg shows no kernel warning for some time after booting,

[    0.000000] Linux version 5.4.154 (builder@buildhost) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r16325-88151b8303)) #0 Sun Oct 24 09:01:35 2021
...boot messages... (below filtered for 'ath')
[   46.118580] ath10k 5.10 driver, optimized for CT firmware, probing pci device: 0x3c.
[   46.139056] ath10k_pci 0000:00:00.0: enabling device (0000 -> 0002)
[   46.147711] ath10k_pci 0000:00:00.0: pci irq legacy oper_irq_mode 1 irq_mode 0 reset_mode 0
[   49.753130] ath10k_pci 0000:00:00.0: qca988x hw2.0 target 0x4100016c chip_id 0x043202ff sub 0000:0000
[   49.762541] ath10k_pci 0000:00:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   49.774766] ath10k_pci 0000:00:00.0: firmware ver 10.1-ct-8x-__fW-022-ecad3248 api 2 features wmi-10.x,has-wmi-mgmt-tx,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,txrate-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT crc32 3e4cf97f
[   50.201961] ath10k_pci 0000:00:00.0: board_file api 1 bmi_id N/A crc32 bebc7c08
[   51.156166] ath10k_pci 0000:00:00.0: 10.1 wmi init: vdevs: 16  peers: 127  tid: 256
[   51.173463] ath10k_pci 0000:00:00.0: wmi print 'P 128 V 8 T 410'
[   51.179823] ath10k_pci 0000:00:00.0: wmi print 'msdu-desc: 1424  sw-crypt: 0 ct-sta: 0'
[   51.187978] ath10k_pci 0000:00:00.0: wmi print 'alloc rem: 24984 iram: 38672'
[   51.241847] ath10k_pci 0000:00:00.0: htt-ver 2.1 wmi-op 2 htt-op 2 cal file max-sta 128 raw 0 hwcrypto 1
[   51.256109] ath10k_pci 0000:00:00.0: NOTE:  Firmware DBGLOG output disabled in debug_mask: 0x10000000
[   51.365603] ath: EEPROM regdomain sanitized
[   51.365612] ath: EEPROM regdomain: 0x64
[   51.365615] ath: EEPROM indicates we should expect a direct regpair map
[   51.365633] ath: Country alpha2 being used: 00
[   51.365636] ath: Regpair used: 0x64
[   51.439567] ath: EEPROM regdomain sanitized
[   51.439576] ath: EEPROM regdomain: 0x64
[   51.439580] ath: EEPROM indicates we should expect a direct regpair map
[   51.439599] ath: Country alpha2 being used: 00
[   51.439602] ath: Regpair used: 0x64
[   69.552897] ath: EEPROM regdomain: 0x8242
[   69.557002] ath: EEPROM indicates we should expect a country code
[   69.563196] ath: doing EEPROM country->regdmn map search
[   69.568592] ath: country maps to regdmn code: 0x37
[   69.573455] ath: Country alpha2 being used: NO
[   69.577967] ath: Regpair used: 0x37
[   69.581507] ath: regdomain 0x8242 dynamically updated by user
[   69.587419] ath: EEPROM regdomain: 0x8242
[   69.591499] ath: EEPROM indicates we should expect a country code
[   69.597682] ath: doing EEPROM country->regdmn map search
[   69.603076] ath: country maps to regdmn code: 0x37
[   69.607940] ath: Country alpha2 being used: NO
[   69.612445] ath: Regpair used: 0x37
[   69.615983] ath: regdomain 0x8242 dynamically updated by user
[   75.704996] ath10k_pci 0000:00:00.0: failed to receive initialized event from target: 00000000
[   75.714215] ath10k_pci 0000:00:00.0: failed to wait for target init: -145
[   75.721151] ath10k_pci 0000:00:00.0: failed to warm reset attempt 1 of 3: -145
[   77.815639] ath10k_pci 0000:00:00.0: 10.1 wmi init: vdevs: 16  peers: 127  tid: 256
[   77.832945] ath10k_pci 0000:00:00.0: wmi print 'P 128 V 8 T 410'
[   77.839302] ath10k_pci 0000:00:00.0: wmi print 'msdu-desc: 1424  sw-crypt: 0 ct-sta: 0'
[   77.847464] ath10k_pci 0000:00:00.0: wmi print 'alloc rem: 24984 iram: 38672'
[   77.915948] ath10k_pci 0000:00:00.0: pdev param 0 not supported by firmware
[   77.930854] ath10k_pci 0000:00:00.0: rts threshold -1
[   78.180690] ath10k_pci 0000:00:00.0: rts threshold -1
...boot messages...
[   78.324973] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0-1: link becomes ready
[307206.145780] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[307206.153350] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
...repeated 17 times...
[307206.423556] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped new beacon
[830516.561734] ------------[ cut here ]------------
[830516.566578] WARNING: CPU: 0 PID: 0 at backports-5.10.68-1/net/mac80211/rate.c:677 0x8779e700 [mac80211@c5bcfd30+0x7cdb0]
[830516.577699] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT x
t_MASQUERADE xt_FLOWOFFLOAD xt_CT pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multipo
rt xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables hwm
on crc_ccitt compat ledtrig_usbport nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 fsl_mph_dr_of ehci_
platform ehci_fsl ehci_hcd gpio_button_hotplug usbcore nls_base usb_common
[830516.640869] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.154 #0
[830516.646874] Stack : 87800000 800b98ec 80660000 805fa498 00000000 00000000 00000000 00000000
[830516.655442]         00000000 00000000 00000000 00000000 00000000 00000001 87c0bc30 dbb221d8
[830516.664012]         87c0bcc8 00000000 00000000 00000137 00000038 8057ac04 352e342e 31353420
[830516.672584]         00000137 28e4002a 00000000 73776170 00000000 87c0bc10 00000000 8779e700
[830516.681155]         00000009 877f6760 871581a0 80630000 00000002 8032041c 00000000 80790000
[830516.689726]         ...
[830516.692295] Call Trace:
[830516.692318] [<800b98ec>] 0x800b98ec
[830516.698514] [<80660000>] 0x80660000
[830516.702154] [<8057ac04>] 0x8057ac04
[830516.705792] [<8779e700>] 0x8779e700 [mac80211@c5bcfd30+0x7cdb0]
[830516.711895] [<8032041c>] 0x8032041c
[830516.715530] [<80069364>] 0x80069364
[830516.719151] [<8006936c>] 0x8006936c
[830516.722785] [<800824f4>] 0x800824f4
[830516.726424] [<8779e700>] 0x8779e700 [mac80211@c5bcfd30+0x7cdb0]
[830516.732518] [<8008259c>] 0x8008259c
[830516.736161] [<8779e700>] 0x8779e700 [mac80211@c5bcfd30+0x7cdb0]
[830516.742267] [<877ae728>] 0x877ae728 [mac80211@c5bcfd30+0x7cdb0]
[830516.748370] [<87010cd0>] 0x87010cd0 [ath10k_core@fcc86344+0x6bb00]
[830516.754730] [<800c3f10>] 0x800c3f10
[830516.758371] [<86c6991c>] 0x86c6991c [ath9k@491ac81d+0x19450]
[830516.764220] [<86c6cc84>] 0x86c6cc84 [ath9k@491ac81d+0x19450]
[830516.770065] [<86c64664>] 0x86c64664 [ath9k@491ac81d+0x19450]
[830516.775908] [<877b0d9c>] 0x877b0d9c [mac80211@c5bcfd30+0x7cdb0]
[830516.782012] [<877b0ef0>] 0x877b0ef0 [mac80211@c5bcfd30+0x7cdb0]
[830516.788122] [<877b8bf0>] 0x877b8bf0 [mac80211@c5bcfd30+0x7cdb0]
[830516.794233] [<877ba838>] 0x877ba838 [mac80211@c5bcfd30+0x7cdb0]
[830516.800342] [<86c64eb8>] 0x86c64eb8 [ath9k@491ac81d+0x19450]
[830516.806173] [<80660000>] 0x80660000
[830516.809806] [<80660000>] 0x80660000
[830516.813432] [<80085380>] 0x80085380
[830516.817061] [<800ba7e4>] 0x800ba7e4
[830516.820692] [<80580170>] 0x80580170
[830516.824315] [<800bf374>] 0x800bf374
[830516.827945] [<802bfc1c>] 0x802bfc1c
[830516.831578] [<80064f78>] 0x80064f78
[830516.835193] 
[830516.836795] ---[ end trace 581f1996b626da89 ]---

...repeated at...

[830517.103580] ------------[ cut here ]------------
[830517.661671] ------------[ cut here ]------------
[830518.138160] ------------[ cut here ]------------
[830518.546624] ------------[ cut here ]------------
[830518.957060] ------------[ cut here ]------------
[830519.366243] ------------[ cut here ]------------
[830519.980244] ------------[ cut here ]------------
[830520.496548] ------------[ cut here ]------------
[830521.106479] ------------[ cut here ]------------
[830521.597312] ------------[ cut here ]------------
[830521.909335] ------------[ cut here ]------------
[830522.338270] ------------[ cut here ]------------
[830522.623123] ------------[ cut here ]------------
[855501.265006] ------------[ cut here ]------------
[864218.482235] ------------[ cut here ]------------
[864218.895587] ------------[ cut here ]------------
[871556.505749] ------------[ cut here ]------------
[935452.434955] ------------[ cut here ]------------
[935452.713552] ------------[ cut here ]------------

Is this something I should be reporting to Candela Technologies in the hope something might be done?

1 Like

What is Candela Tech?

Candela Tech - see my reply in the previous thread, linked below.

tl;dr

The ath10 drivers used by default in OpenWrt are the modified drivers produced under licence by Candela Technologies. It is possible to use the original closed-source/binary blob drivers. Given that Candela Technologies are the upstream provider of the ath10k drivers used in OpenWrt, it might be appropriate to inform them of problems. On the other hand, as I'm not a paying customer, it might not be worth it.

1 Like

I replaced the CT drivers on my Archer C7 v2...no issues.

opkg remove ath10k-firmware-qca988x-ct kmod-ath10k-ct

opkg update && opkg install ath10k-firmware-qca988x kmod-ath10k
2 Likes

Thanks, I tried that before upgrading to 21.02.1, and still had problems, which is why I let the upgrade remain with the -ct drivers. Your suggestion might help others.

I suspect I have a challenging WiFi environment, with more than 50 networks visible* (high density housing development). I have an old 'Internet Radio' which sporadically cannot connect, which appears to be because it does not have enough slots defined in its software to hold the names of all the networks it can see - I have to keep power-cycling it until the right network is 'captured'. Unfortunately, its list of visible networks does not appear to be very dynamic, or ordered by signal strength.

*Just checked with wavemon on my PC: 58 networks visible.

You might try a USB wireless network adapter.

I use a TP-Link T2U for 5 Ghz on my laptop (no 5 Ghz onboard), and it works well.

A large proportion of the networks have the same ESSID on both 2.4 GHz and 5 GHz - that is the way the local ISP sets up their provided router/access points.

My experience of the TP-Link Archer C7 v2 on 5 GHz has not been brilliant, but I think that is a known problem. For my own use, I use a wired network but others connecting to the network via WiFi have poor results. Video conferences drop out - so I end up with long runs of cables draped temporarily about the place when important video conferences take place. The upstream connection is full Gigabit, so connections are not (local) capacity constrained - it is the WiFi performance that is the weak link.

Medium term plan is to get a decent Gigabit-capable wired router, a PoE switch and some (PoE) access points: but I would like to do that with OpenWrt on efficient hardware - I don't want a router and switch that act as room heaters. Something like the Ubiquiti EdgeRouter 4, which claims a power consumption of 13 Watts looks good, but might lack some CPU capacity.

I wouldn’t call ER4 slow and it actually have a crypto engine which almost non standard homerouter have.
The drawback is OpenWRT 21.02 it self since it can’t handle multi cpu on any DSA device.
So you will get one cpu with 21.02 no matter what device you choose.

But I think next release will have multi cpu support?

I'm still getting the crashes.

I recently upgraded another TP-Link Archer C7 v2 from 19.07 branch to 21.02. (Edit to add: in a complete different location in another country, so not the same WiFi / RF environment.) Previously the kernel log was clean: now I'm getting the crashes, so it appears as though something changed.

It looks like a manifestation of the bug worked around by the following upstream commit:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v4.14.81&id=ef5d27e1ed9c97c0670d0aa1f433f68e29de5bfb

When running in AP mode, ath10k sometimes suffers from TX credit
starvation. The issue is hard to reproduce and shows up once in a
few days, but has been repeatedly seen with QCA9882 and a large
range of firmwares, including 10.2.4.70.67.

Once the module is in this state, TX credits are never replenished,
which results in "SWBA overrun" errors, as no beacons can be sent.
Even worse, WMI commands run in a timeout while holding the conf
mutex for three seconds each, making any further operations slow
and the whole system unresponsive.

The firmware/driver never recovers from that state automatically,
and triggering TX flush or warm restarts won't work over WMI. So
issue a hardware restart if a WMI command times out due to missing
TX credits. This implies a connectivity outage of about 1.4s in AP
mode, but brings back the interface and the whole system to a usable
state. WMI command timeouts have not been seen in absent of this
specific issue, so taking such drastic actions seems legitimate.

I don't know if anything can be done in OpenWrt to stop this. Do we have to put in a workaround that prevents Tx credit starvation in the first place (if possible), with an 'emergency fallback' to a hardware reset?

# cat /etc/openwrt_release 
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='21.02.1'
DISTRIB_REVISION='r16325-88151b8303'
DISTRIB_TARGET='ath79/generic'
DISTRIB_ARCH='mips_24kc'
DISTRIB_DESCRIPTION='OpenWrt 21.02.1 r16325-88151b8303'
DISTRIB_TAINTS=''
# cat  /proc/cpuinfo 
system type		: Qualcomm Atheros QCA9558 ver 1 rev 0
machine			: TP-Link Archer C7 v2
processor		: 0
cpu model		: MIPS 74Kc V5.0
BogoMIPS		: 359.42
wait instruction	: yes
microsecond timers	: yes
tlb_entries		: 32
extra interrupt vector	: yes
hardware watchpoint	: yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb]
isa			: mips1 mips2 mips32r1 mips32r2
ASEs implemented	: mips16 dsp dsp2
Options implemented	: tlb 4kex 4k_cache prefetch mcheck ejtag llsc dc_aliases perf_cntr_intr_bit cdmm contextconfig perf
shadow register sets	: 1
kscratch registers	: 0
package			: 0
core			: 0
VCED exceptions		: not available
VCEI exceptions		: not available
# lsmod
ath                    19040  4 ath9k,ath9k_common,ath9k_hw,ath10k_core
ath10k_core           441088  1 ath10k_pci
ath10k_pci             36592  0 
ath9k                 103504  0 
ath9k_common           12064  1 ath9k
ath9k_hw              349536  2 ath9k,ath9k_common
cfg80211              275280  5 ath9k,ath9k_common,ath10k_core,ath,mac80211
compat                  1088  4 ath9k,ath9k_common,mac80211,cfg80211
crc_ccitt               1664  1 ppp_async
ehci_fsl                3424  0 
ehci_hcd               37008  2 ehci_platform,ehci_fsl
ehci_platform           4768  0 
fsl_mph_dr_of           3568  0 
gpio_button_hotplug     6976  0 
hwmon                   7360  1 ath10k_core
ip_tables              10816  4 iptable_nat,iptable_mangle,iptable_filter
ip6_tables             10752 24 ip6table_mangle,ip6table_filter
ip6t_REJECT             1216  2 
ip6table_filter          896  1 
ip6table_mangle         1376  1 
ipt_REJECT              1152  2 
iptable_filter           864  1 
iptable_mangle          1056  1 
iptable_nat             1056  1 
ledtrig_usbport         2736  0 
mac80211              511408  2 ath9k,ath10k_core
nf_conntrack           69040  8 xt_state,xt_nat,xt_conntrack,xt_REDIRECT,xt_MASQUERADE,xt_CT,nf_nat,nf_flow_table
nf_defrag_ipv4          1184  1 nf_conntrack
nf_defrag_ipv6          5360  1 nf_conntrack
nf_flow_table          15088  2 xt_FLOWOFFLOAD,nf_flow_table_hw
nf_flow_table_hw        2240  1 
nf_log_common           2880  2 nf_log_ipv4,nf_log_ipv6
nf_log_ipv4             3520  0 
nf_log_ipv6             4224  0 
nf_nat                 21616  4 iptable_nat,xt_nat,xt_REDIRECT,xt_MASQUERADE
nf_reject_ipv4          2240  1 ipt_REJECT
nf_reject_ipv6          2592  1 ip6t_REJECT
nls_base                5024  1 usbcore
ppp_async               7040  0 
ppp_generic            24016  3 pppoe,ppp_async,pppox
pppoe                   8608  0 
pppox                   1264  1 pppoe
slhc                    5152  1 ppp_generic
usb_common              2672  2 ehci_platform,usbcore
usbcore               140352  4 ledtrig_usbport,ehci_platform,ehci_fsl,ehci_hcd
x_tables               15248 24 xt_state,xt_nat,xt_conntrack,xt_REDIRECT,xt_MASQUERADE,xt_FLOWOFFLOAD,xt_CT,ipt_REJECT,xt_time,xt_tcpudp,xt_multiport,xt_mark,xt_mac,xt_limit,xt_comment,xt_TCPMSS,xt_LOG,iptable_mangle,iptable_filter,ip_tables,ip6table_mangle,ip6table_filter,ip6_tables,ip6t_REJECT
xt_CT                   2848  0 
xt_FLOWOFFLOAD          3200  0 
xt_LOG                   736  0 
xt_MASQUERADE            800  1 
xt_REDIRECT              672  0 
xt_TCPMSS               3136  4 
xt_comment               480135 
xt_conntrack            2400 14 
xt_limit                1312 20 
xt_mac                   608  0 
xt_mark                  640  0 
xt_multiport            1280  0 
xt_nat                  2400  0 
xt_state                 800  0 
xt_tcpudp               1760 18 
xt_time                 1888  2 


kernel log extract:

[82163.342000] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82163.444400] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82163.546800] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82163.649198] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82163.751607] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82163.854008] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82163.956406] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.058804] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.161216] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.263610] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.366013] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.468416] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.570821] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.673219] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.775623] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.878020] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82164.980416] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.082824] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.185220] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.287625] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.390034] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.492428] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.594821] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.697226] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.799631] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82165.902029] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.004433] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.106828] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.209240] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.311638] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.414044] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.516446] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.618848] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.626628] ath10k_pci 0000:00:00.0: Cannot communicate with firmware, previous wmi cmds: 36954:20466008 36904:20465958 36904:20465957 36904:20465957, jiffies: 20466817, attempting to fake crash and restart firmware, dev-flags: 0x42
[82166.648012] ath10k_pci 0000:00:00.0: failed to send pdev bss chan info request: -143
[82166.656155] ath10k_pci 0000:00:00.0: failed to send wmi nop: -143
[82166.662673] ath10k_pci 0000:00:00.0: could not request stats (type -268435456 ret -143 specifier 1)
[82166.685287] ath10k_pci 0000:00:00.0: failed to set beacon mode for vdev 0: -143
[82166.693283] ath10k_pci 0000:00:00.0: failed to set dtim period for vdev 0: -143
[82166.708734] ath10k_pci 0000:00:00.0: firmware crashed! (guid 10c6f6e8-f093-45b9-bfcb-b4ec84a855b8)
[82166.718436] ath10k_pci 0000:00:00.0: qca988x hw2.0 target 0x4100016c chip_id 0x043202ff sub 0000:0000
[82166.718449] ath10k_pci 0000:00:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[82166.722529] ath10k_pci 0000:00:00.0: firmware ver 10.1-ct-8x-__fW-022-ecad3248 api 2 features wmi-10.x,has-wmi-mgmt-tx,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,set-special-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT crc32 3e4cf97f
[82166.722583] ath10k_pci 0000:00:00.0: board_file api 1 bmi_id N/A crc32 bebc7c08
[82166.722597] ath10k_pci 0000:00:00.0: htt-ver 2.1 wmi-op 2 htt-op 2 cal file max-sta 128 raw 0 hwcrypto 1
[82166.722686] ath10k_pci 0000:00:00.0: firmware register dump:
[82166.728510] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.736580] ath10k_pci 0000:00:00.0: [00]: 0x00940750 0x00400C00 0x00980000 0x009AF0DC
[82166.745277] ath10k_pci 0000:00:00.0: [04]: 0x009AF148 0x009AF264 0x00941B6C 0x00941B20
[82166.753528] ath10k_pci 0000:00:00.0: [08]: 0x00941B00 0x009423A4 0x009422DC 0x009422C4
[82166.761778] ath10k_pci 0000:00:00.0: [12]: 0x00941B90 0x009423F4 0x009423D4 0x009423DC
[82166.770868] ath10k_pci 0000:00:00.0: [16]: 0x009423E4 0x009423EC 0x00942888 0x00942520
[82166.779328] ath10k_pci 0000:00:00.0: [20]: 0x009424FC 0x0094241C 0x00942540 0x00942498
[82166.787535] ath10k_pci 0000:00:00.0: [24]: 0x00942450 0x0094250C 0x009424D0 0x00942750
[82166.795616] ath10k_pci 0000:00:00.0: [28]: 0x00942798 0x00942844 0x009428C4 0x00942594
[82166.803900] ath10k_pci 0000:00:00.0: [32]: 0x0094258C 0x00942C28 0x00942F4C 0x00942F7C
[82166.812051] ath10k_pci 0000:00:00.0: [36]: 0x00942F90 0x00942FE4 0x00942FF8 0x00943040
[82166.821181] ath10k_pci 0000:00:00.0: [40]: 0x0094077C 0x00940790 0x00943084 0x00942F00
[82166.829327] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.837587] ath10k_pci 0000:00:00.0: [44]: 0x009AF290 0x00942A80 0x00942D40 0x00942D64
[82166.846555] ath10k_pci 0000:00:00.0: [48]: 0x00942D78 0x00942D9C 0x00957E18 0x00957E28
[82166.855002] ath10k_pci 0000:00:00.0: [52]: 0x00957E20 0x00940788 0x009430EC 0x00000000
[82166.863826] ath10k_pci 0000:00:00.0: [56]: 0x00000000 0x00000000 0x00000000 0x00000000
[82166.872346] ath10k_pci 0000:00:00.0: Copy Engine register dump:
[82166.879144] ath10k_pci 0000:00:00.0: [00]: 0x00057400   8   8   3   3
[82166.885729] ath10k_pci 0000:00:00.0: [01]: 0x00057800  16  16 178 179
[82166.892284] ath10k_pci 0000:00:00.0: [02]: 0x00057c00  30  30  93  94
[82166.898838] ath10k_pci 0000:00:00.0: [03]: 0x00058000   8   8   8   8
[82166.905386] ath10k_pci 0000:00:00.0: [04]: 0x00058400  25  25  54  14
[82166.911932] ath10k_pci 0000:00:00.0: [05]: 0x00058800  10  10 233 234
[82166.918480] ath10k_pci 0000:00:00.0: [06]: 0x00058c00  31  31  31  31
[82166.925026] ath10k_pci 0000:00:00.0: [07]: 0x00059000   0   0   0   0
[82166.931995] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82166.940110] ath10k_pci 0000:00:00.0: debug log header, dbuf: 0x411ac0  dropped: 0
[82166.948790] ath10k_pci 0000:00:00.0: [0] next: 0x411aa8 buf: 0x40fefc sz: 1500 len: 768 count: 48 free: 0
[82166.959579] ath10k_pci 0000:00:00.0: ath10k_pci ATH10K_DBG_BUFFER:
[82166.966612] ath10k: [0000]: 3DEB0305 204CFC0B 08000000 0B000000 52EB0305 204CFC0B 08000000 0B000000
[82166.976412] ath10k: [0008]: 66EB0305 204CFC0B 08000000 0B000000 7BEB0305 204CFC0B 08000000 0B000000
[82166.986448] ath10k: [0016]: 8FEB0305 204CFC0B 08000000 0B000000 A4EB0305 204CFC0B 08000000 0B000000
[82166.996451] ath10k: [0024]: B8EB0305 204CFC0B 08000000 0B000000 CDEB0305 204CFC0B 08000000 0B000000
[82167.006499] ath10k: [0032]: E1EB0305 204CFC0B 08000000 0B000000 F6EB0305 204CFC0B 08000000 0B000000
[82167.016111] ath10k: [0040]: 0AEC0305 204CFC0B 08000000 0B000000 1FEC0305 204CFC0B 08000000 0B000000
[82167.026167] ath10k: [0048]: 33EC0305 204CFC0B 08000000 0B000000 48EC0305 204CFC0B 08000000 0B000000
[82167.035830] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.043640] ath10k: [0056]: 5CEC0305 204CFC0B 08000000 0B000000 71EC0305 204CFC0B 08000000 0B000000
[82167.053839] ath10k: [0064]: 85EC0305 204CFC0B 08000000 0B000000 9AEC0305 204CFC0B 08000000 0B000000
[82167.063490] ath10k: [0072]: AEEC0305 204CFC0B 08000000 0B000000 C3EC0305 204CFC0B 08000000 0B000000
[82167.073372] ath10k: [0080]: D7EC0305 204CFC0B 08000000 0B000000 ECEC0305 204CFC0B 08000000 0B000000
[82167.083578] ath10k: [0088]: 00ED0305 204CFC0B 08000000 0B000000 15ED0305 204CFC0B 08000000 0B000000
[82167.093496] ath10k: [0096]: 29ED0305 204CFC0B 08000000 0B000000 3EED0305 204CFC0B 08000000 0B000000
[82167.102902] ath10k: [0104]: 52ED0305 204CFC0B 08000000 0B000000 67ED0305 204CFC0B 08000000 0B000000
[82167.112294] ath10k: [0112]: 7BED0305 204CFC0B 08000000 0B000000 90ED0305 204CFC0B 08000000 0B000000
[82167.121874] ath10k: [0120]: A4ED0305 204CFC0B 08000000 0B000000 B9ED0305 204CFC0B 08000000 0B000000
[82167.131159] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.138854] ath10k: [0128]: CDED0305 204CFC0B 08000000 0B000000 E2ED0305 204CFC0B 08000000 0B000000
[82167.148077] ath10k: [0136]: F6ED0305 204CFC0B 08000000 0B000000 0BEE0305 204CFC0B 08000000 0B000000
[82167.157367] ath10k: [0144]: 1FEE0305 204CFC0B 08000000 0B000000 34EE0305 204CFC0B 08000000 0B000000
[82167.168554] ath10k: [0152]: 48EE0305 204CFC0B 08000000 0B000000 5DEE0305 204CFC0B 08000000 0B000000
[82167.178141] ath10k: [0160]: 71EE0305 204CFC0B 08000000 0B000000 86EE0305 204CFC0B 08000000 0B000000
[82167.187827] ath10k: [0168]: 9AEE0305 204CFC0B 08000000 0B000000 AFEE0305 204CFC0B 08000000 0B000000
[82167.197159] ath10k: [0176]: C3EE0305 204CFC0B 08000000 0B000000 D8EE0305 204CFC0B 08000000 0B000000
[82167.206381] ath10k: [0184]: ECEE0305 204CFC0B 08000000 0B000000 01EF0305 204CFC0B 08000000 0B000000
[82167.215985] ath10k_pci 0000:00:00.0: ATH10K_END
[82167.220818] ath10k_pci 0000:00:00.0: [1] next: 0x411ac0 buf: 0x40f90c sz: 1500 len: 176 count: 11 free: 0
[82167.231113] ath10k_pci 0000:00:00.0: ath10k_pci ATH10K_DBG_BUFFER:
[82167.237455] ath10k: [0000]: 3EEF0305 204CFC0B 08000000 0B000000 53EF0305 204CFC0B 08000000 0B000000
[82167.246711] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.254755] ath10k: [0008]: 67EF0305 204CFC0B 08000000 0B000000 7CEF0305 204CFC0B 08000000 0B000000
[82167.264164] ath10k: [0016]: 90EF0305 204CFC0B 08000000 0B000000 A5EF0305 204CFC0B 08000000 0B000000
[82167.273428] ath10k: [0024]: B9EF0305 204CFC0B 08000000 0B000000 CEEF0305 204CFC0B 08000000 0B000000
[82167.283100] ath10k: [0032]: E2EF0305 204CFC0B 08000000 0B000000 F7EF0305 204CFC0B 08000000 0B000000
[82167.292343] ath10k: [0040]: 0BF00305 204CFC0B 08000000 0B000000
[82167.298847] ath10k_pci 0000:00:00.0: ATH10K_END
[82167.320341] ath10k_pci 0000:00:00.0: failed to set cts protection for vdev 0: -143
[82167.328225] ath10k_pci 0000:00:00.0: failed to recalculate rts/cts prot for vdev 0: -143
[82167.336527] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.345929] ath10k_pci 0000:00:00.0: failed to set preamble for vdev 0: -143
[82167.354209] ath10k_pci 0000:00:00.0: failed to set mgmt tx rate -143
[82167.406776] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[82167.438066] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.540465] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.642869] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.726987] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[82167.745265] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.847663] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.950077] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[82167.988718] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer 13ac5d6d vdev: 0 addr: 7a:6b:ae:9e:1d:bf 
[82167.999885] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer 97e310fb vdev: 0 addr: 30:b5:c2:d7:86:9d 
[82168.114651] ieee80211 phy0: Hardware restart was requested
[82169.079868] ath10k_pci 0000:00:00.0: 10.1 wmi init: vdevs: 16  peers: 127  tid: 256
[82169.097594] ath10k_pci 0000:00:00.0: wmi print 'P 128 V 8 T 410'
[82169.097807] ath10k_pci 0000:00:00.0: wmi print 'msdu-desc: 1424  sw-crypt: 0 ct-sta: 0'
[82169.098218] ath10k_pci 0000:00:00.0: wmi print 'alloc rem: 24984 iram: 38672'
[82169.159363] ath10k_pci 0000:00:00.0: pdev param 0 not supported by firmware
[82169.174444] ath10k_pci 0000:00:00.0: rts threshold -1
[82169.183043] ath10k_pci 0000:00:00.0: device successfully recovered

I run the CT-smallbuffers drivers, and haven't had any issues.

The non-CT drivers worked well for me on 19.07.8, not so much for 21.02.1

SSH in to the router and run cat /etc/config/wireless.

Redact the passwords in the "option key" sections.

Device (1)

# cat /etc/config/wireless

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11a'
	option country <Country-1>
	option path 'pci0000:00/0000:00:00.0'
	option cell_density '0'
	option htmode 'VHT20'
	option channel '48'

config wifi-iface 'default_radio0'
	option device 'radio0'
	option network 'lan'
	option mode 'ap'
	option ssid <redacted>
	option key <redacted>
	option encryption 'psk2'

config wifi-device 'radio1'
	option type 'mac80211'
	option hwmode '11g'
	option channel '3'
	option country <Country-1>
	option legacy_rates '1'
	option path 'platform/ahb/18100000.wmac'
	option htmode 'HT20'
	option cell_density '0'

config wifi-iface 'default_radio1'
	option device 'radio1'
	option network 'lan'
	option mode 'ap'
	option ssid <redacted>
	option encryption 'psk2'
	option key <redacted>

config wifi-iface 'wifinet0'
	option device 'radio0'
	option mode 'ap'
	option network 'lan'
	option encryption 'psk2'
	option key <redacted>
	option wpa_disable_eapol_key_retries '1'
	option ssid <redacted>

config wifi-iface 'wifinet1'
	option device 'radio1'
	option mode 'ap'
	option ssid <redacted>
	option network 'lan'
	option encryption 'psk2'
	option key <redacted>
	option wpa_disable_eapol_key_retries '1'

Device (2)

# cat /etc/config/wireless

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11a'
	option htmode 'VHT80'
	option txpower '17'
	option country <Country-2>
	option path 'pci0000:00/0000:00:00.0'
	option channel '48'

config wifi-iface 'wifinet0'
	option device 'radio0'
	option network 'lan'
	option mode 'ap'
	option encryption 'psk2'
	option disassoc_low_ack '0'
	option ssid <redacted>
	option key <redacted>

config wifi-device 'radio1'
	option type 'mac80211'
	option hwmode '11g'
	option htmode 'HT20'
	option country <Country-2>
	option txpower '20'
	option channel '8'
	option legacy_rates '1'
	option path 'platform/ahb/18100000.wmac'

config wifi-iface 'wifinet1'
	option device 'radio1'
	option network 'lan'
	option mode 'ap'
	option encryption 'psk2'
	option disassoc_low_ack '0'
	option ssid <redacted>
	option key <redacted>

I run the CT-smallbuffers drivers, and haven't had any issues.

The non-CT drivers worked well for me on 19.07.8, not so much for 21.02.1

I tried the non-default drivers, to no avail. I would prefer for the plain vanilla OpenWrt to work, because expecting non-technical users to swap drivers around is too much - I can do it, but OpenWrt should be usable by people not interested in the intricacies of their hardware and software. To use a phrase from elsewhere, it should "Just Work".

I'm happy to swap things around to diagnose stuff. Unfortunately, I don't have the time to get to the point of compiling development versions and doing bisection to find out what updates caused the new behaviour.

I listen to streaming radio using an old Roberts WiFi-enabled radio (Stream 202), so it is very, very obvious (and irritating) when a hardware reset occurs, as it takes a while for the radio to resync up. It's also a problem for colleagues on video calls over the WiFi connection, so it is not being a good advertisement for OpenWrt at the moment. (I use hard-wired Ethernet, which improves matters, but I also have an IPv6 issue with our ISP that it still ongoing - either OpenWrt is doing things wrong, or their Huawei kit is doing things wrong (or possibly both). Luckily IPv4 is rock solid.)

So a few things...

  1. Run iw list to get the maximum transmit power supported for each channel on each wireless band...for the country specified.

  2. Once you have that, manually set transmit power to those values in your wireless configs.

  3. Change encryption to WPA2-PSK AES.

  4. Set cell density to '1'.

  5. For 5 GHz band, use a higher numbered channel, such as 149 to 161.

  6. For 2.4 GHz band, use one of the non-overlapping channels 1, 6, or 11.

  7. I personally don't use option wpa_disable_eapol_key_retries.

For my specific location, I also manually configure beacon interval, distance, DTM, fragmentation threshold, RTS/CTS, and station activity limit (I have several static devices that I don't want kicked).

I would suggest using a WiFi analyzer to see which channels are being used the most in the surrounding neighborhood, and select the least used. You'll need to do this at different times of the day, including during "peak usage" hours.

You should also be looking at your signal strength, and signal to noise ratio as well on a WiFi analyzer.

I personally have had no issues (uses no patches), and now running 21.02.2 as of earlier this evening...which so far, so good.

And the CT drivers do work well...for some.

If you want to go from CT to CT-smallbuffers...

opkg remove kmod-ath10k-ct
opkg update && opkg install kmod-ath10k-ct-smallbuffers

That's it.

Ath10k has always been problematic.

When I make the next jump in my ISP's speed tier, I'll be getting a MediaTek device to replace the C7 V2.

This video is interesting, and enlightening...

Thank you for the feedback. I'll look into your suggestions, but it won't be quick as I'll be away from home-base for a while. The people on site while I'm away are not in a position to make the changes and review the outcomes.

And yes, the situation with wireless freedom is 'not ideal'. I can understand the push for locked-down certified firmware for regulatory compliance*, but that should not prevent it from being open source; and even proprietary firmware should have stable APIs as basic good practice.

*Regulatory compliance does not require secrecy. If anything, it is a strong argument for openness, as how can you demonstrate compliance without code and data that can be reviewed? Use of a publicly regulated resource should be open to public review.

I personally have had no issues (uses no patches), and now running 21.02.2 as of earlier this evening...which so far, so good.

Thank you for pointing out that 21.02.2 is available. I don't think the announcement is out yet, but following your note, I looked at the download site. Something for me to do when I get back.

I personally don't use option wpa_disable_eapol_key_retries.

I have some old, vulnerable kit that won't get updates. I really don't like 'stuff' that has only built-in WiFi network connectivity and for which no firmware updates are available.
Where practicable, I like to use 'client' access devices that have WiFi and bridge to a fixed Ethernet LAN, and use the Ethernet to connect to the device that needs network connectivity. Then, when the WiFi changes, I can change a cheap bridge and not the expensive doodad with 'convenient' WiFi. Not all expensive doodads offer wired Ethernet connectivity. This doesn't work for small, mobile devices.

No rush.

Make sure the configs are backed up before making any changes.

OK, now upgraded to 21.02.2, and for the first 24 hours everything looked fine.

Then the kernel log shows:

[  729.818266] kmodloader: done loading kernel modules from /etc/modules.d/*
[125579.988564] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125580.039771] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125580.090968] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125580.142175] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
...
repeats
...
[125582.906971] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125582.958163] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.002706] ath10k_pci 0000:00:00.0: Cannot communicate with firmware, previous wmi cmds: 36954:31320473 36904:31320430 36904:31320430 36954:31320430, jiffies: 31321216, attempting to fake crash and restart firmware, dev-flags: 0x42
[125583.023739] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.031438] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -11
[125583.039599] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[125583.047894] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[125583.056205] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[125583.064411] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.072116] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[125583.080448] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[125583.088823] ath10k_pci 0000:00:00.0: failed to transmit management frame via WMI: -143
[125583.111764] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.162958] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.214154] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.258748] ath10k_pci 0000:00:00.0: failed to send wmi nop: -143
[125583.265104] ath10k_pci 0000:00:00.0: could not request stats (type -268435456 ret -143 specifier 1)
[125583.274448] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.283320] ath10k_pci 0000:00:00.0: failed to set beacon mode for vdev 0: -143
[125583.291101] ath10k_pci 0000:00:00.0: failed to set dtim period for vdev 0: -143
[125583.309226] ath10k_pci 0000:00:00.0: failed to set cts protection for vdev 0: -143
[125583.317308] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.325170] ath10k_pci 0000:00:00.0: failed to recalculate rts/cts prot for vdev 0: -143
[125583.333605] ath10k_pci 0000:00:00.0: failed to set preamble for vdev 0: -143
[125583.341033] ath10k_pci 0000:00:00.0: failed to set mgmt tx rate -143
[125583.348507] ath10k_pci 0000:00:00.0: firmware crashed! (guid 2ec61b36-9951-4537-95a6-7f077e27cf9f)
[125583.357888] ath10k_pci 0000:00:00.0: qca988x hw2.0 target 0x4100016c chip_id 0x043202ff sub 0000:0000
[125583.367406] ath10k_pci 0000:00:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[125583.375860] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.387455] ath10k_pci 0000:00:00.0: firmware ver 10.1-ct-8x-__fW-022-ecad3248 api 2 features wmi-10.x,has-wmi-mgmt-tx,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,set-special-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT crc32 3e4cf97f
[125583.418623] ath10k_pci 0000:00:00.0: board_file api 1 bmi_id N/A crc32 bebc7c08
[125583.426193] ath10k_pci 0000:00:00.0: htt-ver 2.1 wmi-op 2 htt-op 2 cal file max-sta 128 raw 0 hwcrypto 1
[125583.435963] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.443873] ath10k_pci 0000:00:00.0: firmware register dump:
[125583.449836] ath10k_pci 0000:00:00.0: [00]: 0x00940750 0x00400C00 0x00980000 0x009AF0DC
[125583.458077] ath10k_pci 0000:00:00.0: [04]: 0x009AF148 0x009AF264 0x00941B6C 0x00941B20
[125583.466227] ath10k_pci 0000:00:00.0: [08]: 0x00941B00 0x009423A4 0x009422DC 0x009422C4
[125583.474414] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.482115] ath10k_pci 0000:00:00.0: [12]: 0x00941B90 0x009423F4 0x009423D4 0x009423DC
[125583.490260] ath10k_pci 0000:00:00.0: [16]: 0x009423E4 0x009423EC 0x00942888 0x00942520
[125583.498827] ath10k_pci 0000:00:00.0: [20]: 0x009424FC 0x0094241C 0x00942540 0x00942498
[125583.507035] ath10k_pci 0000:00:00.0: [24]: 0x00942450 0x0094250C 0x009424D0 0x00942750
[125583.515218] ath10k_pci 0000:00:00.0: [28]: 0x00942798 0x00942844 0x009428C4 0x00942594
[125583.523516] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.531186] ath10k_pci 0000:00:00.0: [32]: 0x0094258C 0x00942C28 0x00942F4C 0x00942F7C
[125583.539392] ath10k_pci 0000:00:00.0: [36]: 0x00942F90 0x00942FE4 0x00942FF8 0x00943040
[125583.547629] ath10k_pci 0000:00:00.0: [40]: 0x0094077C 0x00940790 0x00943084 0x00942F00
[125583.555933] ath10k_pci 0000:00:00.0: [44]: 0x009AF290 0x00942A80 0x00942D40 0x00942D64
[125583.564224] ath10k_pci 0000:00:00.0: [48]: 0x00942D78 0x00942D9C 0x00957E18 0x00957E28
[125583.572475] ath10k_pci 0000:00:00.0: [52]: 0x00957E20 0x00940788 0x009430EC 0x00000000
[125583.580674] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.588322] ath10k_pci 0000:00:00.0: [56]: 0x00000000 0x00000000 0x00000000 0x00000000
[125583.596544] ath10k_pci 0000:00:00.0: Copy Engine register dump:
[125583.602845] ath10k_pci 0000:00:00.0: [00]: 0x00057400   8   8   3   3
[125583.609514] ath10k_pci 0000:00:00.0: [01]: 0x00057800  24  24 378 379
[125583.616163] ath10k_pci 0000:00:00.0: [02]: 0x00057c00  30  30  29  30
[125583.622824] ath10k_pci 0000:00:00.0: [03]: 0x00058000  19  19  19  19
[125583.629467] ath10k_pci 0000:00:00.0: [04]: 0x00058400 6355 6355 147 107
[125583.636291] ath10k_pci 0000:00:00.0: [05]: 0x00058800   8   8 102 104
[125583.642930] ath10k_pci 0000:00:00.0: [06]: 0x00058c00  19  19  19  19
[125583.649562] ath10k_pci 0000:00:00.0: [07]: 0x00059000   0   0   0   0
[125583.656325] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.664236] ath10k_pci 0000:00:00.0: debug log header, dbuf: 0x411aa8  dropped: 0
[125583.672019] ath10k_pci 0000:00:00.0: [0] next: 0x411ac0 buf: 0x40f90c sz: 1500 len: 80 count: 5 free: 0
[125583.681699] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.689462] ath10k_pci 0000:00:00.0: ath10k_pci ATH10K_DBG_BUFFER:
[125583.695939] ath10k: [0000]: 8956AA07 204CFC0B 08000000 0B000000 9E56AA07 204CFC0B 08000000 0B000000
[125583.705439] ath10k: [0008]: B256AA07 204CFC0B 08000000 0B000000 C756AA07 204CFC0B 08000000 0B000000
[125583.714751] ath10k: [0016]: DB56AA07 204CFC0B 08000000 0B000000
[125583.721060] ath10k_pci 0000:00:00.0: ATH10K_END
[125583.725884] ath10k_pci 0000:00:00.0: [1] next: 0x411aa8 buf: 0x40fefc sz: 1500 len: 0 count: 0 free: 0
[125583.735476] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.762050] ath10k_pci 0000:00:00.0: failed to set beacon mode for vdev 1: -143
[125583.769733] ath10k_pci 0000:00:00.0: failed to set dtim period for vdev 1: -143
[125583.777379] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.788023] ath10k_pci 0000:00:00.0: failed to set cts protection for vdev 1: -143
[125583.795970] ath10k_pci 0000:00:00.0: failed to recalculate rts/cts prot for vdev 1: -143
[125583.804472] ath10k_pci 0000:00:00.0: failed to set preamble for vdev 1: -143
[125583.811924] ath10k_pci 0000:00:00.0: failed to set mgmt tx rate -143
[125583.825548] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer d664950b vdev: 0 addr: aa:7b:95:c7:f3:db 
[125583.836824] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer 1f377a10 vdev: 0 addr: e4:9a:79:f0:b4:e1 
[125583.848049] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer 824c4cc0 vdev: 0 addr: 18:cf:5e:d1:13:07 
[125583.859268] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer 21035b54 vdev: 1 addr: 32:b5:c2:d7:8d:fd 
[125583.870485] ath10k_pci 0000:00:00.0: removing peer, cleanup-all, deleting: peer ccbfa68d vdev: 0 addr: 30:b5:c2:d7:8d:fd 
[125583.881769] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125583.889341] ath10k_pci 0000:00:00.0: received scan foreign chan event in an invalid scan state: idle (0)
[125583.899063] ath10k_pci 0000:00:00.0: received chan info event without a scan request, ignoring
[125583.907917] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
[125583.915792] ath10k_pci 0000:00:00.0: received scan bss chan event in an invalid scan state: idle (0)
[125583.925184] ath10k_pci 0000:00:00.0: received chan info event without a scan request, ignoring
[125583.934304] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
[125584.083119] ieee80211 phy0: Hardware restart was requested
[125585.028377] ath10k_pci 0000:00:00.0: 10.1 wmi init: vdevs: 16  peers: 127  tid: 256
[125585.045829] ath10k_pci 0000:00:00.0: wmi print 'P 128 V 8 T 410'
[125585.052282] ath10k_pci 0000:00:00.0: wmi print 'msdu-desc: 1424  sw-crypt: 0 ct-sta: 0'
[125585.060544] ath10k_pci 0000:00:00.0: wmi print 'alloc rem: 24984 iram: 38672'
[125585.129601] ath10k_pci 0000:00:00.0: pdev param 0 not supported by firmware
[125585.137093] ath10k_pci 0000:00:00.0: set-coverage-class, phyclk: 88  value: 0
[125585.145307] ath10k_pci 0000:00:00.0: rts threshold -1
[125585.151356] ath10k_pci 0000:00:00.0: rts threshold -1
[125585.169863] ath10k_pci 0000:00:00.0: device successfully recovered

and it has happened once more since.

# opkg list-installed *ath10k*
ath10k-board-qca988x - 20211216-1
ath10k-firmware-qca988x-ct - 2020-11-08-1
kmod-ath10k-ct - 5.4.179+2021-09-22-e6a7d5b5-1

I'll enable some debugging in the driver, and see if I can generate a useful bug report.

Edit to add: Having upgraded another TP-Link Archer C7 v2 to stock 21.02.2 in another location, it too is seeing the same symptoms.

[ 1316.599900] ath10k_pci 0000:00:00.0: Cannot communicate with firmware, previous wmi cmds: 36904:253611 36954:253387 36904:253383 36904:253381, jiffies: 254144, attempting to fake crash and restart firmware, dev-flags: 0x42
[ 1316.620066] ath10k_pci 0000:00:00.0: failed to send wmi nop: -143
[ 1316.626402] ath10k_pci 0000:00:00.0: could not request stats (type -268435456 ret -143 specifier 1)
[ 1316.636380] ath10k_pci 0000:00:00.0: firmware crashed! (guid 434168d3-16c8-4713-b14c-ee14fd73c6b9)
...
[ 1317.835978] ieee80211 phy0: Hardware restart was requested
...
[ 1318.917918] ath10k_pci 0000:00:00.0: device successfully recovered

Edit to add (5 days later): Having failed to save the crash dump generated by the firmware:

# cat /sys/kernel/debug/ieee80211/[phy0]/ath10k/fw_crash_dump > /tmp/crash-ath10k-ct.bin

(procedure detailed here: https://www.candelatech.com/ath10k-bugs.php) , I am now waiting, with full debugging set for the next crash.

The good news is that I've had no crashes since I set up the debugging. It could be a Heisenbug. Or simply the conditions that used to trigger it are no longer happening.

You might try the ct-full-htt firmware -

opkg remove ath10k-firmware-qca988x-ct
opkg update && opkg install ath10k-firmware-qca988x-ct-full-htt

...or the ct-smallbuffers driver -

opkg remove kmod-ath10k-ct
opkg update && opkg install kmod-ath10k-ct-smallbuffers
1 Like