I got the same crash when setting up Tuya smart devices. After initial setup, it connects okay, but right from the box first powering on crashes wifi.
Tuya's devices even don't know my SSID yet.
Wed Jun 21 17:03:45 2023 daemon.notice hostapd: phy1-ap0: STA 1c:90:ff:56:99:af IEEE 802.11: did not acknowledge authentication response
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] qcom-q6v5-wcss-pil cd00000.q6v5_wcss: fatal error received:
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] QC Image Version: QC_IMAGE_VERSION_STRING=WLAN.HK.2.9.0.1-01385-QCAHKSWPL_SILICONZ-1
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] Image Variant : IMAGE_VARIANT_STRING=8074.wlanfw.eval_v2Q
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203]
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] wal_peer_control.c:2870 Assertion is_graceful_to_handle failedparam0 :zero, param1 :zero, param2 :zero.
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] Thread ID : 0x00000060 Thread name : WLAN RT1 Process ID : 0
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] Register:
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] SP : 0x4bfd5938
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] FP : 0x4bfd5940
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] PC : 0x4b1080c4
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] SSR : 0x00000008
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] BADVA : 0x00020000
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] LR : 0x4b107860
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203]
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] Stack Dump
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] from : 0x4bfd5938
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203] to : 0x4bfd61b0
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.460203]
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.510962] remoteproc remoteproc0: crash detected in cd00000.q6v5_wcss: type fatal error
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.533321] remoteproc remoteproc0: handling crash #1 in cd00000.q6v5_wcss
Wed Jun 21 17:03:45 2023 kern.err kernel: [613358.541230] remoteproc remoteproc0: recovering cd00000.q6v5_wcss
Wed Jun 21 17:03:45 2023 kern.info kernel: [613358.574085] remoteproc remoteproc0: stopped remote processor cd00000.q6v5_wcss
Wed Jun 21 17:03:45 2023 kern.warn kernel: [613358.671071] ath11k c000000.wifi: failed to find peer 1c:90:ff:56:99:af on vdev 2 after creation
Wed Jun 21 17:03:45 2023 kern.warn kernel: [613358.671126] ath11k c000000.wifi: failed to find peer vdev_id 2 addr 1c:90:ff:56:99:af in delete
Wed Jun 21 17:03:45 2023 kern.warn kernel: [613358.678939] ath11k c000000.wifi: failed peer 1c:90:ff:56:99:af delete vdev_id 2 fallback ret -22
Wed Jun 21 17:03:45 2023 kern.warn kernel: [613358.687737] ath11k c000000.wifi: Failed to add peer: 1c:90:ff:56:99:af for VDEV: 2
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.696487] ath11k c000000.wifi: Failed to add station: 1c:90:ff:56:99:af for VDEV: 2
Wed Jun 21 17:03:46 2023 daemon.notice hostapd: phy1-ap1: STA 1c:90:ff:56:99:af IEEE 802.11: Could not add STA to kernel driver
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.801035] ath11k c000000.wifi: failed to send WMI_PEER_DELETE cmd
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.801084] ath11k c000000.wifi: failed to delete peer vdev_id 1 addr 1c:90:ff:56:99:af ret -108
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.806472] ath11k c000000.wifi: Failed to delete peer: 1c:90:ff:56:99:af for VDEV: 1
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.815271] ath11k c000000.wifi: Found peer entry 8c:de:f9:df:33:fa n vdev 1 after it was supposedly removed
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.823120] ------------[ cut here ]------------
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.833025] WARNING: CPU: 3 PID: 3170 at sta_set_sinfo+0xbf0/0xc50 [mac80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.837718] Modules linked in: ath11k_ahb ath11k wireguard pppoe nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet mac80211 libchacha20poly1305 l2tp_ppp iptable_nat ipt_REJECT chacha_neon cfg80211 xt_u32 xt_time xt_tcpudp xt_tcpmss xt_string xt_statistic xt_state xt_recent xt_policy xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange xt_iface xt_hl xt_helper xt_geoip xt_esp xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_bpf xt_TPROXY xt_TEE xt_TCPMSS xt_REDIRECT xt_PROTO xt_NFQUEUE xt_NETMAP xt_MASQUERADE xt_LOG xt_IPMARK xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY ts_fsm ts_bm sch_cake qrtr_smd qrtr qmi_helpers pptp pppox ppp_synctty ppp_mppe ppp_async poly1305_neon nft_tproxy nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_compat nft_chain_nat
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.838000] nfnetlink_queue nf_tproxy_ipv6 nf_tproxy_ipv4 nf_tables nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_log_syslog nf_flow_table nf_dup_ipv6 nf_dup_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast ts_kmp nf_conntrack_amanda nf_conncount libcurve25519_generic libchacha iptable_raw iptable_mangle iptable_filter ipt_ah ipt_ECN ip6table_raw ip_tables crc_ccitt compat_xtables compat asn1_decoder sch_teql sch_sfq sch_multiq sch_gred sch_fq sch_dsmark sch_codel em_text em_nbyte em_meta em_cmp act_simple act_pedit act_csum libcrc32c act_connmark nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact cryptodev xt_set ip_set_list_set ip_set_hash_netportnet
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.909847] ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ipmac ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 nfsv4 nfsv3 pppoatm ppp_generic slhc nfs ip6_gre ip_gre gre ifb l2tp_ip6 l2tp_ip l2tp_eth sit l2tp_netlink l2tp_core udp_tunnel ip6_udp_tunnel ipcomp6 xfrm6_tunnel esp6 ah6 xfrm4_tunnel ipcomp esp4 ah4 ipip ip6_tunnel oid_registry tunnel6 tunnel4 ip_tunnel tun xfrm_user xfrm_ipcomp af_key xfrm_algo lockd sunrpc grace dns_resolver br2684 atm nls_utf8 crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha512_generic seqiv jitterentropy_rng drbg michael_mic md5 hmac echainiv des_generic cmac arc4 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom qca_nss_dp qca_ssdk gpio_button_hotplug aquantia hwmon
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613358.996077] crc32c_generic
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.105159] CPU: 3 PID: 3170 Comm: hostapd Not tainted 5.15.114 #0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.108114] Hardware name: Redmi AX6 (DT)
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.114101] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.118273] pc : sta_set_sinfo+0xbf0/0xc50 [mac80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.125473] lr : sta_set_sinfo+0xbec/0xc50 [mac80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.130421] sp : ffffffc00ec13840
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.135539] x29: ffffffc00ec13840 x28: ffffff8004ea5240 x27: ffffffc00ec13dd0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.138933] x26: ffffff8001c76880 x25: ffffffc008bbf680 x24: ffffffc00123d000
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.146136] x23: ffffffc00ec13a38 x22: ffffff800f9588c0 x21: ffffff8013d92aa8
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.153341] x20: ffffff8006848880 x19: ffffff8013d92000 x18: ffffffc008ac7ba0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.160547] x17: 312076656476206e x16: 2061663a33333a66 x15: 0000000000000531
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.167752] x14: 00000000000001bb x13: ffffffc00ec13378 x12: ffffffc008b1fba0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.174956] x11: 0000000000000004 x10: ffffffc008b1fba0 x9 : 0000000000000000
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.182160] x8 : ffffffc008ac7b50 x7 : ffffffc008ac7ba0 x6 : 0000000000008e20
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.189366] x5 : 000000004c8e78d9 x4 : 0000000000000000 x3 : ffffff8004ea5240
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.196571] x2 : 0000000000000000 x1 : ffffff8004ea5240 x0 : 00000000ffffff94
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.203777] Call trace:
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.210973] sta_set_sinfo+0xbf0/0xc50 [mac80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.213666] sta_info_destroy_addr_bss+0x50/0x7c [mac80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.218270] ieee80211_color_change_finish+0x1dbc/0x2064 [mac80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.224089] cfg80211_check_station_change+0x1408/0x4950 [cfg80211]
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.230424] genl_family_rcv_msg_doit+0xb8/0x11c
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.236759] genl_rcv_msg+0xd4/0x1cc
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.241355] netlink_rcv_skb+0x5c/0x130
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.245002] genl_rcv+0x38/0x50
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.248993] netlink_unicast+0x1f8/0x2f4
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.252206] netlink_sendmsg+0x1a0/0x3dc
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.256113] ____sys_sendmsg+0x288/0x2c0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.260107] ___sys_sendmsg+0x84/0xf0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.264098] __sys_sendmsg+0x48/0xb0
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.267745] __arm64_sys_sendmsg+0x24/0x30
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.271478] invoke_syscall.constprop.0+0x5c/0x104
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.275474] do_el0_svc+0x6c/0x15c
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.280330] el0_svc+0x18/0x54
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.283802] el0t_64_sync_handler+0xe8/0x114
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.286930] el0t_64_sync+0x184/0x188
Wed Jun 21 17:03:46 2023 kern.warn kernel: [613359.291444] ---[ end trace cfc89d7b3a656617 ]---
Wed Jun 21 17:03:50 2023 kern.warn kernel: [613363.025056] ath11k c000000.wifi: failed to send WMI_PDEV_BSS_CHAN_INFO_REQUEST cmd
Wed Jun 21 17:03:50 2023 daemon.info hostapd: phy1-ap0: STA 1c:90:ff:56:99:af IEEE 802.11: deauthenticated due to local deauth request
Wed Jun 21 17:03:52 2023 kern.err kernel: [613364.791007] qcom-q6v5-wcss-pil cd00000.q6v5_wcss: start timed out
Wed Jun 21 17:03:52 2023 kern.err kernel: [613364.791058] remoteproc remoteproc0: can't start rproc cd00000.q6v5_wcss: -110
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.030559] ath11k_warn: 60 callbacks suppressed
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.030582] ath11k c000000.wifi: failed to send WMI_PDEV_BSS_CHAN_INFO_REQUEST cmd
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.034306] ath11k c000000.wifi: failed to send pdev bss chan info request
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.042009] ath11k c000000.wifi: failed to send WMI_PDEV_SET_PARAM cmd
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.048681] ath11k c000000.wifi: Failed to set beacon mode for VDEV: 0
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.055311] ath11k c000000.wifi: failed to send WMI_BCN_TMPL_CMDID
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.061888] ath11k c000000.wifi: failed to submit beacon template command: -108
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.068126] ath11k c000000.wifi: failed to update bcn template: -108
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.075786] ath11k c000000.wifi: failed to send WMI_VDEV_SET_PARAM_CMDID
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.082116] ath11k c000000.wifi: failed to set BA BUFFER SIZE 256 for vdev: 0
Wed Jun 21 17:03:56 2023 kern.warn kernel: [613369.088874] ath11k c000000.wifi: failed to send WMI_VDEV_SET_PARAM_CMDID
Wed Jun 21 17:04:02 2023 kern.warn kernel: [613375.097930] ath11k_warn: 51 callbacks suppressed
The title is incorrect: this is not a driver crash, but a wifi firmware crash.
Not much to do about it, just hope that Qualcomm eventually fix this. To be fair, this mostly happens with Tuya devices, so they also must do something completely non-standard, although whatever they do it should not crash the firmware.
There is a newer version (WLAN.HK.2.9.0.1-01832), hopefully soon it will be merged.
I've seen the same with recent Tuya devices (2.4GHz BK7231) on their factory firmware in out-of-box init mode. They weren't even joining the AX3600's SSID at that point - just doing whatever scan/handshake they do as part of onboarding with the app.
I didn't debug, get logs or see if they would otherwise onboard and function OK. I reflash with LibreTiny / ESPHome as a matter of course. All is OK once completed.
And I got the same crash even on OpenBeken firmware on Tuya device. But unlike the original Tuya's firmware, OpenBeken not crashed the router on first boot, but crashed after I configured wifi settings.
But after restarting both router and device it works fine. To be precise 5 Tuya devices work flawlessly until I change wifi settings in the device.
FWIW touches wood the latest snapshots have a more recent firmware:
WLAN.HK.2.9.0.1-01837-QCAHKSWPL_SILICONZ-1
So far in the last >1d it has not crashed with the error y'all mentioned, like it usually does. Still haven't figured out who's the culpirit - I do have a Tuya device, but it's already setup and associated, I actually run it off the IOT secondary antenna on a different channel and not off the ath11k.
Wifi crashed today, after my computer attempted to disconnect ssid 1 and join ssid 2 shortly after. Both ssid were running on ath11k radio. They are "normal" wpa2-psk ssid. I had to POR reboot thus did not get any logs.
@robimarko I can reproduce the ath11k radio crash by creating two "common" SSIDs with WPA2-PSK, let's call them A and B.
When I use my Windows client (Intel AX210 NGBW PCIe Wireless Card) to connect to A, all is fine. When I stay on A also. When I select SSID "B" and connect to it, the radio driver crashes reproducibly.
It doesn't matter if I switch the Windows client from SSID A to B or vice versa.
Staying on one SSID since boot up does not trigger any problems and ath11k works fine "forever" on my AX3600.
I can support that claim.
I had no problems with my setup of two ax3600. I then created a second ssid for all my iot devices and from then on once and while the wifi crashed and only a reboot helps.
I removed the second ssid again and since then, no fatal crashes anymore.
Again, a crash on the recent firmware from recent snapshot of yesterday.
Is there a way to restart ath11k without reboot from CLI? I am currently this watchcat user script to reboot in case a ping to an always-on WiFi devices fails:
I'm on ax3600 using firmware SNAPSHOT - r23845-abc536f547 . This one works very stable for me. It contains fw_build_id WLAN.HK.2.9.0.1-01862-QCAHKSWPL_SILICONZ-1 .
