When using v22.03.0, v22.03.2 and v22.03.3, I'm experiencing constant crashes, the router can't keep up even for a day, but I'm not experiencing this issue at v21.02.3 and it can run up to 10 days without a problem.
I'm using a custom-made Qualcomm IPQ4019 based router that is not officially supported by openwrt, I add the device and build it on myself.
The root cause of crashes are unclear, it seems random.
I hooked up a serial console to the router and monitored what happened when it crashes, it shows the following different circumstances:
- The router just rebooted, without outputting anything.
- The router output some error messages then reboot.
- The router stalled, no output messages, not responding to anything, I have to disconnect the power cord to reboot it.
- The router spamming error messages non-stoppable, not responding to anything, I have to disconnect the power cord to reboot it.
Did anyone know why is it happening, is that because I did something wrong when I'm adding the device and building?
Here is some part of outputs you might find interesting:
[ 312.147052] ath10k_ahb a800000.wifi: failed to send pdev bss chan info request
[ 312.152525] ath10k_ahb a800000.wifi: failed to set beacon mode for vdev 0: -108
[ 312.153589] ath10k_ahb a800000.wifi: failed to set dtim period for vdev 0: -108
[ 312.170626] ath10k_ahb a800000.wifi: failed to recalculate rts/cts prot for vdev 0: -108
[ 312.171320] ath10k_ahb a800000.wifi: failed to set cts protection for vdev 0: -108
[ 312.178514] ath10k_ahb a800000.wifi: failed to set preamble for vdev 0: -108
[ 312.185551] ath10k_ahb a800000.wifi: failed to set mgmt tx rate -108
[ 139.906495] 8<--- cut here ---
[ 139.906553] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[ 139.908492] pgd = 6a1931ce
[ 139.916707] [00000000] *pgd=00000000
[ 139.919238] Internal error: Oops: 80000005 [#1] SMP ARM
[11563.952750] rcu: INFO: rcu_sched self-detected stall on CPU
[11563.953187] rcu: 0-....: (1 GPs behind) idle=de2/1/0x40000002 softirq=71356/71357 fqs=1046
[11563.957449] (t=2100 jiffies g=113829 q=61)
[16463.457958] Virtual device eth1.1101 asks to queue packet!
[16463.463293] Virtual device eth1.1101 asks to queue packet!
[16463.468839] Virtual device eth1.1101 asks to queue packet!
[16479.773993] Bad mode in undefined instruction handler detected
[16479.774053] Internal error: Oops - bad mode: 0 [#1] SMP ARM
[ 3202.829381] Unhandled prefetch abort: page domain fault (0x00b) at 0xb6f4ed88
[ 3202.829458] 8<--- cut here ---
[ 3202.835522] 8<--- cut here ---
[ 3202.835529] Unable to handle kernel NULL pointer dereference at virtual address 00000001
[ 3202.835539] pgd = d6f3765f
[ 3202.835545] [00000001] *pgd=00000000
[ 3202.835577] Internal error: Oops: 805 [#1] SMP ARM
[29632.394565] 8<--- cut here ---
[29632.394614] Unhandled fault: external abort on non-linefetch (0x1008) at 0xf118aa14
[29632.396536] pgd = d5ab6a86
[29632.404076] [f118aa14] *pgd=81d87811, *pte=0180a653, *ppte=0180a453
[29632.406866] Internal error: : 1008 [#1] SMP ARM
[18823.212934] Bad mode in prefetch abort handler detected
[18823.212941] Bad mode in prefetch abort handler detected
[18823.212966] Internal error: Oops - bad mode: 0 [#1] SMP ARM
[ 5707.149038] ------------[ cut here ]------------
[ 5707.149115] WARNING: CPU: 0 PID: 10 at lib/refcount.c:28 __netif_receive_skb_core.constprop.0+0x14c/0xbe8
[ 5707.152747] refcount_t: underflow; use-after-free.
[ 7297.659478] 8<--- cut here ---
[ 7297.659798] Unable to handle kernel paging request at virtual address 5c5c5c5c
[ 7297.661729] pgd = 5cd64742
[ 7297.668886] [5c5c5c5c] *pgd=00000000
[ 7297.669528] 8<--- cut here ---
[ 7297.671676] Internal error: Oops: 80000005 [#1] SMP ARM
[ 1394.782511] Bad mode in undefined instruction handler detected
[ 1398.070245] Bad mode in undefined instruction handler detected
[ 6240.123026] usercopy: Kernel memory (efault) attempt detected from SLUB object 'kmalloc-1k' (offset 3808805888, size 0)!
[ 3933.793082] unexpected IRQ trap at vector aaaaaaaa