Kernel oops with TP-Link Archer C2600 on 18.06.2

I've got a TP-Link Archer C2600 running mostly dumb AP duties with OpenWRT 18.06.2. It does also act as a wireguard server. The router gets several days of uptime, but has at least once thrown a kernel oops and rebooted. I'd like to help narrow down where the problem might be. Process kworker includes a lot of possible causes. For now, I'll try disabling software flow offloading.

Logs are pasted below.

kernel: [374990.091269] Unable to handle kernel NULL pointer dereference at virtual address 00000000
kernel: [374990.091313] pgd = c0204000
kernel: [374990.098426] [00000000] *pgd=00000000
kernel: [374990.101152] Internal error: Oops: 17 [#1] SMP ARM
kernel: [374990.104854] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT wireguard slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables crc_ccitt compat ledtrig_usbport ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables ip6_udp_tunnel udp_tunnel leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple ohci_platform ohci_hcd
kernel: [374990.158401]  phy_qcom_dwc3 ahci ehci_platform sd_mod ahci_platform libahci_platform libahci libata scsi_mod ehci_hcd gpio_button_hotplug
kernel: [374990.180651] CPU: 1 PID: 11379 Comm: kworker/1:0 Not tainted 4.14.95 #0
kernel: [374990.192961] Hardware name: Generic DT based system
kernel: [374990.199398] Workqueue: events 0xc06467e0
kernel: [374990.204243] task: dba55900 task.stack: dbe36000
kernel: [374990.208412] pc : [<c07abf34>]    lr : [<c066e590>]    psr: 80000013
kernel: [374990.213100] sp : dbe37df8  ip : 00000000  fp : dcc23368
kernel: [374990.219436] r10: 23c34600  r9 : dd695c00  r8 : 16e36000
kernel: [374990.224734] r7 : dd4d6e80  r6 : aeb1e56c  r5 : 00000006  r4 : 00000000
kernel: [374990.230028] r3 : de806024  r2 : 1fffa6f0  r1 : 00000000  r0 : 00000003
kernel: [374990.236368] Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
kernel: [374990.242965] Control: 10c5787d  Table: 5e35006a  DAC: 00000051
kernel: [374990.250253] Process kworker/1:0 (pid: 11379, stack limit = 0xdbe36210)
kernel: [374990.256072] Stack: (0xdbe37df8 to 0xdbe38000)

crypto / netfilter stack me thinks..... try selectively not running some of those xt mods and watching your pids.... ( 11379? )

whatever is relevant in /proc ...... ( irq / mem.... ) i think will show some sort of heaping a long time prior to the crash.....

Hi,

I'm also seeing what the OP reports - same hardware, same OpenWrt version, also basically stock + wireguard. Here's my kernel oops log:

Apr 18 04:00:15 gwen kernel: [1815352.479755] Unable to handle kernel NULL pointer dereference at virtual address 00000c7c
Apr 18 04:00:15 gwen kernel: [1815352.479799] pgd = c0204000
Apr 18 04:00:15 gwen kernel: [1815352.487832] [00000c7c] *pgd=00000000
Apr 18 04:00:15 gwen kernel: [1815352.490106] Internal error: Oops: 17 [#1] SMP ARM
Apr 18 04:00:15 gwen kernel: [1815352.493514] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac802
11 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_li
mit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWO
FFLOAD xt_DSCP xt_CT xt_CLASSIFY wireguard slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4
 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat sc
h_cake nf_conntrack act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw
Apr 18 04:00:15 gwen kernel: [1815352.547239]  sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_comm
on ip6table_mangle ip6table_filter ip6_tables x_tables ifb ip6_udp_tunnel udp_tunnel leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple ohc
i_platform ohci_hcd phy_qcom_dwc3 ahci ehci_platform sd_mod ahci_platform libahci_platform libahci libata scsi_mod ehci_hcd gpio_button_hotplug
Apr 18 04:00:15 gwen kernel: [1815352.583179] CPU: 0 PID: 24570 Comm: kworker/0:3 Not tainted 4.14.95 #0
Apr 18 04:00:15 gwen kernel: [1815352.605408] Hardware name: Generic DT based system
Apr 18 04:00:15 gwen kernel: [1815352.612364] Workqueue: events 0xc06467e0
Apr 18 04:00:15 gwen kernel: [1815352.617297] task: dabb0500 task.stack: da158000
Apr 18 04:00:15 gwen kernel: [1815352.621208] pc : [<c07abf34>]    lr : [<c066e590>]    psr: 80000013
Apr 18 04:00:15 gwen kernel: [1815352.625981] sp : da159d50  ip : 00000000  fp : dcc23228
Apr 18 04:00:15 gwen kernel: [1815352.632402] r10: 00000000  r9 : ffffffff  r8 : 00000002
Apr 18 04:00:15 gwen kernel: [1815352.637786] r7 : da159dd4  r6 : af7f848d  r5 : 00000006  r4 : 00000c7c
Apr 18 04:00:15 gwen kernel: [1815352.643170] r3 : de806024  r2 : 1fffa6f0  r1 : 00000000  r0 : 00000000
Apr 18 04:00:15 gwen kernel: [1815352.649944] Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Apr 18 04:00:15 gwen kernel: [1815352.656625] Control: 10c5787d  Table: 5eecc06a  DAC: 00000051
Apr 18 04:00:15 gwen kernel: [1815352.663655] Process kworker/0:3 (pid: 24570, stack limit = 0xda158210)
Apr 18 04:00:15 gwen kernel: [1815352.669560] Stack: (0xda159d50 to 0xda15a000)

Any reccomendations for how to debug / report this? [I'm afraid that wulfy23's suggestions are a bit too terse / advanced for me.]

Apologies.... was just throwing some ideas out.... i'll try to be a bit more helpful.

I would start removing things..... one by one starting with;

-sched modules / services.
-wireguard

separately.... aka, one at a time. you'll have to be patient and methodical.....

some commands that may help you;

lsmod
rmmod xyz
/etc/init.d/sqm disable; /etc/init.d/sqm stop

(once  - example command)
echo "vm.min_free_kbytes = 16384" >> /etc/sysctl.conf; sysctl -p

(info)
cat /proc/sys/vm/min_free_kbytes
cat /sys/class/thermal/thermal_zone0/temp
cat /proc/meminfo

( to try for a period to see if device changes  behavior)
( the oops had me thinking that something got starved of memory )
echo 1 > /proc/sys/vm/overcommit_memory
http://engineering.pivotal.io/post/virtual_memory_settings_in_linux_-_the_problem_with_overcommit/

think about everything you are running ( tell us ) in top of a "vanilla" install..... because your issue is fairly unique and likely related to a specific combination of modules / or a hardware fault ( memory | thermal )

are you running swap?

also posting a full bootlog may assist.

What do you mean by "sched modules / services?

This one is difficult to remove on a long term basis, since I find it quite useful.

Oh, I had actually forgotten that I had installed and enabled SQM. It doesn't seem to make much of a performance difference for me, so I've disabled it, and we'll see if that affects stability.

When would you suggest running these? By the time the unit crashes and reboots, the pre-crash state is no longer accessible. Do I need to install some sort of monitoring framework to log these constantly?

I read the linked post, but it doesn't really get into overcommit = 1. Can you explain your thinking further here?

I don't keep detailed notes of what I install / activate. I mentioned SQM and Wireguard. I also do dynamic dns updating. I have wireless running on the 2.4 and 5 Ghz bands, with 2 SSIDs on the former.

No.

Here:

root@gwen:~# dmesg
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.95 (buildbot@builds-03.infra.lede-project.org) (gcc version 7.3.0 (OpenWrt GCC 7.3.0 r7549-217219e)) #0 SMP Mon Jan 28 08:54:32 2019
[    0.000000] CPU: ARMv7 Processor [512f04d0] revision 0 (ARMv7), cr=10c5787d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[    0.000000] OF: fdt: Machine model: TP-Link Archer C2600
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] On node 0 totalpages: 122880
[    0.000000] free_area_init_node: node 0, pgdat c0a27dc0, node_mem_map ddc39000
[    0.000000]   Normal zone: 960 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 122880 pages, LIFO batch:31
[    0.000000] random: get_random_bytes called from 0xc09008f0 with crng_init=0
[    0.000000] percpu: Embedded 15 pages/cpu @ddc01000 s29388 r8192 d23860 u61440
[    0.000000] pcpu-alloc: s29388 r8192 d23860 u61440 alloc=15\*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 121920
[    0.000000] Kernel command line: 
[    0.000000] Bootloader command line (ignored): console=ttyHSL1,115200n8 root=mtd:rootfs rootfstype=squashfs
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Memory: 479424K/491520K available (4900K kernel code, 161K rwdata, 756K rodata, 1024K init, 234K bss, 12096K reserved, 0K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xde800000 - 0xff800000   ( 528 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xde000000   ( 480 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc07c9258   (5893 kB)
[    0.000000]       .init : 0xc0900000 - 0xc0a00000   (1024 kB)
[    0.000000]       .data : 0xc0a00000 - 0xc0a286c0   ( 162 kB)
[    0.000000]        .bss : 0xc0a2a000 - 0xc0a64a98   ( 235 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] clocksource: dg_timer: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 305801671480 ns
[    0.000006] sched_clock: 32 bits at 6MHz, resolution 160ns, wraps every 343597383600ns
[    0.000021] Switching to timer-based delay loop, resolution 160ns
[    0.000238] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.50 BogoMIPS (lpj=62500)
[    0.000265] pid_max: default: 32768 minimum: 301
[    0.000393] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000411] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000971] CPU: Testing write buffer coherency: ok
[    0.001730] Setting up static identity map for 0x42300000 - 0x42300060
[    0.001884] Hierarchical SRCU implementation.
[    0.002619] smp: Bringing up secondary CPUs ...
[    0.004429] smp: Brought up 1 node, 2 CPUs
[    0.004446] SMP: Total of 2 processors activated (25.00 BogoMIPS).
[    0.004457] CPU: All CPU(s) started in SVC mode.
[    0.014996] VFP support v0.3: implementor 51 architecture 64 part 4d variant 2 rev 0
[    0.015152] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.015178] futex hash table entries: 512 (order: 3, 32768 bytes)
[    0.015289] pinctrl core: initialized pinctrl subsystem
[    0.016272] NET: Registered protocol family 16
[    0.016522] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.017857] cpuidle: using governor ladder
[    0.017922] cpuidle: using governor menu
[    0.038739] msm_bus_fabric_init_driver
[    0.040101] usbcore: registered new interface driver usbfs
[    0.040193] usbcore: registered new interface driver hub
[    0.040284] usbcore: registered new device driver usb
[    0.040337] pps_core: LinuxPPS API ver. 1 registered
[    0.040352] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.040385] PTP clock support registered
[    0.042221] clocksource: Switched to clocksource dg_timer
[    0.045424] NET: Registered protocol family 2
[    0.046004] TCP established hash table entries: 4096 (order: 2, 16384 bytes)
[    0.046046] TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
[    0.046101] TCP: Hash tables configured (established 4096 bind 4096)
[    0.046192] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    0.046217] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    0.046410] NET: Registered protocol family 1
[    0.046463] PCI: CLS 0 bytes, default 64
[    0.047581] No memory allocated for crashlog
[    0.047869] workingset: timestamp_bits=30 max_order=17 bucket_order=0
[    0.052960] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.052975] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    0.062060] io scheduler noop registered
[    0.062079] io scheduler deadline registered (default)
[    0.063807] qcom-pcie 1b500000.pci: 1b500000.pci supply vdda not found, using dummy regulator
[    0.063918] qcom-pcie 1b500000.pci: 1b500000.pci supply vdda_phy not found, using dummy regulator
[    0.064013] qcom-pcie 1b500000.pci: 1b500000.pci supply vdda_refclk not found, using dummy regulator
[    0.064850] OF: PCI: host bridge /soc/pci@1b500000 ranges:
[    0.064894] OF: PCI:    IO 0x0fe00000..0x0fefffff -> 0x0fe00000
[    0.064920] OF: PCI:   MEM 0x08000000..0x0fdfffff -> 0x08000000
[    0.172615] qcom-pcie 1b500000.pci: link up
[    0.172794] qcom-pcie 1b500000.pci: PCI host bridge to bus 0000:00
[    0.172818] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.172837] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xfe00000-0xfefffff])
[    0.172852] pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fdfffff]
[    0.172868] pci_bus 0000:00: scanning bus
[    0.172914] pci 0000:00:00.0: [17cb:0101] type 01 class 0x060400
[    0.173000] pci 0000:00:00.0: calling 0xc030d6c0
[    0.173071] pci 0000:00:00.0: supports D1
[    0.173081] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    0.173095] pci 0000:00:00.0: PME# disabled
[    0.173344] pci_bus 0000:00: fixups for bus
[    0.173365] PCI: bus0: Fast back to back transfers disabled
[    0.173385] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[    0.173517] pci_bus 0000:01: scanning bus
[    0.173692] pci 0000:01:00.0: [168c:0040] type 00 class 0x028000
[    0.174041] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    0.174507] pci 0000:01:00.0: calling 0xc030d6c0
[    0.175149] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    0.175189] pci 0000:01:00.0: PME# disabled
[    0.175515] pci_bus 0000:01: fixups for bus
[    0.175585] PCI: bus1: Fast back to back transfers disabled
[    0.175601] pci_bus 0000:01: bus scan returning with max=01
[    0.175620] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[    0.175636] pci_bus 0000:00: bus scan returning with max=ff
[    0.175685] pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x081fffff]
[    0.175710] pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x081fffff 64bit]
[    0.175840] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    0.175863] pci 0000:00:00.0:   bridge window [mem 0x08000000-0x081fffff]
[    0.175893] pci 0000:00:00.0: calling 0xc0554090
[    0.176096] pcieport 0000:00:00.0: assign IRQ: got 66
[    0.176410] pcieport 0000:00:00.0: AER enabled with IRQ 34
[    0.176523] pci 0000:01:00.0: calling 0xc0554090
[    0.176938] qcom-pcie 1b700000.pci: 1b700000.pci supply vdda not found, using dummy regulator
[    0.177038] qcom-pcie 1b700000.pci: 1b700000.pci supply vdda_phy not found, using dummy regulator
[    0.177144] qcom-pcie 1b700000.pci: 1b700000.pci supply vdda_refclk not found, using dummy regulator
[    0.177959] OF: PCI: host bridge /soc/pci@1b700000 ranges:
[    0.177999] OF: PCI:    IO 0x31e00000..0x31efffff -> 0x31e00000
[    0.178023] OF: PCI:   MEM 0x2e000000..0x31dfffff -> 0x2e000000
[    0.287920] qcom-pcie 1b700000.pci: link up
[    0.288069] qcom-pcie 1b700000.pci: PCI host bridge to bus 0001:00
[    0.288090] pci_bus 0001:00: root bus resource [bus 00-ff]
[    0.288105] pci_bus 0001:00: root bus resource [mem 0x2e000000-0x31dfffff]
[    0.288118] pci_bus 0001:00: scanning bus
[    0.288155] pci 0001:00:00.0: [17cb:0101] type 01 class 0x060400
[    0.288225] pci 0001:00:00.0: calling 0xc030d6c0
[    0.288287] pci 0001:00:00.0: supports D1
[    0.288295] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[    0.288307] pci 0001:00:00.0: PME# disabled
[    0.288544] pci_bus 0001:00: fixups for bus
[    0.288564] PCI: bus0: Fast back to back transfers disabled
[    0.288582] pci 0001:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[    0.288697] pci_bus 0001:01: scanning bus
[    0.288882] pci 0001:01:00.0: [168c:0040] type 00 class 0x028000
[    0.289253] pci 0001:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    0.289751] pci 0001:01:00.0: calling 0xc030d6c0
[    0.290442] pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
[    0.290484] pci 0001:01:00.0: PME# disabled
[    0.290820] pci_bus 0001:01: fixups for bus
[    0.290897] PCI: bus1: Fast back to back transfers disabled
[    0.290912] pci_bus 0001:01: bus scan returning with max=01
[    0.290927] pci 0001:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[    0.290943] pci_bus 0001:00: bus scan returning with max=ff
[    0.290980] pci 0001:00:00.0: BAR 8: assigned [mem 0x2e000000-0x2e1fffff]
[    0.291002] pci 0001:01:00.0: BAR 0: assigned [mem 0x2e000000-0x2e1fffff 64bit]
[    0.291134] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    0.291154] pci 0001:00:00.0:   bridge window [mem 0x2e000000-0x2e1fffff]
[    0.291180] pci 0001:00:00.0: calling 0xc0554090
[    0.291356] pcieport 0001:00:00.0: assign IRQ: got 99
[    0.291655] pcieport 0001:00:00.0: AER enabled with IRQ 67
[    0.291762] pci 0001:01:00.0: calling 0xc0554090
[    0.294037] L2 @ QSB rate. Forcing new rate.
[    0.294253] L2 @ 384000 KHz
[    0.294423] CPU0 @ 800000 KHz
[    0.294436] CPU1 @ QSB rate. Forcing new rate.
[    0.294553] CPU1 @ 384000 KHz
[    0.298404] gsbi 16300000.gsbi: GSBI port protocol: 6 crci: 0
[    0.299792] gsbi 1a200000.gsbi: GSBI port protocol: 3 crci: 0
[    0.301242] Serial: 8250/16550 driver, 16 ports, IRQ sharing enabled
[    0.303915] msm_serial 16340000.serial: msm_serial: detected port #0
[    0.304023] msm_serial 16340000.serial: uartclk = 7372800
[    0.304107] 16340000.serial: ttyMSM0 at MMIO 0x16340000 (irq = 100, base_baud = 460800) is a MSM
[    0.304146] msm_serial: console setup on port #0
[    1.034735] console [ttyMSM0] enabled
[    1.039446] msm_serial: driver initialized
[    1.048075] loop: module loaded
[    1.049532] spi_qup 1a280000.spi: IN:block:16, fifo:64, OUT:block:16, fifo:64
[    1.051526] m25p80 spi0.0: mx25u25635f (32768 Kbytes)
[    1.057664] 25 fixed-partitions partitions found on MTD device spi0.0
[    1.062354] Creating 25 MTD partitions on "spi0.0":
[    1.068864] 0x000000000000-0x000000020000 : "SBL1"
[    1.074395] 0x000000020000-0x000000040000 : "MIBIB"
[    1.079077] 0x000000040000-0x000000060000 : "SBL2"
[    1.083905] 0x000000060000-0x000000090000 : "SBL3"
[    1.088692] 0x000000090000-0x0000000a0000 : "DDRCONFIG"
[    1.093569] 0x0000000a0000-0x0000000b0000 : "SSD"
[    1.098618] 0x0000000b0000-0x0000000e0000 : "TZ"
[    1.103532] 0x0000000e0000-0x000000100000 : "RPM"
[    1.108129] 0x000000100000-0x000000170000 : "fs-uboot"
[    1.112811] 0x000000170000-0x0000001b0000 : "uboot-env"
[    1.117776] 0x0000001b0000-0x0000001f0000 : "radio"
[    1.122978] 0x0000001f0000-0x0000005f0000 : "os-image"
[    1.127779] 0x0000005f0000-0x000001ef0000 : "rootfs"
[    1.133024] mtd: device 12 (rootfs) set to be root filesystem
[    1.137657] 1 squashfs-split partitions found on MTD device rootfs
[    1.143092] 0x0000008b0000-0x000001ef0000 : "rootfs_data"
[    1.149884] 0x000001ef0000-0x000001ef0200 : "default-mac"
[    1.155377] 0x000001ef0200-0x000001ef0400 : "pin"
[    1.160652] 0x000001ef0400-0x000001f00000 : "product-info"
[    1.165457] 0x000001f00000-0x000001f10000 : "partition-table"
[    1.170728] 0x000001f10000-0x000001f20000 : "soft-version"
[    1.176625] 0x000001f20000-0x000001f30000 : "support-list"
[    1.181948] 0x000001f30000-0x000001f40000 : "profile"
[    1.187460] 0x000001f40000-0x000001f50000 : "default-config"
[    1.192625] 0x000001f50000-0x000001f90000 : "user-config"
[    1.198264] 0x000001f90000-0x000001fd0000 : "qos-db"
[    1.203631] 0x000001fd0000-0x000001fe0000 : "usb-config"
[    1.208609] 0x000001fe0000-0x000002000000 : "log"
[    1.214848] libphy: GPIO Bitbanged MDIO: probed
[    1.238820] switch0: Atheros AR8337 rev. 2 switch registered on gpio-0
[    2.085104] libphy: Fixed MDIO Bus: probed
[    2.086819] ipq806x-gmac-dwmac 37200000.ethernet: PTP uses main clock
[    2.088427] stmmac - user ID: 0x10, Synopsys ID: 0x37
[    2.094693] ipq806x-gmac-dwmac 37200000.ethernet: Ring mode enabled
[    2.099644] ipq806x-gmac-dwmac 37200000.ethernet: DMA HW capability register supported
[    2.105815] ipq806x-gmac-dwmac 37200000.ethernet: Enhanced/Alternate descriptors
[    2.113770] ipq806x-gmac-dwmac 37200000.ethernet: Enabled extended descriptors
[    2.121260] ipq806x-gmac-dwmac 37200000.ethernet: RX Checksum Offload Engine supported
[    2.128360] ipq806x-gmac-dwmac 37200000.ethernet: COE Type 2
[    2.136265] ipq806x-gmac-dwmac 37200000.ethernet: TX Checksum insertion supported
[    2.142006] ipq806x-gmac-dwmac 37200000.ethernet: Wake-Up On Lan supported
[    2.149451] ipq806x-gmac-dwmac 37200000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[    2.157933] ipq806x-gmac-dwmac 37400000.ethernet: PTP uses main clock
[    2.165014] stmmac - user ID: 0x10, Synopsys ID: 0x37
[    2.171085] ipq806x-gmac-dwmac 37400000.ethernet: Ring mode enabled
[    2.176192] ipq806x-gmac-dwmac 37400000.ethernet: DMA HW capability register supported
[    2.182197] ipq806x-gmac-dwmac 37400000.ethernet: Enhanced/Alternate descriptors
[    2.190246] ipq806x-gmac-dwmac 37400000.ethernet: Enabled extended descriptors
[    2.197806] ipq806x-gmac-dwmac 37400000.ethernet: RX Checksum Offload Engine supported
[    2.204836] ipq806x-gmac-dwmac 37400000.ethernet: COE Type 2
[    2.212739] ipq806x-gmac-dwmac 37400000.ethernet: TX Checksum insertion supported
[    2.218484] ipq806x-gmac-dwmac 37400000.ethernet: Wake-Up On Lan supported
[    2.225923] ipq806x-gmac-dwmac 37400000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[    2.233334] i2c /dev entries driver
[    2.242051] Calibration not found.
[    2.244514] qcom-tsens 900000.thermal-sensor: tsens calibration failed
[    2.249013] Speed bin: 0
[    2.254488] PVS bin: 1
[    2.258438] cpuidle: enable-method property 'qcom,kpss-acc-v1' found operations
[    2.259345] cpuidle: enable-method property 'qcom,kpss-acc-v1' found operations
[    2.267075] sdhci: Secure Digital Host Controller Interface driver
[    2.273841] sdhci: Copyright(c) Pierre Ossman
[    2.280020] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.286031] NET: Registered protocol family 10
[    2.291441] Segment Routing with IPv6
[    2.294623] NET: Registered protocol family 17
[    2.298222] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    2.303250] 8021q: 802.1Q VLAN Support v1.8
[    2.315727] Registering SWP/SWPB emulation handler
[    2.332163] qcom_rpm 108000.rpm: RPM firmware 3.0.16777342
[    2.343876] s1a: supplied by regulator-dummy
[    2.343965] s1a: Bringing 0uV into 1050000-1050000uV
[    2.347479] s1b: supplied by regulator-dummy
[    2.352217] s1b: Bringing 0uV into 1050000-1050000uV
[    2.356747] s2a: supplied by regulator-dummy
[    2.361425] s2a: Bringing 0uV into 800000-800000uV
[    2.365943] s2b: supplied by regulator-dummy
[    2.370265] s2b: Bringing 0uV into 800000-800000uV
[    2.378956] hctosys: unable to open rtc device (rtc0)
[    2.384999] VFS: Mounted root (squashfs filesystem) readonly on device 31:12.
[    2.385823] Freeing unused kernel memory: 1024K
[    2.415189] random: fast init done
[    3.204998] random: crng init done
[    3.463918] init: Console is alive
[    3.464054] init: - watchdog -
[    4.721480] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    4.883940] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    4.886477] SCSI subsystem initialized
[    4.892306] libata version 3.00 loaded.
[    4.896210] ehci-platform: EHCI generic platform driver
[    4.899804] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    4.900756] ohci-platform: OHCI generic platform driver
[    5.040638] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[    5.040728] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 1
[    5.045415] xhci-hcd xhci-hcd.0.auto: hcc params 0x0228f065 hci version 0x100 quirks 0x0000000002010010
[    5.052767] xhci-hcd xhci-hcd.0.auto: irq 106, io mem 0x11000000
[    5.062723] hub 1-0:1.0: USB hub found
[    5.068225] hub 1-0:1.0: 1 port detected
[    5.072044] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[    5.075926] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 2
[    5.081137] xhci-hcd xhci-hcd.0.auto: Host supports USB 3.0  SuperSpeed
[    5.088853] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    5.095693] hub 2-0:1.0: USB hub found
[    5.103707] hub 2-0:1.0: 1 port detected
[    5.107418] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    5.111264] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 3
[    5.116771] xhci-hcd xhci-hcd.1.auto: hcc params 0x0228f065 hci version 0x100 quirks 0x0000000002010010
[    5.124206] xhci-hcd xhci-hcd.1.auto: irq 107, io mem 0x10000000
[    5.133974] hub 3-0:1.0: USB hub found
[    5.139744] hub 3-0:1.0: 1 port detected
[    5.143578] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    5.147368] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 4
[    5.152735] xhci-hcd xhci-hcd.1.auto: Host supports USB 3.0  SuperSpeed
[    5.160283] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
[    5.167252] hub 4-0:1.0: USB hub found
[    5.175254] hub 4-0:1.0: 1 port detected
[    5.180272] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    5.189296] init: - preinit -
[    7.346886] Generic PHY fixed-0:01: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:01, irq=POLL)
[    7.348004] dwmac1000: Master AXI performs any burst length
[    7.355855] ipq806x-gmac-dwmac 37400000.ethernet eth1: IEEE 1588-2008 Advanced Timestamp supported
[    7.361289] ipq806x-gmac-dwmac 37400000.ethernet eth1: registered PTP clock
[    7.370959] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[    8.395037] ipq806x-gmac-dwmac 37400000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
[    8.395124] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[   11.170436] jffs2: notice: (141) jffs2_build_xattr_subsystem: complete building xattr subsystem, 8 of xdatum (3 unchecked, 5 orphan) and 69 of xref (5 dead, 0 orphan) found.
[   11.189259] mount_root: switching to jffs2 overlay
[   11.210495] overlayfs: upper fs does not support tmpfile.
[   11.245990] urandom-seed: Seeding with /etc/urandom.seed
[   11.750796] procd: - early -
[   11.750947] procd: - watchdog -
[   12.431336] procd: - watchdog -
[   12.431843] procd: - ubus -
[   12.493036] procd: - init -
[   14.351140] kmodloader: loading kernel modules from /etc/modules.d/*
[   14.358545] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   14.380502] u32 classifier
[   14.380521]     input device check on
[   14.382094]     Actions configured
[   14.389026] Mirror/redirect action on
[   14.394044] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
[   14.424009] Loading modules backported from Linux version wt-2017-11-01-0-gfe248fc2c180
[   14.424042] Backport generated by backports.git v4.14-rc2-1-31-g86cf0e5d
[   14.433099] ip_tables: (C) 2000-2006 Netfilter Core Team
[   14.573784] wireguard: WireGuard 0.0.20190123 loaded. See www.wireguard.com for information.
[   14.573822] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
[   14.607388] xt_time: kernel timezone is -0000
[   14.635871] PPP generic driver version 2.4.2
[   14.636704] NET: Registered protocol family 24
[   14.649312] ath10k_pci 0000:01:00.0: assign IRQ: got 66
[   14.649904] ath10k_pci 0000:01:00.0: enabling device (0140 -> 0142)
[   14.649998] ath10k_pci 0000:01:00.0: enabling bus mastering
[   14.650511] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[   14.830953] ath10k_pci 0000:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/firmware-6.bin failed with error -2
[   14.830989] ath10k_pci 0000:01:00.0: Falling back to user helper
[   14.870828] firmware ath10k!QCA99X0!hw2.0!firmware-6.bin: firmware_loading_store: map pages failed
[   15.506050] ath10k_pci 0000:01:00.0: qca99x0 hw2.0 target 0x01000000 chip_id 0x003b01ff sub 168c:0002
[   15.506084] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 1
[   15.515922] ath10k_pci 0000:01:00.0: firmware ver 10.4.1.00030-1 api 5 features no-p2p crc32 d2901e01
[   15.578854] ath10k_pci 0000:01:00.0: board_file api 2 bmi_id 1:1 crc32 08fa09f2
[   16.843852] ath10k_pci 0000:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 512 raw 0 hwcrypto 1
[   16.926756] ath: EEPROM regdomain: 0x0
[   16.926768] ath: EEPROM indicates default country code should be used
[   16.926778] ath: doing EEPROM country->regdmn map search
[   16.926794] ath: country maps to regdmn code: 0x3a
[   16.926807] ath: Country alpha2 being used: US
[   16.926816] ath: Regpair used: 0x3a
[   16.931100] ath10k_pci 0001:01:00.0: assign IRQ: got 99
[   16.931658] ath10k_pci 0001:01:00.0: enabling device (0140 -> 0142)
[   16.931752] ath10k_pci 0001:01:00.0: enabling bus mastering
[   16.932284] ath10k_pci 0001:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[   23.343526] ath10k_pci 0001:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/firmware-6.bin failed with error -2
[   23.343571] ath10k_pci 0001:01:00.0: Falling back to user helper
[   23.384443] firmware ath10k!QCA99X0!hw2.0!firmware-6.bin: firmware_loading_store: map pages failed
[   23.384614] ath10k_pci 0001:01:00.0: qca99x0 hw2.0 target 0x01000000 chip_id 0x003b01ff sub 168c:0002
[   23.392318] ath10k_pci 0001:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 1
[   23.403757] ath10k_pci 0001:01:00.0: firmware ver 10.4.1.00030-1 api 5 features no-p2p crc32 d2901e01
[   23.469310] ath10k_pci 0001:01:00.0: board_file api 2 bmi_id 1:2 crc32 08fa09f2
[   24.767308] ath10k_pci 0001:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 512 raw 0 hwcrypto 1
[   24.856000] ath: EEPROM regdomain: 0x0
[   24.856009] ath: EEPROM indicates default country code should be used
[   24.856015] ath: doing EEPROM country->regdmn map search
[   24.856025] ath: country maps to regdmn code: 0x3a
[   24.856034] ath: Country alpha2 being used: US
[   24.856040] ath: Regpair used: 0x3a
[   24.877977] kmodloader: done loading kernel modules from /etc/modules.d/*
[   29.639459] Generic PHY fixed-0:01: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:01, irq=POLL)
[   29.640483] dwmac1000: Master AXI performs any burst length
[   29.648489] ipq806x-gmac-dwmac 37400000.ethernet eth1: IEEE 1588-2008 Advanced Timestamp supported
[   29.653967] ipq806x-gmac-dwmac 37400000.ethernet eth1: registered PTP clock
[   29.668336] br-lan: port 1(eth1) entered blocking state
[   29.669639] br-lan: port 1(eth1) entered disabled state
[   29.675214] device eth1 entered promiscuous mode
[   29.687324] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   29.706680] Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=POLL)
[   29.708014] dwmac1000: Master AXI performs any burst length
[   29.715820] ipq806x-gmac-dwmac 37200000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[   29.721104] ipq806x-gmac-dwmac 37200000.ethernet eth0: registered PTP clock
[   29.730686] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   30.712795] ipq806x-gmac-dwmac 37400000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
[   30.724151] br-lan: port 1(eth1) entered blocking state
[   30.724184] br-lan: port 1(eth1) entered forwarding state
[   30.732704] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   30.792734] ipq806x-gmac-dwmac 37200000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[   30.792808] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   33.788006] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   35.380749] IPv6: ADDRCONF(NETDEV_UP): wlan1: link is not ready
[   35.382281] br-lan: port 2(wlan0) entered blocking state
[   35.385599] br-lan: port 2(wlan0) entered disabled state
[   35.391396] device wlan0 entered promiscuous mode
[   35.396585] br-lan: port 2(wlan0) entered blocking state
[   35.400949] br-lan: port 2(wlan0) entered forwarding state
[   35.407572] br-lan: port 2(wlan0) entered disabled state
[   35.421537] br-lan: port 3(wlan1) entered blocking state
[   35.421573] br-lan: port 3(wlan1) entered disabled state
[   35.426393] device wlan1 entered promiscuous mode
[   35.922036] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
[   35.922446] br-lan: port 3(wlan1) entered blocking state
[   35.927502] br-lan: port 3(wlan1) entered forwarding state
[   35.962137] IPv6: ADDRCONF(NETDEV_UP): wlan1-1: link is not ready
[   36.415148] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1-1: link becomes ready
[   36.415445] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[   36.420708] br-lan: port 2(wlan0) entered blocking state
[   36.427172] br-lan: port 2(wlan0) entered forwarding state
[   37.438653] br-lan: port 4(wlan1.sta2) entered blocking state
[   37.438688] br-lan: port 4(wlan1.sta2) entered disabled state
[   37.443783] device wlan1.sta2 entered promiscuous mode
[   37.455647] br-lan: port 4(wlan1.sta2) entered blocking state
[   37.455686] br-lan: port 4(wlan1.sta2) entered forwarding state

Thanks for the help!

Okay, I've been running without SQM for a while. I haven't had an oops for a while, but I just hit one a couple of days ago. This one isn't a "kernel NULL pointer dereference", however, but a "kernel paging request":

Jun  4 22:59:03 gwen kernel: [3050353.107353] Unable to handle kernel paging request at virtual address 000094c4
Jun  4 22:59:03 gwen kernel: [3050353.107391] pgd = c0204000
Jun  4 22:59:03 gwen kernel: [3050353.113813] [000094c4] *pgd=00000000
Jun  4 22:59:03 gwen kernel: [3050353.116784] Internal error: Oops: 80000007 [#1] SMP ARM
Jun  4 22:59:03 gwen kernel: [3050353.120240] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac802
11 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_li
mit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWO
FFLOAD xt_DSCP xt_CT xt_CLASSIFY wireguard slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4
 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat sc
h_cake nf_conntrack act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw
Jun  4 22:59:03 gwen kernel: [3050353.174480]  sch_tbf sch_htb sch_hfsc sch_ingress ledtrig_usbport ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_comm
on ip6table_mangle ip6table_filter ip6_tables x_tables ifb ip6_udp_tunnel udp_tunnel leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple ohc
i_platform ohci_hcd phy_qcom_dwc3 ahci ehci_platform sd_mod ahci_platform libahci_platform libahci libata scsi_mod ehci_hcd gpio_button_hotplug
Jun  4 22:59:03 gwen kernel: [3050353.210425] CPU: 1 PID: 2349 Comm: kworker/1:1 Not tainted 4.14.95 #0
Jun  4 22:59:03 gwen kernel: [3050353.232653] Hardware name: Generic DT based system
Jun  4 22:59:03 gwen kernel: [3050353.239261] Workqueue: events 0xc06467e0
Jun  4 22:59:03 gwen kernel: [3050353.244457] task: dbf10000 task.stack: dc578000
Jun  4 22:59:03 gwen kernel: [3050353.248364] pc : [<000094c4>]    lr : [<c0a5ddec>]    psr: 60000013
Jun  4 22:59:03 gwen kernel: [3050353.253140] sp : dc579e08  ip : 00000000  fp : dcc23368
Jun  4 22:59:03 gwen kernel: [3050353.259561] r10: 23c34600  r9 : dd695c00  r8 : 16e36000
Jun  4 22:59:03 gwen kernel: [3050353.264945] r7 : dd4d6e80  r6 : 20000013  r5 : 00000001  r4 : dd696828
Jun  4 22:59:03 gwen kernel: [3050353.270328] r3 : c07abf64  r2 : e0832290  r1 : 20000013  r0 : 80f57ec8
Jun  4 22:59:03 gwen kernel: [3050353.277102] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Jun  4 22:59:03 gwen kernel: [3050353.283784] Control: 10c5787d  Table: 5eed806a  DAC: 00000051
Jun  4 22:59:03 gwen kernel: [3050353.290813] Process kworker/1:1 (pid: 2349, stack limit = 0xdc578210)
Jun  4 22:59:03 gwen kernel: [3050353.296718] Stack: (0xdc579e08 to 0xdc57a000)
Jun  4 22:59:03 gwen kernel: [3050353.303327] 9e00:                   dd695c80 00000000 23c34600 c0559f4c dd695c80 23c34600
Jun  4 22:59:03 gwen kernel: [3050353.307851] 9e20: 00000000 dd695c80 dcc232c0 16e36000 23c34600 c055a40c dcc23080 23c34600
Jun  4 22:59:03 gwen kernel: [3050353.316185] 9e40: dcc23340 ddc10010 dcc232c0 c055a468 dcc05200 00000000 dcc23340 c05a1490
Jun  4 22:59:03 gwen kernel: [3050353.324516] 9e60: c0a60c8c 00000000 dcc8d300 dcc232e8 23c34600 16e36000 c093eca0 23c34600
Jun  4 22:59:03 gwen kernel: [3050353.332849] 9e80: dcc8d400 dcc8d41c 00000000 dcc8d440 000927c0 dcca9400 ddc16540 c064771c
Jun  4 22:59:03 gwen kernel: [3050353.341183] 9ea0: 00000000 c0646a64 00009c40 dcca9400 00000000 c0a60c8c 00000000 00000001
Jun  4 22:59:03 gwen kernel: [3050353.349517] 9ec0: 000927c0 00000000 ddc16540 c064216c 00000002 0005dc00 000927c0 00000021
Jun  4 22:59:03 gwen kernel: [3050353.357850] 9ee0: dcc8aa00 dcca9400 dcc8aa00 dcc8a880 dcc8a880 dcc87fc0 00000008 c0646398
Jun  4 22:59:03 gwen kernel: [3050353.366183] 9f00: dcc8aa38 00000000 dcc8aa04 dcca9400 c0a1f758 00000000 00000008 c0646818
Jun  4 22:59:03 gwen kernel: [3050353.374517] 9f20: dcc8aa38 dadaae00 ddc16540 ddc19400 00000000 c032fb34 ddc16558 ffffe000
Jun  4 22:59:03 gwen kernel: [3050353.382850] 9f40: dadaae00 ddc16540 dadaae18 ddc16558 ffffe000 c0a02d00 00000008 c0330be0
Jun  4 22:59:03 gwen kernel: [3050353.391183] 9f60: dc2e7ebc d9524840 dc578000 d9524c00 dc2e7ebc d952485c dadaae00 c033089c
Jun  4 22:59:03 gwen kernel: [3050353.399517] 9f80: 00000000 c03357a4 00000000 d9524c00 c0335664 00000000 00000000 00000000
Jun  4 22:59:03 gwen kernel: [3050353.407849] 9fa0: 00000000 00000000 00000000 c0307e68 00000000 00000000 00000000 00000000
Jun  4 22:59:03 gwen kernel: [3050353.416182] 9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Jun  4 22:59:03 gwen kernel: [3050353.424516] 9fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
Jun  4 22:59:03 gwen kernel: [3050353.432837] Code: bad PC value
Jun  4 22:59:03 gwen kernel: [3050353.441339] ---[ end trace 7b75b777b8e38d50 ]---