Belkin RT3200/Linksys E8450 WiFi AX discussion

I just ran into the second crash that I have experienced in a few weeks with this device, which is disappointing considering the rock solid stability of my previous mt7621 platform. The first I was not aware logs after a reboot could even be saved, so I never got to fetch those. However, I learned about the ability of the UBI installation to save crash logs which is definitely a game changer for debugging! Hopefully these logs are useful.

I found two different logs in the /sys/fs/pstore/ folder. The first one was named dmesg-ramoops-0:

root@OpenWrt:~# cat /sys/fs/pstore/dmesg-ramoops-0
Oops#1 Part1
<6>[1187085.545421] br-untrusted: port 1(lan4) entered blocking state
<6>[1187085.551346] br-untrusted: port 1(lan4) entered forwarding state
<6>[1194212.514187] br-lan: port 2(lan2) entered disabled state
<6>[1194212.520184] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1194214.596282] mt7530 mdio-bus:00 lan2: Link is Up - 100Mbps/Full - flow control off
<6>[1194214.604009] br-lan: port 2(lan2) entered blocking state
<6>[1194214.609414] br-lan: port 2(lan2) entered forwarding state
<6>[1194234.353897] br-lan: port 2(lan2) entered disabled state
<6>[1194234.359821] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1194237.486388] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1194237.494103] br-lan: port 2(lan2) entered blocking state
<6>[1194237.499500] br-lan: port 2(lan2) entered forwarding state
<6>[1194255.153731] br-lan: port 2(lan2) entered disabled state
<6>[1194255.159723] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1194257.236287] mt7530 mdio-bus:00 lan2: Link is Up - 100Mbps/Full - flow control off
<6>[1194257.243995] br-lan: port 2(lan2) entered blocking state
<6>[1194257.249392] br-lan: port 2(lan2) entered forwarding state
<6>[1194261.393694] br-lan: port 2(lan2) entered disabled state
<6>[1194261.399611] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1201429.974414] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1201429.979530] br-lan: port 1(lan1) entered disabled state
<6>[1236558.925969] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control off
<6>[1236558.933513] br-lan: port 2(lan2) entered blocking state
<6>[1236558.938910] br-lan: port 2(lan2) entered forwarding state
<6>[1236572.443713] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1236572.448974] br-lan: port 2(lan2) entered disabled state
<6>[1236575.565853] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1236575.573610] br-lan: port 2(lan2) entered blocking state
<6>[1236575.579021] br-lan: port 2(lan2) entered forwarding state
<6>[1266652.057790] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control off
<6>[1266652.065336] br-lan: port 1(lan1) entered blocking state
<6>[1266652.070737] br-lan: port 1(lan1) entered forwarding state
<6>[1266731.094898] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1266731.100026] br-lan: port 1(lan1) entered disabled state
<6>[1266734.217060] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1266734.224772] br-lan: port 1(lan1) entered blocking state
<6>[1266734.230178] br-lan: port 1(lan1) entered forwarding state
<6>[1289590.186998] br-lan: port 2(lan2) entered disabled state
<6>[1289590.192915] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1290883.857681] br-lan: port 1(lan1) entered disabled state
<6>[1290883.863685] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1332204.834373] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control off
<6>[1332204.841910] br-lan: port 1(lan1) entered blocking state
<6>[1332204.847308] br-lan: port 1(lan1) entered forwarding state
<6>[1332245.392009] br-lan: port 1(lan1) entered disabled state
<6>[1332245.395954] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1332248.514043] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1332248.521749] br-lan: port 1(lan1) entered blocking state
<6>[1332248.527149] br-lan: port 1(lan1) entered forwarding state
<6>[1333747.223050] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control off
<6>[1333747.230589] br-lan: port 2(lan2) entered blocking state
<6>[1333747.235993] br-lan: port 2(lan2) entered forwarding state
<6>[1333760.740976] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1333760.746159] br-lan: port 2(lan2) entered disabled state
<6>[1333763.862971] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1333763.870683] br-lan: port 2(lan2) entered blocking state
<6>[1333763.876088] br-lan: port 2(lan2) entered forwarding state
<6>[1334527.215224] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1334527.220350] br-lan: port 2(lan2) entered disabled state
<6>[1335881.205330] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1335881.210486] br-lan: port 1(lan1) entered disabled state
<6>[1358400.242015] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control off
<6>[1358400.249564] br-lan: port 2(lan2) entered blocking state
<6>[1358400.254965] br-lan: port 2(lan2) entered forwarding state
<6>[1358413.759999] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1358413.765133] br-lan: port 2(lan2) entered disabled state
<6>[1358415.841885] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1358415.849591] br-lan: port 2(lan2) entered blocking state
<6>[1358415.854990] br-lan: port 2(lan2) entered forwarding state
<6>[1358427.202002] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control off
<6>[1358427.210108] br-lan: port 1(lan1) entered blocking state
<6>[1358427.215527] br-lan: port 1(lan1) entered forwarding state
<6>[1358481.279258] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1358481.284510] br-lan: port 1(lan1) entered disabled state
<6>[1358485.441348] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1358485.449059] br-lan: port 1(lan1) entered blocking state
<6>[1358485.454459] br-lan: port 1(lan1) entered forwarding state
<1>[1375173.702401] Unable to handle kernel paging request at virtual address dead000000000110
<1>[1375173.710492] Mem abort info:
<1>[1375173.713473]   ESR = 0x96000004
<1>[1375173.716800]   EC = 0x25: DABT (current EL), IL = 32 bits
<1>[1375173.722274]   SET = 0, FnV = 0
<1>[1375173.725582]   EA = 0, S1PTW = 0
<1>[1375173.728887] Data abort info:
<1>[1375173.731931]   ISV = 0, ISS = 0x00000004
<1>[1375173.735939]   CM = 0, WnR = 0
<1>[1375173.739141] [dead000000000110] address between user and kernel address ranges
<0>[1375173.746458] Internal error: Oops: 96000004 [#1] SMP
<7>[1375173.751593] Modules linked in: xt_connlimit pppoe ppp_async nf_conncount iptable_nat xt_state xt_nat xt_helper xt_conntrack xt_connmark xt_connbytes xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT wireguard pppox ppp_generic nf_nat nf_flow_table nf_conntrack_netlink nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 libchacha20poly1305 libblake2s ipt_REJECT chacha_neon cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_recent xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY slhc sch_cake poly1305_neon nfnetlink nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 libcurve25519_generic libchacha libblake2s_generic iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables hwmon crc_ccitt compat br_netfilter sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact nf_log_ipv6 nf_log_common ip6table_mangle
<7>[1375173.751766]  ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ifb ip6_udp_tunnel udp_tunnel seqiv leds_gpio xhci_plat_hcd gpio_button_hotplug
<7>[1375173.853539] CPU: 1 PID: 29337 Comm: kworker/u4:4 Tainted: G S                5.10.88 #0
<7>[1375173.861707] Hardware name: Linksys E8450 (UBI) (DT)
<7>[1375173.866762] Workqueue: nf_ft_offload_del nf_flow_table_offload_setup [nf_flow_table]
<7>[1375173.874674] pstate: 20000005 (nzCv daif -PAN -UAO -TCO BTYPE=--)
<7>[1375173.880846] pc : nf_flow_offload_ip_hook+0x778/0x1340 [nf_flow_table]
<7>[1375173.887452] lr : nf_flow_offload_ip_hook+0x78c/0x1340 [nf_flow_table]
<7>[1375173.894056] sp : ffffffc0169cbc20
<7>[1375173.897535] x29: ffffffc0169cbc20 x28: 0000000000000000 
<7>[1375173.903014] x27: ffffff80034929c8 x26: 0000000000000000 
<7>[1375173.908491] x25: ffffff80021e4d00 x24: ffffffc008b0b648 
<7>[1375173.913969] x23: 0000000000000000 x22: 0000000000000001 
<7>[1375173.919447] x21: ffffffc008b0b638 x20: 0000000000000000 
<7>[1375173.924925] x19: dead0000000000f0 x18: 0000000000000000 
<7>[1375173.930403] x17: 0000000000000000 x16: ffffffc0107f83e0 
<7>[1375173.935880] x15: 000000000000001f x14: 0000000000000000 
<7>[1375173.941358] x13: ffffffc008b0b540 x12: ffffffffffffffff 
<7>[1375173.946836] x11: 0000000000000008 x10: ffffff8000005100 
<7>[1375173.952313] x9 : ffffff80021e457c x8 : 0000000000000000 
<7>[1375173.957791] x7 : ffffffffffffffff x6 : 00a0400000000000 
<7>[1375173.963268] x5 : 0000000000000001 x4 : 0000000000000000 
<7>[1375173.968746] x3 : ffffff80003a0b00 x2 : 0000000000000000 
<7>[1375173.974223] x1 : dead000000000100 x0 : 0000000000000001 
<7>[1375173.979700] Call trace:
<7>[1375173.982316]  nf_flow_offload_ip_hook+0x778/0x1340 [nf_flow_table]
<7>[1375173.988575]  nf_flow_table_offload_setup+0x334/0x660 [nf_flow_table]
<7>[1375173.995096]  process_one_work+0x1d4/0x370
<7>[1375173.999271]  worker_thread+0x178/0x4e0
<7>[1375174.003187]  kthread+0x120/0x124
<7>[1375174.006583]  ret_from_fork+0x10/0x18
<0>[1375174.010326] Code: d1004013 eb0002bf 54000580 52800014 (f9401263) 
<4>[1375174.016585] ---[ end trace ad6bf8a1dbe70d73 ]---

And the second one was called dmesg-ramoops-1:

root@OpenWrt:~# cat /sys/fs/pstore/dmesg-ramoops-1
Panic#2 Part1
<6>[1194212.520184] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1194214.596282] mt7530 mdio-bus:00 lan2: Link is Up - 100Mbps/Full - flow control off
<6>[1194214.604009] br-lan: port 2(lan2) entered blocking state
<6>[1194214.609414] br-lan: port 2(lan2) entered forwarding state
<6>[1194234.353897] br-lan: port 2(lan2) entered disabled state
<6>[1194234.359821] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1194237.486388] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1194237.494103] br-lan: port 2(lan2) entered blocking state
<6>[1194237.499500] br-lan: port 2(lan2) entered forwarding state
<6>[1194255.153731] br-lan: port 2(lan2) entered disabled state
<6>[1194255.159723] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1194257.236287] mt7530 mdio-bus:00 lan2: Link is Up - 100Mbps/Full - flow control off
<6>[1194257.243995] br-lan: port 2(lan2) entered blocking state
<6>[1194257.249392] br-lan: port 2(lan2) entered forwarding state
<6>[1194261.393694] br-lan: port 2(lan2) entered disabled state
<6>[1194261.399611] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1201429.974414] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1201429.979530] br-lan: port 1(lan1) entered disabled state
<6>[1236558.925969] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control off
<6>[1236558.933513] br-lan: port 2(lan2) entered blocking state
<6>[1236558.938910] br-lan: port 2(lan2) entered forwarding state
<6>[1236572.443713] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1236572.448974] br-lan: port 2(lan2) entered disabled state
<6>[1236575.565853] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1236575.573610] br-lan: port 2(lan2) entered blocking state
<6>[1236575.579021] br-lan: port 2(lan2) entered forwarding state
<6>[1266652.057790] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control off
<6>[1266652.065336] br-lan: port 1(lan1) entered blocking state
<6>[1266652.070737] br-lan: port 1(lan1) entered forwarding state
<6>[1266731.094898] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1266731.100026] br-lan: port 1(lan1) entered disabled state
<6>[1266734.217060] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1266734.224772] br-lan: port 1(lan1) entered blocking state
<6>[1266734.230178] br-lan: port 1(lan1) entered forwarding state
<6>[1289590.186998] br-lan: port 2(lan2) entered disabled state
<6>[1289590.192915] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1290883.857681] br-lan: port 1(lan1) entered disabled state
<6>[1290883.863685] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1332204.834373] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control off
<6>[1332204.841910] br-lan: port 1(lan1) entered blocking state
<6>[1332204.847308] br-lan: port 1(lan1) entered forwarding state
<6>[1332245.392009] br-lan: port 1(lan1) entered disabled state
<6>[1332245.395954] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1332248.514043] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1332248.521749] br-lan: port 1(lan1) entered blocking state
<6>[1332248.527149] br-lan: port 1(lan1) entered forwarding state
<6>[1333747.223050] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control off
<6>[1333747.230589] br-lan: port 2(lan2) entered blocking state
<6>[1333747.235993] br-lan: port 2(lan2) entered forwarding state
<6>[1333760.740976] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1333760.746159] br-lan: port 2(lan2) entered disabled state
<6>[1333763.862971] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1333763.870683] br-lan: port 2(lan2) entered blocking state
<6>[1333763.876088] br-lan: port 2(lan2) entered forwarding state
<6>[1334527.215224] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1334527.220350] br-lan: port 2(lan2) entered disabled state
<6>[1335881.205330] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1335881.210486] br-lan: port 1(lan1) entered disabled state
<6>[1358400.242015] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control off
<6>[1358400.249564] br-lan: port 2(lan2) entered blocking state
<6>[1358400.254965] br-lan: port 2(lan2) entered forwarding state
<6>[1358413.759999] mt7530 mdio-bus:00 lan2: Link is Down
<6>[1358413.765133] br-lan: port 2(lan2) entered disabled state
<6>[1358415.841885] mt7530 mdio-bus:00 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1358415.849591] br-lan: port 2(lan2) entered blocking state
<6>[1358415.854990] br-lan: port 2(lan2) entered forwarding state
<6>[1358427.202002] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control off
<6>[1358427.210108] br-lan: port 1(lan1) entered blocking state
<6>[1358427.215527] br-lan: port 1(lan1) entered forwarding state
<6>[1358481.279258] mt7530 mdio-bus:00 lan1: Link is Down
<6>[1358481.284510] br-lan: port 1(lan1) entered disabled state
<6>[1358485.441348] mt7530 mdio-bus:00 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
<6>[1358485.449059] br-lan: port 1(lan1) entered blocking state
<6>[1358485.454459] br-lan: port 1(lan1) entered forwarding state
<1>[1375173.702401] Unable to handle kernel paging request at virtual address dead000000000110
<1>[1375173.710492] Mem abort info:
<1>[1375173.713473]   ESR = 0x96000004
<1>[1375173.716800]   EC = 0x25: DABT (current EL), IL = 32 bits
<1>[1375173.722274]   SET = 0, FnV = 0
<1>[1375173.725582]   EA = 0, S1PTW = 0
<1>[1375173.728887] Data abort info:
<1>[1375173.731931]   ISV = 0, ISS = 0x00000004
<1>[1375173.735939]   CM = 0, WnR = 0
<1>[1375173.739141] [dead000000000110] address between user and kernel address ranges
<0>[1375173.746458] Internal error: Oops: 96000004 [#1] SMP
<7>[1375173.751593] Modules linked in: xt_connlimit pppoe ppp_async nf_conncount iptable_nat xt_state xt_nat xt_helper xt_conntrack xt_connmark xt_connbytes xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT wireguard pppox ppp_generic nf_nat nf_flow_table nf_conntrack_netlink nf_conntrack mt7915e mt7615e mt7615_common mt76_connac_lib mt76 mac80211 libchacha20poly1305 libblake2s ipt_REJECT chacha_neon cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_recent xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY slhc sch_cake poly1305_neon nfnetlink nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 libcurve25519_generic libchacha libblake2s_generic iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables hwmon crc_ccitt compat br_netfilter sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact nf_log_ipv6 nf_log_common ip6table_mangle
<7>[1375173.751766]  ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ifb ip6_udp_tunnel udp_tunnel seqiv leds_gpio xhci_plat_hcd gpio_button_hotplug
<7>[1375173.853539] CPU: 1 PID: 29337 Comm: kworker/u4:4 Tainted: G S                5.10.88 #0
<7>[1375173.861707] Hardware name: Linksys E8450 (UBI) (DT)
<7>[1375173.866762] Workqueue: nf_ft_offload_del nf_flow_table_offload_setup [nf_flow_table]
<7>[1375173.874674] pstate: 20000005 (nzCv daif -PAN -UAO -TCO BTYPE=--)
<7>[1375173.880846] pc : nf_flow_offload_ip_hook+0x778/0x1340 [nf_flow_table]
<7>[1375173.887452] lr : nf_flow_offload_ip_hook+0x78c/0x1340 [nf_flow_table]
<7>[1375173.894056] sp : ffffffc0169cbc20
<7>[1375173.897535] x29: ffffffc0169cbc20 x28: 0000000000000000 
<7>[1375173.903014] x27: ffffff80034929c8 x26: 0000000000000000 
<7>[1375173.908491] x25: ffffff80021e4d00 x24: ffffffc008b0b648 
<7>[1375173.913969] x23: 0000000000000000 x22: 0000000000000001 
<7>[1375173.919447] x21: ffffffc008b0b638 x20: 0000000000000000 
<7>[1375173.924925] x19: dead0000000000f0 x18: 0000000000000000 
<7>[1375173.930403] x17: 0000000000000000 x16: ffffffc0107f83e0 
<7>[1375173.935880] x15: 000000000000001f x14: 0000000000000000 
<7>[1375173.941358] x13: ffffffc008b0b540 x12: ffffffffffffffff 
<7>[1375173.946836] x11: 0000000000000008 x10: ffffff8000005100 
<7>[1375173.952313] x9 : ffffff80021e457c x8 : 0000000000000000 
<7>[1375173.957791] x7 : ffffffffffffffff x6 : 00a0400000000000 
<7>[1375173.963268] x5 : 0000000000000001 x4 : 0000000000000000 
<7>[1375173.968746] x3 : ffffff80003a0b00 x2 : 0000000000000000 
<7>[1375173.974223] x1 : dead000000000100 x0 : 0000000000000001 
<7>[1375173.979700] Call trace:
<7>[1375173.982316]  nf_flow_offload_ip_hook+0x778/0x1340 [nf_flow_table]
<7>[1375173.988575]  nf_flow_table_offload_setup+0x334/0x660 [nf_flow_table]
<7>[1375173.995096]  process_one_work+0x1d4/0x370
<7>[1375173.999271]  worker_thread+0x178/0x4e0
<7>[1375174.003187]  kthread+0x120/0x124
<7>[1375174.006583]  ret_from_fork+0x10/0x18
<0>[1375174.010326] Code: d1004013 eb0002bf 54000580 52800014 (f9401263) 
<4>[1375174.016585] ---[ end trace ad6bf8a1dbe70d73 ]---
<0>[1375174.034746] Kernel panic - not syncing: Oops: Fatal exception
<2>[1375174.040664] SMP: stopping secondary CPUs
<0>[1375174.044757] Kernel Offset: disabled
<0>[1375174.048411] CPU features: 0x0000002,04002004
<0>[1375174.052844] Memory Limit: none

Curious if this contains anything useful. If there is anything else I can provide, please let me know!