Nothing surprising there, but worth the check.
Oh, actually, I'll take a screenshot of that page again whenever the issue occurs again. Maybe there'll be a difference then.
edit: it happened again. no difference when I checked the page.
1 Like
Good Evening,
I installed Openwrt recently on the EA6350v3, and encountered this same issue.
The WAN and LAN ports are handled by the same ethernet controller on this device.
If you pump too much data through the controller it locks up.
I was only able to trigger this with a hardwired device, wifi doesn't appear to run enough bandwidth to trigger the bug.
It seems to be similar to the issue plaguing rk3328/rk3399 ethernet controllers, where heavy load causes the ethernet controller to lock up.
On the rockchips, if you send too much data through the controller with tx coe offloading enabled, the buffers get overloaded and the controller crashes.
Unfortunately disabling offloading on the ea6350 only raises the threshold for the bug to trigger.
I was unable to fully diagnose the issue, before being forced to return to stock firmware.
This seems related to the issues people have with certain websites not working correctly, which can be mitigated by lowering the MTU.
I found references of a new ethernet driver being developed by @blogic, but I cannot find the actual driver.
Mainline linux does not have a driver for this ethernet controller.
I'm also experiencing this. My parents gigabit internet will cause the ethernet switch to stop working after most internet speed tests are run. Can still connect to the router over wifi with nothing shown in the logs, a reboot brings it back up.
I've read that the current driver is essedma and possibly in the future this will switch to DSA qca8k? I'm brand new to OpenWRT, is there any hope of this being fixed or should I look into getting another router?
1 Like
fwiw, perhaps try @NoTengoBattery 's latest snapshot to see if there is any difference?
https://forum.openwrt.org/t/optimized-nas-latency-memory-build-for-linksys-ea6350v3-civic/44125
I should have added, I started with release version of 19.07, then the latest snapshot and @NoTengoBattery latest build, all with settings reset (just installing Luci in the snapshot). They all had the same issue eventually.
It does not occur every time, and going by gut feeling it seems like the later builds required more runs of the speedtest before failing but that's just a hunch. The speed tests generally failed showing >500Mbps on screen at the time.
I have flashed stock 19.07 back on. I hooked up the EA6350 WAN port to my existing LAN switch. I then attached a computer to the EA6350 LAN as iperf3 client and on my real LAN had a machine as iperf3 server.
Does this testing make sense? I would assume it should be close to the speed tests that caused the switch to stop working at my parents. ~870 Mbits/sec and ~940 in reverse mode.
I could not manage to make the switch stop working.
Some differences:
- Parents have IPv6, this test no IPv6. I don't recall but I imagine most of the parents test would have been over IPv6.
- Parents home the LAN computer was a desktop running Ubuntu machine, this test was an iMac running macOS
- Parents WAN was connected to Cable Modem, this test was a HP switch
Any thoughts?
Maybe install collectd statistics? Might get lucky and capture a trend on memory or processor usage that is helpful - or it just tells you you crashed again.
I've run a few iperf tests on my EA6350v3 running snapshot r11735 that pushed its wired speed near Gigabit without issues, but my current ISP is too slow to challenge WAN throughput.
Every year or so I swap back and forth from 40/2 bonded VDSL2 and 200/10 cable to keep the introductory rate. Our connected, streaming, VOIP family is hard pressed to notice a slowdown on 40/2 with CAKE SQM/QoS. We're currently on 40/2.
As a simple test, have you considered disabling IPv6 by deleting WAN6 interface, assuming your parents have IPv4 ?
The OP in this thread thought problems may be caused by IPv6.
Fritzbox 4040 problem
(Unable to ping router at 192.168.1.1 when it fails)
Thanks for the help! Since my home does not have IPv6, I assigned a prefix to the WAN of my real router and with the same iperf set up as before now using IPv6... it crashes!
I was logged in with SSH reading the logs and here they are:
Wed Jan 15 00:37:11 2020 kern.alert kernel: [ 1094.936813] BUG: Bad page state in process swapper/0 pfn:8d4d0
Wed Jan 15 00:37:11 2020 kern.emerg kernel: [ 1094.936855] page:cffa2a00 count:-1 mapcount:0 mapping: (null) index:0x0
Wed Jan 15 00:37:11 2020 kern.emerg kernel: [ 1094.941546] flags: 0x0()
Wed Jan 15 00:37:11 2020 kern.alert kernel: [ 1094.948495] raw: 00000000 00000000 00000000 ffffffff ffffffff 00000100 00000200 00000000
Wed Jan 15 00:37:11 2020 kern.alert kernel: [ 1094.951008] page dumped because: nonzero _count
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1094.959074] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple gpio_button_hotplug
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.007768] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.162 #0
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.029997] Hardware name: Generic DT based system
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.036257] Function entered at [<c030e2a8>] from [<c030a7a8>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.040938] Function entered at [<c030a7a8>] from [<c073f994>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.046754] Function entered at [<c073f994>] from [<c03a2fb4>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.052570] Function entered at [<c03a2fb4>] from [<c03a5430>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.058386] Function entered at [<c03a5430>] from [<c03a5d08>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.064201] Function entered at [<c03a5d08>] from [<c03a69a4>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.070016] Function entered at [<c03a69a4>] from [<c0628114>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.075834] Function entered at [<c0628114>] from [<c0628190>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.081647] Function entered at [<c0628190>] from [<c05cda7c>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.087466] Function entered at [<c05cda7c>] from [<c05d0c24>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.093280] Function entered at [<c05d0c24>] from [<c063c41c>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.099097] Function entered at [<c063c41c>] from [<c0301520>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.104912] Function entered at [<c0301520>] from [<c0321e3c>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.110729] Function entered at [<c0321e3c>] from [<c035b308>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.116546] Function entered at [<c035b308>] from [<c030140c>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.122360] Function entered at [<c030140c>] from [<c030b30c>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.128176] Exception stack(0xc0a01f40 to 0xc0a01f88)
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.133997] 1f40: 00000001 00000000 00000000 c0313960 ffffe000 c0a03cb8 c0a03c6c 00000000
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.139124] 1f60: 00000000 00000001 cfffcd40 c092da28 c0a01f88 c0a01f90 c0307d88 c0307d8c
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.147276] 1f80: 60000013 ffffffff
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.155429] Function entered at [<c030b30c>] from [<c0307d8c>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.158733] Function entered at [<c0307d8c>] from [<c0352058>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.164635] Function entered at [<c0352058>] from [<c0352378>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.170451] Function entered at [<c0352378>] from [<c0900c04>]
Wed Jan 15 00:37:11 2020 kern.warn kernel: [ 1095.176267] Disabling lock debugging due to kernel taint
This run iperf3 client "error - unable to write to stream socket: Broken pipe", the link seemed to restore and when I tried again at [467.147594] the switch stayed down:
Wed Jan 15 00:51:36 2020 kern.alert kernel: [ 443.412420] BUG: Bad page state in process swapper/0 pfn:8d8fa
Wed Jan 15 00:51:36 2020 kern.emerg kernel: [ 443.412461] page:cffaaf40 count:-1 mapcount:0 mapping: (null) index:0x0
Wed Jan 15 00:51:36 2020 kern.emerg kernel: [ 443.417152] flags: 0x0()
Wed Jan 15 00:51:36 2020 kern.alert kernel: [ 443.424100] raw: 00000000 00000000 00000000 ffffffff ffffffff 00000000 cffaaf54 00000000
Wed Jan 15 00:51:36 2020 kern.alert kernel: [ 443.426615] page dumped because: nonzero _count
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.434680] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple gpio_button_hotplug
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.483374] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.162 #0
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.505602] Hardware name: Generic DT based system
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.511863] Function entered at [<c030e2a8>] from [<c030a7a8>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.516543] Function entered at [<c030a7a8>] from [<c073f994>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.522360] Function entered at [<c073f994>] from [<c03a2fb4>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.528175] Function entered at [<c03a2fb4>] from [<c03a5430>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.533992] Function entered at [<c03a5430>] from [<c03a5d08>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.539807] Function entered at [<c03a5d08>] from [<c03a69a4>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.545621] Function entered at [<c03a69a4>] from [<c0628114>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.551440] Function entered at [<c0628114>] from [<c0628190>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.557254] Function entered at [<c0628190>] from [<c05cda7c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.563071] Function entered at [<c05cda7c>] from [<c05d0c24>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.568886] Function entered at [<c05d0c24>] from [<c063c41c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.574704] Function entered at [<c063c41c>] from [<c0301520>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.580520] Function entered at [<c0301520>] from [<c0321e3c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.586335] Function entered at [<c0321e3c>] from [<c035b308>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.592152] Function entered at [<c035b308>] from [<c030140c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.597965] Function entered at [<c030140c>] from [<c030b30c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.603783] Exception stack(0xc0a01f40 to 0xc0a01f88)
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.609604] 1f40: 00000001 00000000 00000000 c0313960 ffffe000 c0a03cb8 c0a03c6c 00000000
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.614729] 1f60: 00000000 00000001 cfffcd40 c092da28 c0a01f88 c0a01f90 c0307d88 c0307d8c
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.622883] 1f80: 60000013 ffffffff
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.631035] Function entered at [<c030b30c>] from [<c0307d8c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.634337] Function entered at [<c0307d8c>] from [<c0352058>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.640241] Function entered at [<c0352058>] from [<c0352378>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.646056] Function entered at [<c0352378>] from [<c0900c04>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.651873] Disabling lock debugging due to kernel taint
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.663306] ------------[ cut here ]------------
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.663359] WARNING: CPU: 0 PID: 1767 at net/core/dev.c:4700 0xc063c104
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.667803] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple gpio_button_hotplug
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.718585] CPU: 0 PID: 1767 Comm: dropbear Tainted: G B 4.14.162 #0
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.740761] Hardware name: Generic DT based system
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.748407] Function entered at [<c030e2a8>] from [<c030a7a8>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.753089] Function entered at [<c030a7a8>] from [<c073f994>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.758905] Function entered at [<c073f994>] from [<c031dd34>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.764721] Function entered at [<c031dd34>] from [<c031de00>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.770534] Function entered at [<c031de00>] from [<c063c104>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.776353] Function entered at [<c063c104>] from [<c063c184>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.782167] Function entered at [<c063c184>] from [<c063c48c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.787983] Function entered at [<c063c48c>] from [<c0301520>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.793800] Function entered at [<c0301520>] from [<c0321e3c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.799615] Function entered at [<c0321e3c>] from [<c035b308>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.805433] Function entered at [<c035b308>] from [<c030140c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.811246] Function entered at [<c030140c>] from [<c030b6b0>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.817064] Exception stack(0xce95ffb0 to 0xce95fff8)
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.822881] ffa0: 00000001 bed43480 000001f0 0000002b
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.828011] ffc0: bed43480 b6f76230 0000005e 000001d0 bed434a4 00000020 b6f73010 00025a9c
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.836169] ffe0: 00000010 bed43468 0001f5d0 000123c4 60000010 ffffffff
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.844409] ---[ end trace 963ab92f22dbc474 ]---
Wed Jan 15 00:51:36 2020 kern.alert kernel: [ 443.853363] BUG: Bad page state in process ksoftirqd/0 pfn:8d928
Wed Jan 15 00:51:36 2020 kern.emerg kernel: [ 443.855616] page:cffab500 count:-1 mapcount:0 mapping: (null) index:0x0
Wed Jan 15 00:51:36 2020 kern.emerg kernel: [ 443.861595] flags: 0x0()
Wed Jan 15 00:51:36 2020 kern.alert kernel: [ 443.868372] raw: 00000000 00000000 00000000 ffffffff ffffffff 00000100 00000200 00000000
Wed Jan 15 00:51:36 2020 kern.alert kernel: [ 443.870884] page dumped because: nonzero _count
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.878951] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple gpio_button_hotplug
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.927646] CPU: 0 PID: 7 Comm: ksoftirqd/0 Tainted: G B W 4.14.162 #0
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.949876] Hardware name: Generic DT based system
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.957522] Function entered at [<c030e2a8>] from [<c030a7a8>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.962204] Function entered at [<c030a7a8>] from [<c073f994>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.968020] Function entered at [<c073f994>] from [<c03a2fb4>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.973835] Function entered at [<c03a2fb4>] from [<c03a5430>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.979650] Function entered at [<c03a5430>] from [<c03a5d08>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.985467] Function entered at [<c03a5d08>] from [<c03a69a4>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.991281] Function entered at [<c03a69a4>] from [<c0628114>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 443.997099] Function entered at [<c0628114>] from [<c0628190>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.002913] Function entered at [<c0628190>] from [<c05cda7c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.008731] Function entered at [<c05cda7c>] from [<c05d0bac>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.014545] Function entered at [<c05d0bac>] from [<c063c41c>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.020363] Function entered at [<c063c41c>] from [<c0301520>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.026179] Function entered at [<c0301520>] from [<c0321b68>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.031995] Function entered at [<c0321b68>] from [<c033bea4>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.037811] Function entered at [<c033bea4>] from [<c0339060>]
Wed Jan 15 00:51:36 2020 kern.warn kernel: [ 444.043626] Function entered at [<c0339060>] from [<c03074a8>]
Wed Jan 15 00:51:59 2020 kern.warn kernel: [ 467.147594] ------------[ cut here ]------------
Wed Jan 15 00:51:59 2020 kern.warn kernel: [ 467.147640] WARNING: CPU: 0 PID: 0 at net/core/dev.c:4700 0xc063c104
Wed Jan 15 00:51:59 2020 kern.warn kernel: [ 467.151298] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple gpio_button_hotplug
Wed Jan 15 00:51:59 2020 kern.warn kernel: [ 467.201852] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G B W 4.14.162 #0
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.224032] Hardware name: Generic DT based system
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.231328] Function entered at [<c030e2a8>] from [<c030a7a8>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.236184] Function entered at [<c030a7a8>] from [<c073f994>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.242001] Function entered at [<c073f994>] from [<c031dd34>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.247817] Function entered at [<c031dd34>] from [<c031de00>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.253630] Function entered at [<c031de00>] from [<c063c104>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.259447] Function entered at [<c063c104>] from [<c063c184>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.265262] Function entered at [<c063c184>] from [<c063c1d0>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.271078] Function entered at [<c063c1d0>] from [<c05d0cc0>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.276896] Function entered at [<c05d0cc0>] from [<c063c41c>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.282709] Function entered at [<c063c41c>] from [<c0301520>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.288527] Function entered at [<c0301520>] from [<c0321e3c>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.294344] Function entered at [<c0321e3c>] from [<c035b308>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.300159] Function entered at [<c035b308>] from [<c030140c>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.305974] Function entered at [<c030140c>] from [<c030b30c>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.311791] Exception stack(0xc0a01f40 to 0xc0a01f88)
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.317613] 1f40: 00000001 00000000 00000000 c0313960 ffffe000 c0a03cb8 c0a03c6c 00000000
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.322738] 1f60: 00000000 00000001 cfffcd40 c092da28 c0a01f88 c0a01f90 c0307d88 c0307d8c
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.330892] 1f80: 60000013 ffffffff
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.339045] Function entered at [<c030b30c>] from [<c0307d8c>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.342347] Function entered at [<c0307d8c>] from [<c0352058>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.348249] Function entered at [<c0352058>] from [<c0352378>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.354066] Function entered at [<c0352378>] from [<c0900c04>]
Wed Jan 15 00:52:00 2020 kern.warn kernel: [ 467.359978] ---[ end trace 963ab92f22dbc475 ]---
Wed Jan 15 00:52:00 2020 kern.alert kernel: [ 467.370760] Unable to handle kernel NULL pointer dereference at virtual address 00000000
Wed Jan 15 00:52:00 2020 kern.alert kernel: [ 467.370804] pgd = c0204000
Another event:
Wed Jan 15 00:56:36 2020 kern.err kernel: [ 121.687088] ess_edma c080000.edma: Invalid RRD source port bit set
Wed Jan 15 00:56:36 2020 kern.err kernel: [ 121.687195] ess_edma c080000.edma: Invalid RRD source port bit set
Wed Jan 15 00:56:36 2020 kern.err kernel: [ 121.695201] ess_edma c080000.edma: Invalid RRD source port bit set
Wed Jan 15 00:56:36 2020 kern.alert kernel: [ 121.703775] Unable to handle kernel NULL pointer dereference at virtual address 00000004
Wed Jan 15 00:56:36 2020 kern.alert kernel: [ 121.704505] pgd = c0204000
Wed Jan 15 00:56:36 2020 kern.alert kernel: [ 121.712797] [00000004] *pgd=00000000
I have reset the router to defaults and only changed the WAN IPv6 prefix length and during the failures many error lines are in logs like posts above.
Enabling the Wifi seems to generate less errors.
2.4GHz only:
Wed Jan 15 01:40:36 2020 kern.alert kernel: [ 103.230457] Unable to handle kernel NULL pointer dereference at virtual address 00000004
Wed Jan 15 01:40:36 2020 kern.alert kernel: [ 103.230497] pgd = c0204000
Wed Jan 15 01:40:36 2020 kern.alert kernel: [ 103.237612] [00000004] *pgd=00000000
Wed Jan 15 01:40:36 2020 kern.emerg kernel: [ 103.240191] Internal error: Oops: 817 [#1] SMP ARM
5GHz only:
Wed Jan 15 01:43:58 2020 kern.warn kernel: [ 184.317800] backlog: Budget exhausted after napi rescheduled
With both 2.4GHz and 5GHz enabled it seems to fail so you can still connect to Luci over WiFi but with no errors in logs (like I experienced at parents).
WiFi status could be random and mean nothing but hoping the logs can help.
So would deleting the WAN6 interface be a temporarily fix for this issue?
You may wish to raise a bug report with your findings.
https://bugs.openwrt.org/
Update: trr's bug report: [FS#2741]
https://bugs.openwrt.org/index.php?do=details&task_id=2741
Should be OK, I'll bring the router back and try it out.
Reported the issue. Thanks!
Good Morning,
I've lost access to the EA6350v3 I was working with (though I am tempted to buy one).
Would you be able to flash the debug kernel and re-run the test?
Thanks!
Peter
Is this related to the OPs problem?
If this is a general issue with flashing the EA6350v3, I'd suggest making a new thread for wider viewing.
I would like to add that I have these issues on 2 Fritz!Box 4040's and a Linksys EA8300. In my case I have to have IPv6 enabled, use the lan interface on my desktop and create some network traffic with YouTube or wget, this will drop out reproducible my complete network. As of the affected models with this issue it might be related to the Qualcomm IPQ40xx driver.
Here is my bug report: https://bugs.openwrt.org/index.php?do=details&task_id=2591
Completely, if you look at the kernel panic, it is missing all of the debugging symbols to trace what actually happened.
Unless you know where the debugging symbols can be located for that build, the only other option is to flash a debug kernel so the stack trace has access to the symbols.
Once we can trace the functions, we can find where things are breaking.
IPv6 only issues tend to be variable size problems.
1 Like
I can certainly try flashing a debug kernel. I'm assuming I would have to build my own firmware? I've gotten to the "make menuconfig", what specific options should I select? Under "Global build settings" is "Collect kernel debug information", is this it? Thanks!
Update... I've flashed release build of 19.07 with just adding Luci and checking "Collect kernel debug information" in menuconfig
root@OpenWrt:~# logread -f
Wed Jan 22 20:26:55 2020 kern.alert kernel: [ 118.339257] BUG: Bad page state in process swapper/0 pfn:8d44a
Wed Jan 22 20:26:55 2020 kern.emerg kernel: [ 118.339305] page:cffa1940 count:-1 mapcount:0 mapping: (null) index:0x0
Wed Jan 22 20:26:55 2020 kern.emerg kernel: [ 118.344006] flags: 0x0()
Wed Jan 22 20:26:55 2020 kern.alert kernel: [ 118.350960] raw: 00000000 00000000 00000000 ffffffff ffffffff 00000000 cffa1954 00000000
Wed Jan 22 20:26:55 2020 kern.alert kernel: [ 118.353453] page dumped because: nonzero _count
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.361521] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple gpio_button_hotplug
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.409867] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.162 #0
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.432094] Hardware name: Generic DT based system
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.438200] [<c030e3a8>] (unwind_backtrace) from [<c030a8a0>] (show_stack+0x10/0x14)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.442876] [<c030a8a0>] (show_stack) from [<c071a454>] (dump_stack+0x94/0xa8)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.450773] [<c071a454>] (dump_stack) from [<c03a4988>] (bad_page+0x11c/0x138)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.457802] [<c03a4988>] (bad_page) from [<c03a6e04>] (get_page_from_freelist+0x4d4/0x800)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.465008] [<c03a6e04>] (get_page_from_freelist) from [<c03a76dc>] (__alloc_pages_nodemask+0x10c/0xd0c)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.473255] [<c03a76dc>] (__alloc_pages_nodemask) from [<c03a8378>] (page_frag_alloc+0x3c/0x140)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.482893] [<c03a8378>] (page_frag_alloc) from [<c0619e64>] (__netdev_alloc_skb+0x8c/0xfc)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.491658] [<c0619e64>] (__netdev_alloc_skb) from [<c0619ee0>] (__netdev_alloc_skb_ip_align+0xc/0x30)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.499732] [<c0619ee0>] (__netdev_alloc_skb_ip_align) from [<c05c0b64>] (edma_alloc_rx_buf+0xa8/0x414)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.509109] [<c05c0b64>] (edma_alloc_rx_buf) from [<c05c3d0c>] (edma_poll+0xcbc/0xe4c)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.518397] [<c05c3d0c>] (edma_poll) from [<c062dc60>] (net_rx_action+0x138/0x2fc)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.526378] [<c062dc60>] (net_rx_action) from [<c0301520>] (__do_softirq+0xe0/0x240)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.533930] [<c0301520>] (__do_softirq) from [<c0321f3c>] (irq_exit+0xd4/0x138)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.541830] [<c0321f3c>] (irq_exit) from [<c035b278>] (__handle_domain_irq+0x9c/0xac)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.548860] [<c035b278>] (__handle_domain_irq) from [<c030140c>] (gic_handle_irq+0x5c/0x90)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.556847] [<c030140c>] (gic_handle_irq) from [<c030b40c>] (__irq_svc+0x6c/0x90)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.564999] Exception stack(0xc0a01f40 to 0xc0a01f88)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.572642] 1f40: 00000001 00000000 00000000 c0313a60 ffffe000 c0a03cb8 c0a03c6c 00000000
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.577680] 1f60: 00000000 00000001 cfffcd40 c092ca28 c0a01f88 c0a01f90 c0307e48 c0307e4c
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.585833] 1f80: 60000013 ffffffff
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.593993] [<c030b40c>] (__irq_svc) from [<c0307e4c>] (arch_cpu_idle+0x34/0x38)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.597301] [<c0307e4c>] (arch_cpu_idle) from [<c0351fc8>] (do_idle+0xdc/0x1a0)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.604938] [<c0351fc8>] (do_idle) from [<c03522e8>] (cpu_startup_entry+0x18/0x1c)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.611972] [<c03522e8>] (cpu_startup_entry) from [<c0900c80>] (start_kernel+0x3b8/0x3c4)
Wed Jan 22 20:26:55 2020 kern.warn kernel: [ 118.619603] Disabling lock debugging due to kernel taint
2 Likes