Hi,
I have 2 Netgear R6220 / MT7621 devices. After upgrading from 19.07.3 to 19.07.4 both of them are randomly rebooting. This only happens with HW flow offloading enable. If I have disable hw slow offloading (just leaving software) then they are stable (uptime >7 days). One is on my desk with a clean config and the other is on the network.
This is a cat /sys/kernel/debug/crashlog that I found on the device on the network:
Time: 1605435354.554435
Modules: pppoe@87010000+2400 ppp_async@8700a000+1b90 pppox@86feb000+4b0 ppp_generic@86df8000+5970 nf_conntrack_netlink@86df0000+5a70 nf_conntrack_ipv6@86de8000+1590 mt76x2e@86fdc000+26c0 mt76x2_common@86f94000+2a40 mt76x02_lib@86ce0000+9700 mt7603e@877b0000+9100 mt76@87700000+79e0 mac80211@86c00000+6cca0 iptable_nat@86fec000+2c0 ipt_REJECT@87678000+3c0 ipt_MASQUERADE@86f54000+2a0 cfg80211@87740000+36c20 cdc_ether@86ffe000+1300 xt_time@86fed000+6e0 xt_tcpudp@86fea000+700 xt_state@86f47000+2c0 xt_recent@86fee000+1bf0 xt_nat@86fbf000+600 xt_multiport@86fad000+500 xt_mark@86fab000+2a0 xt_mac@86fa9000+260 xt_limit@86fa2000+4a0 xt_helper@876ea000+360 xt_conntrack@86e00000+900 xt_connmark@8767b000+4e0 xt_connlimit@876e8000+1510 xt_connbytes@86f07000+640 xt_comment@87e5e000+1e0 xt_TCPMSS@86fb5000+ae0 xt_REDIRECT@876f2000+2c0 xt_LOG@8767a000+300 xt_IPMARK@86f69000+3a0 xt_FLOWOFFLOAD@86f50000+c10 xt_CT@876d0000+a60 wireguard@86fc0000+166c0 usbnet@87e58000+4490 slhc@86fb2000+1400 nfnetlink@86f92000+1110 nf_reject_ipv4@86f67000+840 nf_nat_redirect@86f61000+580 nf_nat_masquerade_ipv4@86f44000+650 nf_conntrack_ipv4@86f58000+1420 nf_nat_ipv4@86f6c000+f30 nf_nat@86f98000+37b0 nf_log_ipv4@86f63000+d60 nf_flow_table_hw@86f6a000+8f0 nf_flow_table@86f7c000+3350 nf_defrag_ipv6@86f78000+12e0 nf_defrag_ipv4@876ff000+440 nf_conntrack_rtcache@876ee000+aa0 nf_conntrack@86f80000+10770 iptable_raw@86f55000+260 iptable_mangle@86f52000+380 iptable_filter@876f9000+280 ip_tables@86f4c000+2b30 crc_ccitt@87674000+3e0 compat_xtables@876da000+220 compat@876ce000+1740 ledtrig_usbport@876f1000+ab0 nf_log_ipv6@876f4000+1020 nf_log_common@876ed000+aa0 ip6table_mangle@876d9000+4a0 ip6table_filter@876d3000+280 ip6_tables@876e0000+2a20 ip6t_REJECT@876a0000+400 x_tables@876d4000+3290 nf_reject_ipv6@86f01000+9e0 ip6_udp_tunnel@876c9000+640 udp_tunnel@876c8000+980 sit@876a4000+37c0 tunnel4@86f00000+820 ip_tunnel@876a8000+2c20 mii@86f32000+1480 leds_gpio@875df000+b60 xhci_plat_hcd@875e8000+15c0 xhci_pci@875d4000+c20 xhci_mtk@875d6000+1460 xhci_hcd@87620000+16310 gpio_button_hotplug@875be000+1ac0 usbcore@87600000+1fe50 nls_base@875b4000+1430 usb_common@875a9000+8c0
<5>[ 3.143483] 0x000000100000-0x000000200000 : "SC PID"
<5>[ 3.154427] 0x000000200000-0x000000600000 : "kernel"
<5>[ 3.165424] 0x000000600000-0x000002200000 : "ubi"
<5>[ 3.176052] 0x000002e00000-0x000002f00000 : "factory"
<5>[ 3.187228] 0x000004200000-0x000007e00000 : "reserved"
<4>[ 3.199034] [mtk_nand] probe successfully!
<4>[ 3.207908] Signature matched and data read!
<4>[ 3.216415] load_fact_bbt success 1023
<6>[ 3.224655] libphy: Fixed MDIO Bus: probed
<6>[ 3.304350] libphy: mdio: probed
<6>[ 4.710402] mtk_soc_eth 1e100000.ethernet: loaded mt7530 driver
<6>[ 4.722932] mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 17
<6>[ 4.742614] NET: Registered protocol family 10
<6>[ 4.753062] Segment Routing with IPv6
<6>[ 4.760448] NET: Registered protocol family 17
<6>[ 4.769427] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
<6>[ 4.795216] 8021q: 802.1Q VLAN Support v1.8
<5>[ 4.806761] UBI: auto-attach mtd3
<5>[ 4.813459] ubi0: attaching mtd3
<6>[ 4.907872] mtk_soc_eth 1e100000.ethernet eth0: port 1 link up
<5>[ 5.087487] ubi0: scanning is finished
<5>[ 5.109891] ubi0: attached mtd3 (name "ubi", size 28 MiB)
<5>[ 5.120685] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
<5>[ 5.134386] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
<5>[ 5.147903] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
<5>[ 5.161777] ubi0: good PEBs: 224, bad PEBs: 0, corrupted PEBs: 0
<5>[ 5.173739] ubi0: user volume: 2, internal volumes: 1, max. volumes count: 128
<5>[ 5.188120] ubi0: max/mean erase counter: 9/4, WL threshold: 4096, image sequence number: 1523296686
<5>[ 5.206306] ubi0: available PEBs: 0, total reserved PEBs: 224, PEBs reserved for bad PEB handling: 20
<5>[ 5.224695] ubi0: background thread "ubi_bgt0d" started, PID 363
<6>[ 5.226541] block ubiblock0_0: created from ubi0:0(rootfs)
<5>[ 5.247627] ubiblock: device ubiblock0_0 (rootfs) set to be root filesystem
<6>[ 5.261496] hctosys: unable to open rtc device (rtc0)
<6>[ 5.271845] usb_vbus: disabling
<5>[ 5.280563] random: fast init done
<6>[ 5.291320] VFS: Mounted root (squashfs filesystem) readonly on device 254:0.
<6>[ 5.309805] Freeing unused kernel memory: 1260K
<4>[ 5.318857] This architecture does not have kernel memory protection.
<14>[ 5.792004] init: Console is alive
<14>[ 5.799005] init: - watchdog -
<6>[ 6.011532] mtk_soc_eth 1e100000.ethernet eth0: port 4 link up
<6>[ 6.143988] mtk_soc_eth 1e100000.ethernet eth0: port 0 link up
<14>[ 6.425173] kmodloader: loading kernel modules from /etc/modules-boot.d/*
<6>[ 6.540158] usbcore: registered new interface driver usbfs
<6>[ 6.551237] usbcore: registered new interface driver hub
<6>[ 6.561978] usbcore: registered new device driver usb
<4>[ 6.579311] xhci-mtk 1e1c0000.xhci: 1e1c0000.xhci supply vusb33 not found, using dummy regulator
<6>[ 6.597093] xhci-mtk 1e1c0000.xhci: xHCI Host Controller
<6>[ 6.607743] xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
<6>[ 6.631948] xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
<6>[ 6.650236] xhci-mtk 1e1c0000.xhci: irq 16, io mem 0x1e1c0000
<6>[ 6.662732] hub 1-0:1.0: USB hub found
<6>[ 6.670266] hub 1-0:1.0: 2 ports detected
<6>[ 6.678882] xhci-mtk 1e1c0000.xhci: xHCI Host Controller
<6>[ 6.689539] xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
<6>[ 6.704474] xhci-mtk 1e1c0000.xhci: Host supports USB 3.0 SuperSpeed
<6>[ 6.717498] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
<6>[ 6.734525] hub 2-0:1.0: USB hub found
<6>[ 6.742092] hub 2-0:1.0: 1 port detected
<14>[ 6.755196] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
<14>[ 6.772170] init: - preinit -
<6>[ 7.503396] mtk_soc_eth 1e100000.ethernet: PPE started
<5>[ 7.594284] random: procd: uninitialized urandom read (4 bytes read)
<5>[ 10.747647] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 463
<5>[ 10.820388] UBIFS (ubi0:1): recovery needed
<5>[ 11.019025] UBIFS (ubi0:1): recovery completed
<5>[ 11.028057] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
<5>[ 11.043656] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
<5>[ 11.063399] UBIFS (ubi0:1): FS size: 21458944 bytes (20 MiB, 169 LEBs), journal size 1015809 bytes (0 MiB, 8 LEBs)
<5>[ 11.084002] UBIFS (ubi0:1): reserved for root: 1013558 bytes (989 KiB)
<5>[ 11.096998] UBIFS (ubi0:1): media format: w4/r0 (latest is w5/r0), UUID <hidden>, small LPT model
<14>[ 11.136494] mount_root: switching to ubifs overlay
<12>[ 11.175887] urandom-seed: Seeding with /etc/urandom.seed
<6>[ 11.386486] mtk_soc_eth 1e100000.ethernet: 0x100 = 0x6060000c, 0x10c = 0x80818
<14>[ 11.410425] procd: - early -
<14>[ 11.416320] procd: - watchdog -
<5>[ 11.984857] random: jshn: uninitialized urandom read (4 bytes read)
<14>[ 12.122068] procd: - watchdog -
<14>[ 12.128911] procd: - ubus -
<5>[ 12.168851] random: jshn: uninitialized urandom read (4 bytes read)
<5>[ 12.265771] random: ubusd: uninitialized urandom read (4 bytes read)
<14>[ 12.307183] procd: - init -
<14>[ 13.496287] kmodloader: loading kernel modules from /etc/modules.d/*
<6>[ 13.524462] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
<6>[ 13.557930] ip6_tables: (C) 2000-2006 Netfilter Core Team
<6>[ 13.597645] Loading modules backported from Linux version v4.19.137-0-gc076c79e03c6
<6>[ 13.613038] Backport generated by backports.git v4.19.137-1-0-g60c3a249
<6>[ 13.634922] ip_tables: (C) 2000-2006 Netfilter Core Team
<6>[ 13.656450] nf_conntrack version 0.5.0 (2048 buckets, 8192 max)
<14>[ 13.710984] urngd: v1.0.2 started.
<6>[ 13.728334] Netfilter messages via NETLINK v0.30.
<6>[ 13.794867] wireguard: WireGuard 1.0.20200611 loaded. See www.wireguard.com for information.
<6>[ 13.811746] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
<6>[ 13.868044] xt_time: kernel timezone is -0000
<6>[ 13.880803] usbcore: registered new interface driver cdc_ether
<5>[ 13.971161] random: crng init done
<5>[ 13.977997] random: 6 urandom warning(s) missed due to ratelimiting
<4>[ 14.044484] bus=0x2, slot = 0x1, irq=0xff
<6>[ 14.052777] mt7603e 0000:02:00.0: ASIC revision: 76030010
<6>[ 15.093030] mt7603e 0000:02:00.0: Firmware Version: ap_pcie
<6>[ 15.104214] mt7603e 0000:02:00.0: Build Time: 20160107100755
<6>[ 15.151801] mt7603e 0000:02:00.0: firmware init done
<7>[ 15.330761] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht'
<4>[ 15.344985] bus=0x1, slot = 0x0, irq=0xff
<6>[ 15.353287] mt76x2e 0000:01:00.0: ASIC revision: 76120044
<6>[ 15.981826] usb 1-2: new high-speed USB device number 2 using xhci-mtk
<6>[ 16.066304] mt76x2e 0000:01:00.0: ROM patch build: 20141115060606a
<6>[ 16.141560] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
<6>[ 16.152538] mt76x2e 0000:01:00.0: Build: 1
<6>[ 16.160676] mt76x2e 0000:01:00.0: Build Time: 201507311614____
<6>[ 16.191941] mt76x2e 0000:01:00.0: Firmware running!
<7>[ 16.204650] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'
<6>[ 16.222943] ctnetlink v0.93: registering with nfnetlink.
<6>[ 16.236797] PPP generic driver version 2.4.2
<6>[ 16.248184] NET: Registered protocol family 24
<14>[ 16.261061] kmodloader: done loading kernel modules from /etc/modules.d/*
<6>[ 16.590760] usb 1-2: USB disconnect, device number 2
<6>[ 17.321950] usb 1-2: new high-speed USB device number 3 using xhci-mtk
<6>[ 17.523060] cdc_ether 1-2:1.0 eth1: register 'cdc_ether' at usb-1e1c0000.xhci-2, CDC Ethernet Device, 0c:5b:8f:27:9a:64
<6>[ 27.078786] mtk_soc_eth 1e100000.ethernet: PPE started
<6>[ 27.096174] br-lan: port 1(eth0.1) entered blocking state
<6>[ 27.107179] br-lan: port 1(eth0.1) entered disabled state
<6>[ 27.118502] device eth0.1 entered promiscuous mode
<6>[ 27.128136] device eth0 entered promiscuous mode
<6>[ 27.142947] br-lan: port 1(eth0.1) entered blocking state
<6>[ 27.153755] br-lan: port 1(eth0.1) entered forwarding state
<6>[ 27.165315] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
<3>[ 27.206683] cdc_ether 1-2:1.0 eth1: kevent 12 may have been dropped
<3>[ 27.219290] cdc_ether 1-2:1.0 eth1: kevent 11 may have been dropped
<3>[ 27.231946] cdc_ether 1-2:1.0 eth1: kevent 11 may have been dropped
<6>[ 28.072396] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
<6>[ 30.619737] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
<6>[ 30.679172] br-lan: port 2(wlan0) entered blocking state
<6>[ 30.689878] br-lan: port 2(wlan0) entered disabled state
<6>[ 30.701117] device wlan0 entered promiscuous mode
<6>[ 31.159880] IPv6: ADDRCONF(NETDEV_UP): wlan1: link is not ready
<6>[ 31.199026] br-lan: port 3(wlan1) entered blocking state
<6>[ 31.209719] br-lan: port 3(wlan1) entered disabled state
<6>[ 31.220946] device wlan1 entered promiscuous mode
<6>[ 31.230979] br-lan: port 3(wlan1) entered blocking state
<6>[ 31.241677] br-lan: port 3(wlan1) entered forwarding state
<6>[ 31.673212] br-lan: port 3(wlan1) entered disabled state
<6>[ 32.092160] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
<6>[ 32.105236] br-lan: port 2(wlan0) entered blocking state
<6>[ 32.115898] br-lan: port 2(wlan0) entered forwarding state
<6>[ 33.393752] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
<6>[ 33.406921] br-lan: port 3(wlan1) entered blocking state
<6>[ 33.417611] br-lan: port 3(wlan1) entered forwarding state
<6>[ 80.041853] device eth1 entered promiscuous mode
<6>[ 207.237912] device eth1 left promiscuous mode
<4>[89224.524019] conntrack: generic helper won't handle protocol 47. Please consider loading the specific helper module.
<1>[144203.969138] CPU 0 Unable to handle kernel paging request at virtual address 00dba0be, epc == 86f7c384, ra == 86f7c3ac
<4>[144203.990468] Oops[#1]:
<4>[144203.995154] CPU: 0 PID: 5871 Comm: kworker/0:1 Not tainted 4.14.195 #0
<4>[144204.008318] Workqueue: events_power_efficient 0x86f50304 [xt_FLOWOFFLOAD@86f50000+0xc10]
<4>[144204.024582] task: 87d6d940 task.stack: 8613a000
<4>[144204.033758] $ 0 : 00000000 00000001 00dba090 00000001
<4>[144204.044333] $ 4 : 00dba090 8613be4f 8613be4f ffff00fe
<4>[144204.054903] $ 8 : 8613bfe0 00008400 00008327 000fcc00
<4>[144204.065470] $12 : 00000000 00000000 fe649300 00008326
<4>[144204.076039] $16 : 8613be4f 86f50000 00000000 fffffff5
<4>[144204.086608] $20 : 8056bba0 00000001 00000000 fffffffe
<4>[144204.097175] $24 : 3b9aca00 80008e34
<4>[144204.107743] $28 : 8613a000 8613bdf0 80550000 86f7c3ac
<4>[144204.118311] Hi : 0000000a
<4>[144204.124198] Lo : 66666669
<4>[144204.130101] epc : 86f7c384 0x86f7c384 [nf_flow_table@86f7c000+0x3350]
<4>[144204.143435] ra : 86f7c3ac 0x86f7c3ac [nf_flow_table@86f7c000+0x3350]
<4>[144204.156756] Status: 11008403 KERNEL EXL IE
<4>[144204.165249] Cause : 40800008 (ExcCode 02)
<4>[144204.173385] BadVA : 00dba0be
<4>[144204.179273] PrId : 0001992f (MIPS 1004Kc)
<4>[144204.187579] Modules linked in: pppoe ppp_async pppox ppp_generic nf_conntrack_netlink nf_conntrack_ipv6 mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 cdc_ether xt_time xt_tcpudp xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_helper xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_IPMARK xt_FLOWOFFLOAD xt_CT wireguard usbnet slhc nfnetlink nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_filter ip_tables crc_ccitt compat_xtables compat ledtrig_usbport nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter
<4>[144204.330023] ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ip6_udp_tunnel udp_tunnel sit tunnel4 ip_tunnel mii leds_gpio xhci_plat_hcd xhci_pci xhci_mtk xhci_hcd gpio_button_hotplug usbcore nls_base usb_common
<4>[144204.367438] Process kworker/0:1 (pid: 5871, threadinfo=8613a000, task=87d6d940, tls=00000000)
<4>[144204.384560] Stack : 80550000 00008327 805a8da0 8054a1e8 86f50b80 00dba090 00000019 00000100
<4>[144204.401363] 00000200 860d5800 0000000f 00000001 80550000 00000000 86f50b70 86f50000
<4>[144204.418162] 86f50000 86f50374 80549d00 8049eac0 00000001 87d6d940 81116a20 01119800
<4>[144204.434962] 00000000 86f50b40 8645db00 81116a20 81119c00 00000000 00000000 80045e48
<4>[144204.451765] 81116bc0 81116a38 80550000 81116bc0 80550000 fffffffe 8645db00 81116a20
<4>[144204.468571] ...
<4>[144204.473599] Call Trace:
<4>[144204.473626] [<86f50000>] 0x86f50000 [xt_FLOWOFFLOAD@86f50000+0xc10]
<4>[144204.491276] [<86f50000>] 0x86f50000 [xt_FLOWOFFLOAD@86f50000+0xc10]
<4>[144204.503910] [<86f50374>] 0x86f50374 [xt_FLOWOFFLOAD@86f50000+0xc10]
<4>[144204.516540] [<8049eac0>] 0x8049eac0
<4>[144204.523660] [<80045e48>] 0x80045e48
<4>[144204.530761] [<80046340>] 0x80046340
<4>[144204.537859] [<80067870>] 0x80067870
<4>[144204.544965] [<80045ff0>] 0x80045ff0
<4>[144204.552081] [<8004bf38>] 0x8004bf38
<4>[144204.559182] [<8004be08>] 0x8004be08
<4>[144204.566283] [<8004be08>] 0x8004be08
<4>[144204.573386] [<8004be08>] 0x8004be08
<4>[144204.580486] [<80006f78>] 0x80006f78
<4>[144204.587591]
<4>[144204.590712] Code: 00409025 1000000f 00000000 <9042002e> 14400006 00000000 0220f809 00000000 10000002
<4>[144204.610286]
<4>[144204.614155] ---[ end trace ae96bb8b777fc719 ]---
<1>[144204.620443] CPU 1 Unable to handle kernel paging request at virtual address 00dba14e, epc == 86f7c8d4, ra == 86f7cc34
==========================
