Mtk_soc_eth watchdog timeout after r11573

saub · April 8, 2020, 3:16pm

Netgear R6220 - tested on on OpenWRT 19.07.1 and OpenWRT 19.07.2 have problems with

Memory leak in kernel
mtk_soc_eth 1e100000.ethernet eth0: transmit timed out
interrupts errors
Occasionally restarts

All problems gone (so far) after switch to the original firmware:

# cat /proc/interrupts
           CPU0       CPU1
  3:    4761112          0        MIPS GIC  eth2
  4:        188    7484398        MIPS GIC  rai0
  7:   15364666   15363980        MIPS GIC  timer
 12:          0          0        MIPS GIC  ralink_gpio
 18:          0          0        MIPS GIC  Ralink_SPDIF
 22:   61482632          0        MIPS GIC  xhci-hcd:usb1
 23:         12          0        MIPS GIC  Ralink_ESW
 25:        798    5997703        MIPS GIC  ra0
 26:     422129          0        MIPS GIC  serial
 56:     262949          0        MIPS GIC  IPI_resched
 57:          0     308253        MIPS GIC  IPI_resched
 58:          0          0        MIPS GIC  IPI_resched
 59:          0          0        MIPS GIC  IPI_resched
 60:    2333758          0        MIPS GIC  IPI_call
 61:          0     966205        MIPS GIC  IPI_call
 62:          0          0        MIPS GIC  IPI_call
 63:          0          0        MIPS GIC  IPI_call

ERR:          0

# free
              total         used         free       shared      buffers
  Mem:       122308        50656        71652            0         2764
 Swap:            0            0            0
Total:       122308        50656        71652
#

The original firmware is partially open - I added to it Dropbear and a few other things.

So, now I am sure this is 100% OpenWRT software problem - it's a sad conclusion.

Regards,
Samuel

apocalypse · April 8, 2020, 3:43pm

In my case no memory leak, no transmit timed outs or interrupt errors (with fc off, interrupt handling and mt7530_fix patches). But yes, random restarts that can occur within 24 hours or 10 days. I am now testing a build without interrupt handling patch and with fc off on all ports.

https://github.com/openwrt/openwrt/pull/2815#issuecomment-602978547
https://github.com/openwrt/openwrt/pull/2847 (also with disable EEE).

saub · April 8, 2020, 4:42pm

Yep but it crash during boot up on R6220

dchard · April 18, 2020, 3:54pm

@neheb

Out of curiosity, does Blogic have another public repo other than the one at git.openwrt.org, where it is possible to take a look at his HW offload work for 7621? If there is one, would be nice to take a look at

neheb · April 18, 2020, 7:13pm

He has a GitHub account which is fairly inactive. Nothing of the sort available anywhere though.

apocalypse · April 18, 2020, 8:05pm

Well, with fc off on all ports + mt7530_fix its running for now Uptime 11d 12h 51m 37s but with some errors.

First occurred at 4 hours of uptime. I've never seen it before:

Tue Apr  7 12:54:50 2020 kern.alert kernel: [14076.077097] BUG: Bad page state in process swapper/0  pfn:0f22f
Tue Apr  7 12:54:50 2020 kern.emerg kernel: [14076.088918] page:811e75e0 count:0 mapcount:0 mapping:  (null) index:0x0
Tue Apr  7 12:54:50 2020 kern.emerg kernel: [14076.102114] flags: 0x0()
Tue Apr  7 12:54:50 2020 kern.alert kernel: [14076.107169] raw: 00000000 00000000 00000000 ffffffff 00000000 00000000 811e75f4 00000000
Tue Apr  7 12:54:50 2020 kern.alert kernel: [14076.123258] page dumped because: non-NULL mapping
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.132605] Modules linked in: pppoe ppp_async pppox ppp_generic nf_nat_pptp nf_conntrack_pptp nf_conntrack_ipv6 iptable_nat ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY ts_fsm ts_bm slhc nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_rtsp nf_nat_redirect nf_nat_proto_gre nf_nat_masquerade_ipv4 nf_nat_irc nf_conntrack_ipv4 nf_nat_ipv4 nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtsp nf_conntrack_rtcache
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.276369]  nf_conntrack_proto_gre nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast ts_kmp nf_conntrack_amanda nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 tun nls_utf8 nls_iso8859_15 nls_cp852 nls_cp850 nls_cp437 nls_base leds_gpio gpio_button_hotplug
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.350330] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.167 #0
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.362445] Stack : 00000000 00000008 805a4380 8007265c 805a0000 80546510 00000000 00000000
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.379091]         80512100 8fc09c5c 80580d5c 805808e7 8050cef0 00000001 8fc09c00 53261646
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.395735]         00000000 00000000 806e0000 00004598 00000000 000000ee 00000008 00000000
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.412379]         00000000 80580000 0005587a 00000000 00000000 805a0000 00000000 80710000
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.429023]         805173f4 80580000 00000003 00000008 00000000 80299210 00000000 806e0000
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.445668]         ...
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.450530] Call Trace:
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.455416] [<8000c7b0>] show_stack+0x58/0x100
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.464279] [<8044f9d4>] dump_stack+0xa4/0xe0
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.472961] [<800cb6d4>] bad_page+0x110/0x148
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.481634] [<800ce464>] get_page_from_freelist+0x534/0x8e4
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.492722] [<800ceeb0>] __alloc_pages_nodemask+0x120/0xd0c
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.503810] [<800cfbe8>] page_frag_alloc+0x54/0x170
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.513535] [<803062d0>] fe_poll+0x340/0x800
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.522057] [<80349b2c>] net_rx_action+0x150/0x30c
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.531613] [<8046d870>] __do_softirq+0x128/0x2ec
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.540987] [<80033f84>] irq_exit+0xac/0xc8
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.549327] [<8024c1c0>] plat_irq_dispatch+0xfc/0x138
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.559377] [<80007588>] except_vec_vi_end+0xb8/0xc4
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.569253] [<80008f50>] r4k_wait_irqoff+0x1c/0x24
Tue Apr  7 12:54:50 2020 kern.warn kernel: [14076.578787] Disabling lock debugging due to kernel taint

Second at 4 days 12 hours:

Sat Apr 11 21:01:56 2020 kern.warn kernel: [388902.689722] ------------[ cut here ]------------
Sat Apr 11 21:01:56 2020 kern.warn kernel: [388902.699139] WARNING: CPU: 2 PID: 0 at net/sched/sch_generic.c:320 dev_watchdog+0x1ac/0x324
Sat Apr 11 21:01:56 2020 kern.info kernel: [388902.715789] NETDEV WATCHDOG: eth0 (mtk_soc_eth): transmit queue 0 timed out
Sat Apr 11 21:01:56 2020 kern.warn kernel: [388902.729875] Modules linked in: pppoe ppp_async pppox ppp_generic nf_nat_pptp nf_conntrack_pptp nf_conntrack_ipv6 iptable_nat ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY ts_fsm ts_bm slhc nf_reject_ipv4 nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_rtsp nf_nat_redirect nf_nat_proto_gre nf_nat_masquerade_ipv4 nf_nat_irc nf_conntrack_ipv4 nf_nat_ipv4 nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_rtsp nf_conntrack_rtcache
Sat Apr 11 21:01:56 2020 kern.warn kernel: [388902.873962]  nf_conntrack_proto_gre nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast ts_kmp nf_conntrack_amanda nf_conntrack iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 tun nls_utf8 nls_iso8859_15 nls_cp852 nls_cp850 nls_cp437 nls_base leds_gpio gpio_button_hotplug
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388902.948252] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G    B           4.14.167 #0
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388902.962982] Stack : 00000000 8fea1740 80580000 8007265c 805a0000 80546510 00000000 00000000
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388902.979811]         80512100 8fc0ddc4 8fc3c99c 805808e7 8050cef0 00000001 8fc0dd68 53261646
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388902.996617]         00000000 00000000 806e0000 00005ca8 00000000 00000121 00000008 00000000
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.013418]         00000000 80580000 000e781c 20202020 00000000 805a0000 00000000 80540718
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.030221]         80370160 00000140 00000002 8fea1740 00000003 80299210 00000008 806e0008
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.047026]         ...
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.052058] Call Trace:
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.057113] [<8000c7b0>] show_stack+0x58/0x100
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.066145] [<8044f9d4>] dump_stack+0xa4/0xe0
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.074988] [<8002f5f8>] __warn+0xe0/0x138
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.083306] [<8002f680>] warn_slowpath_fmt+0x30/0x3c
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.093361] [<80370160>] dev_watchdog+0x1ac/0x324
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.102910] [<8008932c>] call_timer_fn.isra.25+0x24/0x84
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.113646] [<800895e8>] run_timer_softirq+0x1bc/0x248
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.124060] [<8046d870>] __do_softirq+0x128/0x2ec
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.133592] [<80033f84>] irq_exit+0xac/0xc8
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.142102] [<8024c1c0>] plat_irq_dispatch+0xfc/0x138
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.152319] [<80007588>] except_vec_vi_end+0xb8/0xc4
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.162365] [<80008f50>] r4k_wait_irqoff+0x1c/0x24
Sat Apr 11 21:01:57 2020 kern.warn kernel: [388903.172213] ---[ end trace 3f1f5eb5e3775b79 ]---
Sat Apr 11 21:01:57 2020 kern.err kernel: [388903.181624] mtk_soc_eth 1e100000.ethernet eth0: transmit timed out
Sat Apr 11 21:01:57 2020 kern.info kernel: [388903.194180] mtk_soc_eth 1e100000.ethernet eth0: dma_cfg:80000065
Sat Apr 11 21:01:57 2020 kern.info kernel: [388903.206397] mtk_soc_eth 1e100000.ethernet eth0: tx_ring=0, base=0ec80000, max=0, ctx=2588, dtx=2588, fdx=2587, next=2588
Sat Apr 11 21:01:57 2020 kern.info kernel: [388903.228287] mtk_soc_eth 1e100000.ethernet eth0: rx_ring=0, base=0e010000, max=0, calc=3057, drx=3075
Sat Apr 11 21:01:57 2020 kern.info kernel: [388903.272721] mtk_soc_eth 1e100000.ethernet: 0x100 = 0x5a60000c, 0x10c = 0x80818
Sat Apr 11 21:01:57 2020 kern.info kernel: [388903.293259] mtk_soc_eth 1e100000.ethernet: PPE started

Third at 7 days 20 hours:

Wed Apr 15 04:28:13 2020 kern.err kernel: [674879.413092] mtk_soc_eth 1e100000.ethernet eth0: transmit timed out
Wed Apr 15 04:28:13 2020 kern.info kernel: [674879.425618] mtk_soc_eth 1e100000.ethernet eth0: dma_cfg:80000065
Wed Apr 15 04:28:13 2020 kern.info kernel: [674879.437794] mtk_soc_eth 1e100000.ethernet eth0: tx_ring=0, base=0d2f0000, max=0, ctx=2632, dtx=2632, fdx=2631, next=2632
Wed Apr 15 04:28:13 2020 kern.info kernel: [674879.459710] mtk_soc_eth 1e100000.ethernet eth0: rx_ring=0, base=0c0f0000, max=0, calc=1171, drx=1172
Wed Apr 15 04:28:13 2020 kern.info kernel: [674879.481384] mtk_soc_eth 1e100000.ethernet: 0x100 = 0x5960000c, 0x10c = 0x80818
Wed Apr 15 04:28:13 2020 kern.info kernel: [674879.504536] mtk_soc_eth 1e100000.ethernet: PPE started

And last was today:

Sat Apr 18 07:23:23 2020 kern.err kernel: [944589.146024] mtk_soc_eth 1e100000.ethernet eth0: transmit timed out
Sat Apr 18 07:23:23 2020 kern.info kernel: [944589.158525] mtk_soc_eth 1e100000.ethernet eth0: dma_cfg:80000065
Sat Apr 18 07:23:23 2020 kern.info kernel: [944589.170688] mtk_soc_eth 1e100000.ethernet eth0: tx_ring=0, base=0e3b0000, max=0, ctx=2598, dtx=2598, fdx=2597, next=2598
Sat Apr 18 07:23:23 2020 kern.info kernel: [944589.192515] mtk_soc_eth 1e100000.ethernet eth0: rx_ring=0, base=0c6e0000, max=0, calc=1132, drx=1133
Sat Apr 18 07:23:23 2020 kern.info kernel: [944589.214739] mtk_soc_eth 1e100000.ethernet: 0x100 = 0x6060000c, 0x10c = 0x80818
Sat Apr 18 07:23:23 2020 kern.info kernel: [944589.234975] mtk_soc_eth 1e100000.ethernet: PPE started

cwbsw · April 19, 2020, 5:22am

Has this issue been fixed completely in 5.4 kernel?

Enig123 · April 19, 2020, 7:07am

Now I am running yesterday's master branch on DIR-860L B1 without a den, if without offloading stuff. Even the 5G WiFi dropout sometimes is gone.

So far it's quite stable I'd say. Now start holding my breath for the hw offloading part.

Edit: The other things that I spotted is, most of the interrupt ERRs are from 5G Hz WiFi, since the increase is not that much on 2.4 G Hz usage only.

dchard · April 19, 2020, 10:44am

I am using mine on kernel 5.4.31 for 8 days, not a single error, drop or reboot. Some more patches will likely arrive, but so far it is much better than the 4.14 branch.

@apocalypse

Why are you not trying out the 5.4 branch? Most of us have reasonably good results with it.

Bartvz · April 19, 2020, 2:21pm

So master builds work again for the dir-860l? If so, it's time to fire up my buildmachine again

dchard · April 19, 2020, 3:36pm

Yes, it does work.

neheb · April 19, 2020, 8:55pm

With some newer mt7615 devices, it seems there are pcie issues: https://github.com/openwrt/openwrt/pull/2798#issuecomment-616129243

I assume it'll get fixed in time

Good to hear that everyone here is having good results.

JonFo · April 22, 2020, 10:13pm

How bad is the Interupt error rate now on the 5.4 masters?

It used to be >5000 per day for me on the 2020/3/13 master. Mostly driven by 5GHz activity.

Also I'm seeing these regularly on that older bild, and want to hear if this is fixed.
[139643.589897] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00 [139643.589917] mt76x2e 0000:01:00.0: Build: 1 [139643.589925] mt76x2e 0000:01:00.0: Build Time: 201507311614____ [139643.608589] mt76x2e 0000:01:00.0: Firmware running!

Not so worried about max throughput as I am about having a build that is stable and usable.

Mushoz · April 23, 2020, 7:10am

So I am using two mt7621 devices in production. One of them was rock solid stable, and the other one was randomly hanging or restarting. There is one big difference with regards to the VLAN configuration, which I suspected might be a workaround for this issue. To test this out, I used the same configuration on my problematic device and lo and behold, it seems to be stable.

The important thing to notice is that both devices are using 19.07.1 and both devices are still getting a watchdog timeout error in the kernel log. However, it does not seem to affect functionality at all. Connections never hang and the device doesn't randomly reboot. First device that has always been stable:

Screenshot from 2020-04-23 09-03-42

Second device that used to crash within a couple of days:

Screenshot from 2020-04-23 09-03-53

So what is this workaround I am talking about? The issue seems to be with the switch driver. By using different VLANs on ALL my ports, and bridging them together, I am effectively not using the driver to switch packets. And hence my theory is that the bug seems to be in the switch logic, so not using it seems to prevent the bug from occurring. This is what my switch configuration looks like:

All those individuals ports (VLAN 11, 12, 13 and 14) are bridged together in a Linux software bridge (the default br-lan bridge). For completeness sake (although I don't think it matters), IGMP snooping is enabled on the bridge.

Maybe this workaround also works for other people that are still on current "stable" builds instead of bleeding edge kernel 5.4 builds

apocalypse · April 23, 2020, 5:26pm

Great, one user already commented long ago that having each port in a separate VLAN had no transmit timed out or reboots. But I thought it was because he got a "good" SoC. I will prove it. At the moment I am using the mt7530_fix patch with fcoff and although i have transmit timed outs every 2 days, it seems that it does not restart or hang. Are you using any patch or stable version as it comes? Is there any degradation in performance between local transfers on the ports of the software bridge or use more cpu?

I still don't want to switch to kernel 5.4 for "complexity" to configure VLANs in the DSA driver.

Mushoz · April 24, 2020, 7:22am

Unfortunately, it seems like I jinxed it. The connected was down and the router unreachable for ~20 minutes. The issue cleared itself up eventually, but at the time of the issue the dreaded error was found in the kernel log.

@dchard and others that are testing a build with the 5.4 kernel, is this problem fixed on the 5.4 kernel? How is overall stability on these bleeding edge builds? Is it worth using over a 19.07.2 build on production hardware? Or should I bite the bullet and abandon this platform?

Bartvz · April 24, 2020, 9:53am

My mir3g has an uptime of 3 days with r13042 + my modifications and I haven't run into the problem.
However, I don't have a very fancy setup. Just an USB 4GLTE dongle and atm 3-4 WiFi clients connected to 5 GHz.

shevalier · April 24, 2020, 10:09am

I think, its not Mediatek specific problem.

When I had uplink 80mbit @ 100mbit port, I got no problem.
Now, 100@100 whith FC on at provider switch, I get "transmit queue 0 timed out" on all OWRT device, including MT7621 (Mi3G)

For example, after 3-10 minuts at torrent whith 100 mbit download
tl-wr741nd ver 2.4 target ar71xx/ath79

[ 713.044509] ------------[ cut here ]------------
[ 713.049263] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:306 0x802971f8()


[ 713.056534]NETDEV WATCHDOG: eth0 (ag71xx): transmit queue 0 timed out


[ 713.063096] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath pppox ppp_generic nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 mac80211 ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables crc_ccitt compat ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables gpio_button_hotplug
[ 713.120963] CPU: 0 PID: 0 Comm: swapper Not tainted 4.4.153 #0
[ 713.126828] Stack : 803938d8 00000000 00000001 803f0000 00000000 00000000 00000000 00000000
[ 713.126828] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 713.126828] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 713.126828] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 713.126828] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 713.126828] ...
[ 713.162738] Call Trace:[<80071c44>] 0x80071c44
[ 713.167279] [<80071c44>] 0x80071c44
[ 713.170783] [<80081ac8>] 0x80081ac8
[ 713.174289] [<802971f8>] 0x802971f8
[ 713.177811] [<80081b24>] 0x80081b24
[ 713.181320] [<800b0e54>] 0x800b0e54
[ 713.184843] [<802971f8>] 0x802971f8
[ 713.188342] [<8024c57c>] 0x8024c57c
[ 713.191847] [<8029701c>] 0x8029701c
[ 713.195367] [<800b0e54>] 0x800b0e54
[ 713.198871] [<800b10d8>] 0x800b10d8
[ 713.202370] [<800a8fd0>] 0x800a8fd0
[ 713.205894] [<80084054>] 0x80084054
[ 713.209396] [<800ac3ac>] 0x800ac3ac
[ 713.212896] [<800a88a4>] 0x800a88a4
[ 713.216418] [<8006aa50>] 0x8006aa50
[ 713.219926] [<80060bf8>] 0x80060bf8
[ 713.223417]
[ 713.224926] ---[ end trace c952f5c8864e6bd0 ]---
[ 713.229554] eth0: tx timeout
[ 823.044261] eth0: tx timeout
[ 913.043891] eth0: tx timeout
[ 983.859962] device wlan0 left promiscuous mode

mediatek

[883282.702056] WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:320 0x8038b620

[883282.709182] NETDEV WATCHDOG: eth0 (mtk_soc_eth): transmit queue 0 timed out

[883282.716192] Modules linked in: pppoe ppp_async pppox ppp_generic nf_conntrack_ipv6 mt76x2e mt76x2_common mt76x02_lib mt7603e mt76 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables crc_ccitt compat sg ledtrig_usbport nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 msdos vfat fat autofs4 nls_utf8 nls_koi8_r nls_iso8859_1 nls_cp866 nls_cp852 nls_cp850 nls_cp437
[883282.787064]  nls_cp1251 nls_cp1250 uas usb_storage sd_mod scsi_mod ext4 mbcache jbd2 crc32c_generic leds_gpio xhci_plat_hcd xhci_pci xhci_mtk xhci_hcd gpio_button_hotplug usbcore nls_base usb_common
[883282.804792] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.171 #0

Mushoz · April 24, 2020, 10:38am

It's such a shame flow control is giving us so many issues, despite flow control being super stupid even if it was working properly. Nice comment on why you should hate flow control: https://www.reddit.com/r/networking/comments/1vosfv/flow_control/

apocalypse · April 24, 2020, 2:11pm

Unfortunately it has NOT been fixed in kernel 5.4: ER-X-SFP: VLANs not working properly with kernel 5.4