Intel I210AT network card caused Tx Unit Hang error on x86

Short story: Intel i210 network card has frequent "Tx Unit Hang" error caused my openWrt cannot connect with openWRT x86, while it works perfectly on Windows Server.
I found a few topics about "Tx Unit Hang" but they are all about e1000e network card, and they don't solve my problem on Intel i210 network card.

Detailed story:
Tried many openwrt versions from
openwrt-18.06.1-x86-64-combined-ext4.img
to
openwrt-19.07.5-x86-64-combined-squashfs.img

  • A DIY Fanless Mini PC: CPU: i3 4030u, 8GB DDR3, SSD on mini-PCIe, HDD on SATA
  • Onboard RTL 8111e network card connect to PCI bus
  • Intel WGI 211 AT on mini-PCIe port.

I flashed the i211AT to i210AT by using eeupdate - they are same hardware but different firmware. This step allows windows server install driver for it. I have no i211AT firmware to rollback.

The system works well in Windows Server 2019. I installed OpenWRT over Hyper V, Everything works perfectly over the hyper-v virtual network cards.

I want to optimize performance by using PCI passthrough and the Intel i210AT doesn't work.

So I created a USB flash drive with OpenWRT, directly boot from the flash drive. I disabled SSD and HDD.
RTL8111e network card works fine but Intel i211AT shows the error message "igb 0000:03:00.0: Detected Tx Unit Hang" every a few seconds.
My client PC may connect to this port, even assigned with an IP from DHCP for a very short period and it disconnects due to this error:

Thu Jan 14 09:16:10 2021 user.notice mwan3[31625]: Execute ifdown event on interface wan (unknown)
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454] igb 0000:03:00.0: Detected Tx Unit Hang
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   Tx Queue             <2>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   TDH                  <0>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   TDT                  <1>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   next_to_use          <1>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   next_to_clean        <0>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454] buffer_info[next_to_clean]
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   time_stamp           <100095171>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   next_to_watch        <00000000c2bf44b8>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   jiffies              <1000953c0>
Thu Jan 14 09:16:11 2021 kern.err kernel: [ 2745.165454]   desc.status          <2a8000>
Thu Jan 14 09:16:12 2021 daemon.info netdata[7996]: RRDSET: chart name 'net.pppoe_wan' on host 'OpenWrt' already exists.
Thu Jan 14 09:16:12 2021 daemon.err netdata[7996]: PROCFILE: Cannot open file '/proc/sysvipc/shm'
Thu Jan 14 09:16:14 2021 daemon.notice netifd: Network device 'eth0' link is down
Thu Jan 14 09:16:14 2021 daemon.notice netifd: Interface 'wan6' has link connectivity loss
Thu Jan 14 09:16:14 2021 daemon.notice netifd: Interface 'wan' has link connectivity loss
Thu Jan 14 09:16:14 2021 kern.err kernel: [ 2747.980212] igb 0000:03:00.0 eth0: Reset adapter
Thu Jan 14 09:16:14 2021 daemon.info pppd[31660]: Terminating on signal 15
Thu Jan 14 09:16:14 2021 daemon.err netdata[7996]: PROCFILE: Cannot open file '/proc/sysvipc/shm'
Thu Jan 14 09:16:14 2021 daemon.notice netifd: Interface 'wan6' is now down
Thu Jan 14 09:16:16 2021 daemon.err netdata[7996]: PROCFILE: Cannot open file '/proc/sysvipc/shm'
Thu Jan 14 09:16:17 2021 kern.info kernel: [ 2751.500653] igb 0000:03:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Thu Jan 14 09:16:17 2021 daemon.notice netifd: Network device 'eth0' link is up
Thu Jan 14 09:16:17 2021 daemon.notice netifd: Interface 'wan6' has link connectivity
Thu Jan 14 09:16:17 2021 daemon.notice netifd: Interface 'wan6' is setting up now
Thu Jan 14 09:16:17 2021 daemon.notice netifd: Interface 'wan' has link connectivity
Thu Jan 14 09:16:18 2021 daemon.err netdata[7996]: PROCFILE: Cannot open file '/proc/sysvipc/shm'
Thu Jan 14 09:16:19 2021 daemon.notice netifd: Interface 'wan' is now down
Thu Jan 14 09:16:19 2021 daemon.notice netifd: Interface 'wan' is setting up now
Thu Jan 14 09:16:19 2021 daemon.err insmod: module is already loaded - slhc
Thu Jan 14 09:16:19 2021 daemon.err insmod: module is already loaded - ppp_generic
Thu Jan 14 09:16:19 2021 daemon.err insmod: module is already loaded - pppox
Thu Jan 14 09:16:19 2021 daemon.err insmod: module is already loaded - pppoe
Thu Jan 14 09:16:19 2021 daemon.info pppd[32333]: Plugin rp-pppoe.so loaded.
Thu Jan 14 09:16:19 2021 daemon.info pppd[32333]: RP-PPPoE plugin version 3.8p compiled against pppd 2.4.8
Thu Jan 14 09:16:19 2021 daemon.notice pppd[32333]: pppd 2.4.8 started by root, uid 0
Thu Jan 14 09:16:19 2021 user.notice mwan3[32283]: Execute ifdown event on interface wan (unknown)
Thu Jan 14 09:16:20 2021 daemon.err netdata[7996]: PROCFILE: Cannot open file '/proc/sysvipc/shm'
Thu Jan 14 09:16:20 2021 kern.err kernel: [ 2754.125442] igb 0000:03:00.0: Detected Tx Unit Hang

I found some posts related to this error but they are all about 'e1000e' driver network card such as 82566 , not 'igb' driver like i210AT.
but still follow the instructions on those posts: installed ethtool via opkg and tried following commands:

ethtool -K eth0 tx off rx off
ethtool -K eth0 gso off gro off tso off

No improvements. Some posts suggested change e1000e firmware using ethtool but i210AT has different data in the firmware. The bit is unset on i210AT so no need to change.
https://downloadmirror.intel.com/15817/eng/README.txt

I also tried different version of openwrt from 2018 to the latest, official and unofficial...

Please advise what I can do to solve this issue.

In my past experiences with both e1000 and e1000e with this continuing menace (using general purpose linux distributions), five bucks for a PCIe based r8168/ r8169 realtek card were well worth it.

Agree - the Realtek 8111e onboard network card on the same computer works fine.

I also bought another mPCIe Realtek network card. but it is too big to fit my mini PC.
This Intel i211 AT card is the smallest mpcie card can fit my fanless mini PC. :frowning:
I heard that Intel network card has better compatibility and lower CPU usage. maybe this is only true for Windows, not for Linux world.

After changing another i210AT card, it works when directly booting from USB drive. but it still has the same error when booting from win server 2019 Hyper-V + PCI passthrough to OpenWrt.

I did a search and only found a similar issue was reported on Ubuntu + Hyper V. Not sure if it is related: