MT7621 - 21.02/Master feedback firmware image test - IPV6 offload and disabled Flow Control

For Developers
1

Hi MT7621 owners,

As you know, having owned an mt7621 router there is an annoying bug that plagues us all.

the netdev timeout issue! - this issue is related to the way the flow control is forced on the mt7530 switch and because its connected to the mt7621, there is a weird bug that triggers miss timing causing the mt7621 cpu to fall over and crash.

I've played around with several of my mt7621 routers and noticed all of them will have this issue when the flow control is forced for RX/TX - by default this mode is forced on all of the ports. If you don't have a switch that can be told to switch it off, then you pretty much going to have this issue.

From the router shell, confirm feature no longer there:

To confirm flow control is not there anymore.

ethtool -a eth0
Pause parameters for eth0:
Cannot get device pause settings: Not supported

My repo i've done a disable patch for 21.02-snapshot and master (snapshot) branch:

disable flow control initial patch - gitlab
disable flow control soc mt7621 only improvement - gitlab

Also added IPV6 HW flow offload support (routing only) - 21.02 and master (snapshot) branch:
enable ipv6 hw offload - gitlab - now in master (snapshot)

I've precompiled for all of the supported mt7621 devices and uploaded to google drive:
Download firmware image - MT7621

I've included some useful tools in the 21.02 image:

  • Iperf3 - test the network bandwidth
  • speedtest - test you internet speed
  • tcpdump - nano and uboot-envtools
  • ethtool - to see the port status

REMEMBER this is for testing purposes and won't be held responsible for bricked devices!

I've tested on - Xiaomi MI4 AG Gigabit model and Unielec u7621-01 model

You will notice the ports connecting and disconnecting several times due to the other end trying to force FC for RX/TX. It will settle and stay on the port speed and no flow control.

Once there is enough feedback, then we can improve on the patch and then get it pushed upstream to the official openwrt repo.

PLEASE feedback any issues and suggestions?

Also if you have a better idea for the patches then please, do a pull request on my repo.

Thanks

14 Likes
2

Thank you for sharing your findings and patch!

BTW, recently OpenWrt switched ramips to kernel 5.10. Will your patch work with the new kernel or only with 5.4?

3

Hi,
Currently it targeted for kernel 5.4 only. And at the moment testing feedback stage.

Work can be done for the 5.10 on the master branch.

So a direct patching will come up with errors. Im currently only working on the stable versions snapshot branch for feedback as this patch is very generic and needs improving.

Just need feedback to see if anyone who has had a constant netdev timeout issue to feedback any improvement.

3 Likes
4

I have the Archer C6 v3.2 and I'm doing my own builds from master.

How can I check this issue? I've noticed the errors below which happen occasionally and I'm wondering that they may be related to the issue you fixed.

(...)
 4366.197781] mt7530 mdio-bus:1f lan2: Link is Down
[ 4366.202605] br-lan: port 2(lan2) entered disabled state
[ 4372.437817] mt7530 mdio-bus:1f lan2: Link is Up - 1Gbps/Full - flow control rx/tx
[ 4372.445328] br-lan: port 2(lan2) entered blocking state
[ 4372.450586] br-lan: port 2(lan2) entered forwarding state
[ 4374.517613] mt7530 mdio-bus:1f lan2: Link is Down
[ 4374.522439] br-lan: port 2(lan2) entered disabled state
[ 4377.637707] mt7530 mdio-bus:1f lan2: Link is Up - 1Gbps/Full - flow control rx/tx
[ 4377.645219] br-lan: port 2(lan2) entered blocking state
[ 4377.650492] br-lan: port 2(lan2) entered forwarding state
5

That indicates the other end your router is connecting to, is insisting on having flow control on.
So what issues have you been having?

6

The WAN interface occasionally goes down and up. It's very sporadic and just for 2-3 seconds, so far I haven't tried to debug it (it is connected to my ISP cable modem).

7

Tested on Xiaomi R3G v1.
IPv6 offloading work correctly.
0% usage with ipv4 and ipv6.
0 error on logread.

I never have problem on this router with RX/TX with or without this patch.

It's possible test ipv6 offload on master with kernel 5.10? It's needed amend patch?

8

Thanks for the feedback!

So triggering the netdev timeout will require the other end (the port the router is connecting to) to have flow control on by default - its the only way i could trigger this issue fully.

Glad to see the ipv6 is working as it should.

I will take a look at the master branch for this patch to be included...

9

What command i need use on my linux computer for enable flow control? I use Linux Mint 20.2.
I suspect that is with ethtool. My interface is named eno0.

10

In 21.02 will show this on the router:
To confirm flow control is not there anymore.

ethtool -a eth0
Pause parameters for eth0:
Cannot get device pause settings: Not supported

On your pc just change to the correct interface name - en0

ethtool -a en0

11

image
image778×628 83.4 KB

I suspect my driver of card not support ...

12

Usually switches have a flow control policy - unmanaged switches just do an Auto and managed switches can control the flow control setting.

PC cards depend on model and manufacturer.

If your router is plugged directly into the router then reason you probably have no issues, if you had a more complex setup, including switches, this is where the issue arises for some people.

1 Like
13

OKEY. I have on simple setup.
ISP(bridge mode) -> Xiaomi R3G v1.

This explain because i don't have this problem.

14

@db260179 thanks for your work!
Does your flow control patch fix this bug?

1 Like
15

I dont think it will. This auto neg issue is to do with the way the dsa model of driver is attempting port negotiation. I've seen forks of openwrt 21.02 go back to the old driver due to this issue.

1 Like
16

Great job! I have a Mi Router 4A Gigabit and the WAN port disconnects and connects several times at day...

Testing disable flow control patch with my own master builds.

I will feedback.

Thanks!

1 Like
17

@db260179, stable: 2 days and 20 hours :slight_smile:
image

image
image817×408 12.6 KB

1 Like
18

Look extremely promising, care to share the firmware ?
I will also test and provide logs, also with a 1gb down and 700 up I will surely hit any possible snag if there is any left :slight_smile:

Best regards, and once again thank you all for the outstanding work

19

Ive done a pre build of master with these patches

https://drive.google.com/drive/u/0/mobile/folders/1MAbfyk_OhENHnbK-fM4K_rp2puA3E2L9/1Q4X7tnYNtxeqFbS3Cr50HN7jNT8vfZA0?usp=sharing&sort=13&direction=a

3 Likes
20

I've tried your build on my RB760iGS with a MT7621 SoC.

Compared to the official build of OpenWRT i got 200+Mb/s in IPv6.

That's mean :

160Mb/s in IPv6 with OpenWRT build, and 380Mb/s in IPv6 with your build.

But i can't get a 850-900Mb/s bandwidth... (excepted in IPv4 ofc).

Btw on RouterOS i can have 850-900Mb/s in IPv6.

It's a reason for me to search a another router with OpenWRT support and SFP port (still not found atm).

I'm sick of this, i've been looking for 2 weeks now (Solid-Run products, Banana Pi, x86 Mini PC, Embedded Routers, all "coffee Bin" products..).

Can't sleep correctly :sweat_smile:

1 Like