2 Gbps WAN/LAN NAT Routing on ramips MT7621 devices

Community Builds, Projects & Packages
1

18th of April 2023 Update

Here is some information to prevent repeat questions directed at me. This won't be on 22.03, only newer versions will have it as the patch was made at that time frame, and there's no need to spend time and effort backporting to older versions.

For the devices listed under the "RGMII2 pins are used with an external phy on these:" and "RGMII2 pins are used as GPIO on these:" lists, the feature will work with changes on the devicetree of the said device, except some rare cases. You're expected to do the changes yourself.

21st of August 2022 Update

The pull request is applied to the master branch!

10th of July 2022 Update

PR is open on GitHub.

It is possible to achieve 2 Gbps routing in total on devices with MT7621 SoCs.
Currently you can only get a maximum of 500/500 Mbps (download/upload) speed on LAN/WAN routing (this includes NAT too). The reason is explained at the bottom of the post.

With this patch, 1000/1000 Mbps routing or 2 Gbps total bandwidth to the CPU is possible.

Speed Test

Here are the results for iperf3 bidirectional speed test with hardware offloading.
Routing and NAT performance is identical.

Without the patch

[  5][RX-S]   0.00-20.00  sec   353 MBytes   148 Mbits/sec                  receiver
[  8][TX-S]   0.00-20.00  sec  1.79 GBytes   769 Mbits/sec    0             sender

With the patch

[  5][RX-S]   0.00-20.00  sec  2.16 GBytes   926 Mbits/sec                  receiver
[  8][TX-S]   0.00-20.00  sec  1.80 GBytes   775 Mbits/sec    0             sender

Devices That Benefit This

Check out the changes of the phy muxing commit for supported devices.

Devices confirmed working:
See GitHub PR above.

Devices That Can't Benefit This

List

Single port devices:

  • mt7621_asus_rp-ac56.dts
  • mt7621_asus_rp-ac87.dts
  • mt7621_edimax_re23s.dts
  • mt7621_thunder_timecloud.dts
  • mt7621_ubnt_unifi.dtsi
    • mt7621_ubnt_unifi-6-lite.dts
    • mt7621_ubnt_unifi-nanohd.dts
  • mt7621_zyxel_nwa55axe.dts

No muxable phy on these:

  • mt7621_xiaomi_mi-router-3g.dts

RGMII2 pins are used with an external phy on these:

  • mt7621_gnubee_gb-pc2.dts
  • mt7621_mikrotik_routerboard-760igs.dts
  • mt7621_ubnt_edgerouter-x-sfp.dts
  • mt7621_zyxel_wap6805.dts

RGMII2 pins are used as GPIO on these:

  • mt7621_alfa-network_quad-e4g.dts
  • mt7621_bolt_arion.dts
  • mt7621_d-team_newifi-d2.dts
  • mt7621_d-team_pbr-m1.dts
  • mt7621_dual-q_h721.dts
  • mt7621_firefly_firewrt.dts
  • mt7621_mikrotik_routerboard-m11g.dts
  • mt7621_mtc_wr1201.dts
  • mt7621_netgear_ex6150.dts
  • mt7621_sercomm_na502.dts
  • mt7621_sercomm_na502s.dts
  • mt7621_telco-electronics_x1.dts
  • mt7621_tplink_archer-x6-v3.dtsi
    • mt7621_tplink_archer-a6-v3.dts
    • mt7621_tplink_archer-c6-v3.dts
  • mt7621_tplink_re350-v1.dts
  • mt7621_tplink_re650-v2.dts
  • mt7621_tplink_rexx0-v1.dtsi
    • mt7621_tplink_re500-v1.dts
    • mt7621_tplink_re650-v1.dts
  • mt7621_tplink_tl-wpa8631p-v3.dts
  • mt7621_wavlink_wl-wn53xax.dtsi
    • mt7621_wavlink_wl-wn531a6.dts
    • mt7621_wavlink_wl-wn533a8.dts
  • mt7621_wevo_w2914ns-v2.dtsi
  • mt7621_wevo_11acnas.dts
    • mt7621_wevo_w2914ns-v2.dts
    • mt7621_zio_freezio.dts
  • mt7621_winstars_ws-wn583a6.dts
  • mt7621_xzwifi_creativebox-v1.dts
  • mt7621_zbtlink_zbt-wg2626.dts
  • mt7621_zbtlink_zbt-wg3526.dtsi
    • mt7621_zbtlink_zbt-wg3526-16m.dts
    • mt7621_zbtlink_zbt-wg3526-32m.dts
  • mt7621_zyxel_lte3301-plus.dts
  • mt7621_zyxel_nwa50ax.dts

500/500 Mbps Speed Example

You download a file from internet with 1000 Mbps speed.[*] Traffic will flow through wan to lan. On the CPU port, this causes 1000/1000 Mbps load because it receives 1 Gbps traffic from wan then transmits 1 Gbps traffic to lan.

Let's say someone uploads a file with 1000 Mbps speed at the same time. Traffic will flow through lan to wan. Since there's no bandwidth left on the CPU port it will halve the speed.
Therefore, you can only download a file at 500 Mbps and upload a file at 500 Mbps at the same time.

With this patch, wan port is connected directly to the CPU on a second link.
Now, you can download and upload a file at 1 Gbps at the same time.

[*] In a real life scenario, your computer downloading the file has to send some packets back (upload) because the TCP packets for downloading the file may be out of order, some of it may have to be re-sent, etc.. So you won't even get to download your file with 1 Gbps download speed on a single link to CPU.
Therefore, this patch should also improve 1 Gbps speed for just downloading or uploading.

My Speed Testing Method

I do the test on a single computer. I have two gigabit ports on my motherboard.
I isolate a port by putting it on another network namespace to do the test.

Client Network
iperf client: 192.168.2.2/24
router: 192.168.2.1/24

Server Network
router: 192.168.3.2/24
iperf server: 192.168.3.1/24

iperf Client

ip a add 192.168.2.2/24 dev enp9s0
ip l set up enp9s0
ip route add 192.168.3.1 via 192.168.2.1
iperf3 -c 192.168.3.1 --bidir -t 20

iperf Server

ip netns add iperfserver
ip link set dev eno1 netns iperfserver
ip netns exec iperfserver ip a add 192.168.3.1/24 dev eno1
ip netns exec iperfserver ip l set up eno1
ip netns exec iperfserver iperf3 -s

Exceptions

The ports called "wan" are muxed where possible. On a minority of devices, this is not possible. Those cases:

mt7621_ampedwireless_ally-r1900k.dts: lan3
mt7621_ubnt_edgerouter-x.dts: eth0
mt7621_gnubee_gb-pc1.dts: ethblue
mt7621_linksys_re6500.dts: lan1
mt7621_netgear_wac104.dts: lan4
mt7621_tplink_eap235-wall-v1.dts: lan0
mt7621_tplink_eap615-wall-v1.dts: lan0
mt7621_ubnt_usw-flex.dts: lan1

The "wan" port is just what the vendor designated on the board/plastic chasis of the device. On a technical level, there is no difference between a lan and wan port on MT7621AT/MT7621ST SoCs. Prefer connecting to WAN via the port described above for these devices to benefit the feature brought with this patch.

26 Likes
2

D-Link DIR-878 rev. A1 and DIR-882 rev. A1

Installation process was fine.
No issues encountered with internet access.

As per arinc9's suggestion in another topic,
Software and Hardware flow offloading enabled.

Question: Any compatibility issues with irqbalance?

EDIT/Update:
Tested on D-Link DIR-2640 and no issues encountered with installation.

3

I don't see how this would affect distributing hardware interrupts across threads, so no.

2 Likes
4

@banisodaro can you try this on your Linksys EA7500v2?

5

Xiaomi Mi Router 3 Pro, everything seems fine.

1 Like
6

As i see, there is need to patch only dts file of the device or something else?

7

It's just the devicetree that needs changing. The rest is already implemented on the driver. The phy0/4 muxing on the MT7621AT/MT7621ST SoCs is documented on Linux documentation for MT7530 DSA driver.

1 Like
8

Ok. will try patch 22.03.0-rc5 on Xiaomi R4A gigabit and test.

9

If you could build an image for Beeline Smartbox Giga, I could test it.

10

I added it, let me know how it goes.

11

Just tested, seems also ok.

1 Like
12

Hey, I just tested Beeline Smartbox Giga a little bit more, made some iperf3s and I couldn't get more than 1 gigabit in sum.

 ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec   561 MBytes   470 Mbits/sec    0             sender
[  5][TX-C]   0.00-10.00  sec   559 MBytes   468 Mbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec   554 MBytes   465 Mbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec   552 MBytes   463 Mbits/sec                  receiver

Then I tried Beeline Smartbox Turbo+ with your patches and I saw the same picture 1Gb max, but because it has more ports I could test speed between lan1 and lan2 and there I had full 2Gb.

13

is hardware offloading enabled

14

Something strange happened. IDK. I used your factory image (because my GIGA was using vendor firmware) and it showed poor performance, then i sysupgraded it to generic openwrt with the same performance. Now i sysupgraded again to your image and now I get my 2 gigabits. And yes, i had HW offloading enabled.

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  1.09 GBytes   937 Mbits/sec    8             sender
[  5][TX-C]   0.00-10.00  sec  1.09 GBytes   935 Mbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  1.06 GBytes   914 Mbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  1.06 GBytes   911 Mbits/sec                  receiver

Also in your factory image wan had type "switch port" and in your sysupgrade it has type "Ethernet adapter". Perhaps there's something wrong with factory IDK.

15

A question about Turbo+. We won't get 2Gbps even if its PR with your patches is accepted right? We need to wait for your PR to get accepted also?

16

Like I said on the Turbo+ OpenWrt Support post, my PR is irrelevant to Turbo and Turbo+ since the changes my PR brings are incorporated into the PRs for the said devices.

There is a lot of variables as to why you do not get 2G total speed off the iperf3 test.
Put output from the ā€˜ip linkā€™ command here so we can at least make sure the wan port is muxed to gmac1 on your Turbo+.

If the interface is called ā€œEthernet adapterā€ that means the port is muxed which is what we want to achieve. Make sure this is the same case for Turbo+. Iā€™m not sure why thereā€™s a difference between factory and sysupgrade images.

17 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 qdisc fq_codel state UP qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
3: wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
4: lan1@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
5: lan2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
6: lan3@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
7: lan4@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
8: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
11: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff
13: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether REDACTED brd ff:ff:ff:ff:ff:ff

MACs have been REDACTED. Wan is in bridge with lan ports.
Maybe we need some kernel patches of yours? Because kernel in your images seems incompatible with mainstream kernel modules.

18

You can see that the wan interface is not a slave of eth0, this confirms itā€™s using gmac1, the second gmac.

No, that is unrelated.

You must be using bridge VLAN filtering since wan port is in the same bridge as lan ports?

19

Just tested Turbo+ with your patch but after resetting settings and turning on HW nat.

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec   920 MBytes   772 Mbits/sec   50             sender
[  5][TX-C]   0.00-10.00  sec   917 MBytes   769 Mbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  1.06 GBytes   911 Mbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  1.06 GBytes   908 Mbits/sec                  receiver

Seems legit. But it looks like it's not compatible with my config. I use turbo+ as dumb switch and ap. In order to do that I turned off dhcp-server, added wan-port to br-lan and made dhcp-client interface with br-lan.

cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option packet_steering '1'
        option ula_prefix 'fd4f:6661:7041::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'
        list ports 'wan'

config interface 'lan'
        option device 'br-lan'
        option proto 'dhcp'
        option delegate '0'

config interface 'lan6'
        option proto 'dhcpv6'
        option device '@lan'
        option reqaddress 'try'
        option delegate '0'
        option reqprefix 'no
20

Tested and working fine on Netgear WAC124.
WAN port is acting normal and shows increased speed with --bidir option. :star_struck:

1 Like