2 Gbps WAN/LAN NAT Routing on ramips MT7621 devices

arinc9 · 2022-08-25T12:53:54.462Z

Ok, I’ve never checked the output of ethtool so I don’t know what to tell you. If wan port works, I don’t see a problem.

Lucky1 · 2022-08-25T13:35:14.284Z

when I use these devices as access points
I map the wan port back to the br-lan
so in this mode is the cpu now needed
to move traffic between wan and lan ports
or will it reattach it back to the switch ?

arinc9 · 2022-08-25T15:22:50.758Z

Yes, frames will go through the CPU for wan port.

Cheddoleum · 2022-08-25T20:32:04.873Z

arinc9:

The pull request is applied to the master branch!

Verified snapshot build downloaded 24 August 2022, openwrt-ramips-mt7621-linksys_ea7500-v2-squashfs-factory.bin on, yes, a Linksys EA7500 v2:

Concurrent tests 172.18.64.148 (Core i5 Intel i340 igb diriver connected to "Internet" i.e. WAN port) <--> 172.18.1.94 (Core i7 Marvell sky2 driver connected to LAN 1 port). Both subnets /24 therefore routed.

Connecting to host 172.18.64.148, port 5201
[  4] local 172.18.1.94 port 59426 connected to 172.18.64.148 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   113 MBytes   944 Mbits/sec    0    417 KBytes       
[  4]   1.00-2.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   2.00-3.00   sec   110 MBytes   927 Mbits/sec    0    417 KBytes       
[  4]   3.00-4.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   4.00-5.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   5.00-6.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   6.00-7.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   7.00-8.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   8.00-9.00   sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]   9.00-10.00  sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]  10.00-11.00  sec   110 MBytes   926 Mbits/sec    0    417 KBytes       
[  4]  11.00-12.00  sec   112 MBytes   938 Mbits/sec    0    658 KBytes       
[  4]  12.00-13.00  sec   110 MBytes   927 Mbits/sec    0    658 KBytes       
[  4]  13.00-14.00  sec   110 MBytes   926 Mbits/sec    0    658 KBytes       
[  4]  14.00-15.00  sec   110 MBytes   926 Mbits/sec    0    658 KBytes       
[  4]  15.00-16.00  sec   110 MBytes   927 Mbits/sec    0    658 KBytes       
[  4]  16.00-17.00  sec   110 MBytes   926 Mbits/sec    0    658 KBytes       
[  4]  17.00-18.00  sec   110 MBytes   926 Mbits/sec    0    658 KBytes       
[  4]  18.00-19.00  sec   110 MBytes   927 Mbits/sec    0    658 KBytes       
[  4]  19.00-20.00  sec   110 MBytes   926 Mbits/sec    0    658 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-20.00  sec  2.16 GBytes   928 Mbits/sec    0             sender
[  4]   0.00-20.00  sec  2.16 GBytes   926 Mbits/sec                  receiver

iperf Done.

Connecting to host 172.18.1.94, port 5201
[  5] local 172.18.64.148 port 52098 connected to 172.18.1.94 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   111 MBytes   934 Mbits/sec    0    638 KBytes       
[  5]   1.00-2.00   sec   111 MBytes   933 Mbits/sec    0    638 KBytes       
[  5]   2.00-3.00   sec   112 MBytes   943 Mbits/sec    0    638 KBytes       
[  5]   3.00-4.00   sec   111 MBytes   932 Mbits/sec    0    638 KBytes       
[  5]   4.00-5.00   sec   111 MBytes   933 Mbits/sec    0    638 KBytes       
[  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec    0    638 KBytes       
[  5]   6.00-7.00   sec   112 MBytes   943 Mbits/sec    0    638 KBytes       
[  5]   7.00-8.00   sec   111 MBytes   933 Mbits/sec    0    638 KBytes       
[  5]   8.00-9.00   sec   111 MBytes   933 Mbits/sec    0    638 KBytes       
[  5]   9.00-10.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  10.00-11.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  11.00-12.00  sec   112 MBytes   944 Mbits/sec    0    670 KBytes       
[  5]  12.00-13.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  13.00-14.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  14.00-15.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  15.00-16.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  16.00-17.00  sec   112 MBytes   944 Mbits/sec    0    670 KBytes       
[  5]  17.00-18.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  18.00-19.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
[  5]  19.00-20.00  sec   111 MBytes   933 Mbits/sec    0    670 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec  2.18 GBytes   935 Mbits/sec    0             sender
[  5]   0.00-20.00  sec  2.17 GBytes   934 Mbits/sec                  receiver

iperf Done.

Also tested using LAN 3 port (removed from default bridge configured as above) in place of WAN. As might be expected, speed was slightly worse than halved, roughly 450/450.

Note: Again, the above tests were concurrent. Did it this way because the iperf3 version on one of the hosts did not support the --bidir option.

vevere · 2022-08-26T17:04:58.494Z

Hi,

I try to understand this very interesting patch.
Before your patch:

poor rooting perf in router mode (wan <-> br-lan)
good switching perf in full bridged mode (br-lan containing all the interfaces WAN + LAN[1-x] as I do each time I use a router as an AP)

After your patch:

good rooting perf (twice as before) in router mode (wan <-> lan)
poor switching perf (half as before) between "WAN" device and "LANx" device in full bridged mode (br-lan containing all the interfaces WAN + LAN[1-x] )

So if I want to use an recent openwrt snapshot on a MT7621 compatible device as an AP/managed switch it should be really better to not use the WAN port as the uplink and reserve it for a device that consumes little bandwidth (printer/iot) ?

Am I right ?

I also ask that because I don't know where the wireless radio is "plugged" inside the hardware, so I don't know if is better to associate the wireless radio to bridge that uses the "wan" port or a lan port as the uplink (sorry, hard to explain for me in English, I hope you will understand my question) with the use of your patch

Regards

Edit:
I made a quick test on my Cudy WR1300.
Switching only (my use case acting as an AP/managed switch) with or without sw and hw offloading (same result)

22.03.0-rc6:

lan <=> wan:

lan <=> lan:

Actual snapshot (r20386-15cae55cec):

lan <=> wan:

lan <=> lan:

So switching perf are altered with this patch as expected.
Is there any way to make it a "conditionnal patch" ?
Something like:
if "wan iface" not in br-lan
then "use this patch"
elseif "wan iface" in br-lan
then "don't use it"

or:
if hw offloading in use
then use this patch
elseif
then don't use it

sorry, I'm not a dev

arinc9 · 2022-08-26T17:54:14.151Z

vevere:

So if I want to use an recent openwrt snapshot on a MT7621 compatible device as an AP/managed switch it should be really better to not use the WAN port as the uplink and reserve it for a device that consumes little bandwidth (printer/iot) ?

Am I right ?

This makes sense for the use case of switching frames between switch ports.

Also read this post.

vevere:

I also ask that because I don't know where the wireless radio is "plugged" inside the hardware, so I don't know if is better to associate the wireless radio to bridge that uses the "wan" port or a lan port as the uplink (sorry, hard to explain for me in English, I hope you will understand my question) with the use of your patch

Wireless is directly connected to the CPU, you can use any switch port to switch frames between. There won't be any effect to performance.

vevere · 2022-08-26T18:08:41.499Z

thanks for the reply, it's clear
I edited my previous post with some screenshots

is there any possibility to make the use of this patch an option (like the hw/sw offloading checkbox) ?

arinc9 · 2022-08-26T18:58:14.765Z

vevere:

I edited my previous post with some screenshots

is there any possibility to make the use a this patch an option (like the hw/sw offloading checkbox) ?

I’ve read the edit. That’s not possible, for now.

badulesia · 2022-08-29T17:05:57.706Z

Hi
I have tested the latest (29/8) snapshot on a Netgear R6220. CPU load was monitored with htop

native : 580 Mbit/s, heavy CPU load
SW offloading : 750 Mbit/s, still heavy CPU load
both SW and HW offloading : 940 Mbit/s, negligible CPU load (can't remember ... 2 or 3% ?)

So the patch seems to work very well on bandwidth tests. Nevertheless, I don't use this device as a router, it was just a test. So I can't confirm stability on the long run, someone will.

I use an x64 router as main device, and I keep an old x64 as spare router in case. I know now that I can use back the R6220 as spare router with a Gbit/s fiber.

Thank you for this important contribution.

arinc9 · 2022-08-29T17:18:54.028Z

Cheers! By the way, the section for selecting software and hardware offload on LuCI is inaccurate. Either hardware or software offloading can be used. They cannot be used at the same time.

badulesia · 2022-08-29T19:22:26.904Z

Agreed, it is obvious.
What I meant, and I assume you understood, is that I clicked on both checkboxes in Luci.
As far as I understand, checking SW activates SW. Than checking HW deactivates SW to activate HW. I see this in some kind of two steps.
A change in Luci may clarify this ? A checkbox to activate offloading, than a radio to choose between SW or HW.

arinc9 · 2022-08-29T20:32:03.985Z

badulesia:

A change in Luci may clarify this ? A checkbox to activate offloading, than a radio to choose between SW or HW.

That looks good. Maybe @jow could do this.

badulesia · 2022-08-29T21:31:13.692Z

something simple like this (forget about the colors, I picked up what I can to mimic)

enable offloading : software hardware

SW/HW radios being disabled if the checkbox is not checked.
SW selected as default.

maxmib · 2022-08-31T16:09:04.077Z

This patch achieves 2 Gbps total bandwidth to the CPU by muxing the MT7530 switch's phy0/4 to the SoC's gmac1 on devices where RGMII2 pins are available.So it means that phy0 or phy4 can benefit from this patch if it's used as wan port ? I use my er-x router's eth4 as wan port .Is eth4 the phy4? Because eth0 is phy0.Can this patch works to eth4 ?Thanks.

neheb · 2022-09-01T00:44:30.367Z

phy0 is assigned to gmac1 and phy4 to gmac0. What's the issue?

maxmib · 2022-09-01T01:10:57.468Z

I use ubnt erx 's eth4 as wan port. In this case,Can eth4 benefit from this patch?Thanks.

neheb · 2022-09-01T01:27:17.321Z

I mean...you can edit the dts

vevere · 2022-09-03T13:10:27.402Z

badulesia:

something simple like this (forget about the colors, I picked up what I can to mimic)

enable offloading : software hardware

SW/HW radios being disabled if the checkbox is not checked.
SW selected as default.

or something similar to the other choices on this same FW page, a drop list:

image612×204 4.87 KB

Texte: Flow offloading

choice 1: Off/disable/No thanks ...
choice 2: Software
choice 3: Hardware

anon72830772 · 2022-09-07T09:47:13.661Z

Hello arinc9,

Quick question, does the current 22.03.0 stable release include your patch(s) as well?
I tried reading the posted changelog but the bleepin' commit diff? is not in any alphabetical or numerical order. Unless there is some trick to it that I'm missing ...

Thanks.

arinc9 · 2022-09-07T10:12:07.717Z

Other than the patch for GB-PC, they are not on 22.03.