Speed issue between br-lan and switch, I think?

Installing and Using OpenWrt Network and Wireless Configuration
1

I have a TP-Link Archer A9 v6 running OpenWrt 22.03.2 r19803-9a599fee93
The device has a 1GB WAN port and 1G ports on the switch.
My cable modem should get 500Mbps down. I verified by connecting a laptop with GB ethernet directly to the modem and got 520Mbps down.
But none of my computers connected to the router's switch ports can get more than ~200Mbps. All my wiring is up to snuff, cat 6 or better. The PCs' NICs are showing 1000Mbps link speed.

I installed iperf3 on the router and tested a few paths. With iperf3 -s running on the router, and iperf3 -c 192.xxx.xxx.xxx on any client machine, none could surpass 245Mbps. Example:

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  9.17 MBytes  76.9 Mbits/sec    0    259 KBytes       
[  5]   1.00-2.00   sec  10.4 MBytes  87.0 Mbits/sec    5    273 KBytes       
[  5]   2.00-3.00   sec  17.0 MBytes   142 Mbits/sec    1    287 KBytes       
[  5]   3.00-4.00   sec  23.6 MBytes   198 Mbits/sec    0    313 KBytes       
[  5]   4.00-5.00   sec  13.2 MBytes   111 Mbits/sec    0    336 KBytes       
[  5]   5.00-6.00   sec  10.1 MBytes  85.0 Mbits/sec    3    250 KBytes       
[  5]   6.00-7.00   sec  9.83 MBytes  82.5 Mbits/sec    0    255 KBytes       
[  5]   7.00-8.00   sec  11.6 MBytes  97.5 Mbits/sec    0    262 KBytes       
[  5]   8.00-9.00   sec  18.3 MBytes   153 Mbits/sec    0    266 KBytes       
[  5]   9.00-10.00  sec  16.0 MBytes   134 Mbits/sec    0    277 KBytes

Is it odd that the bitrate jumps around so much? (Max 198Mb/s; min 76.9Mb/s)

However, if I ran iperf3 -s on one of the client machines and iperf3 -c on another client (yes, pointing to that first client), I get about 1000MBPS.

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   114 MBytes   955 Mbits/sec    0    362 KBytes       
[  5]   1.00-2.00   sec   112 MBytes   941 Mbits/sec    0    397 KBytes       
[  5]   2.00-3.00   sec   113 MBytes   945 Mbits/sec    0    397 KBytes       
[  5]   3.00-4.00   sec   112 MBytes   941 Mbits/sec    0    397 KBytes       
[  5]   4.00-5.00   sec   112 MBytes   936 Mbits/sec    0    397 KBytes       
[  5]   5.00-6.00   sec   112 MBytes   940 Mbits/sec    0    397 KBytes       
[  5]   6.00-7.00   sec   113 MBytes   945 Mbits/sec    0    397 KBytes       
[  5]   7.00-8.00   sec   112 MBytes   943 Mbits/sec    0    397 KBytes       
[  5]   8.00-9.00   sec   112 MBytes   939 Mbits/sec    0    397 KBytes       
[  5]   9.00-10.00  sec   112 MBytes   940 Mbits/sec    0    397 KBytes

So the switch is happy to handle 1GBPS but the router can't communicate with it at 1GBPS.

I did try streaming and downloading a load of stuff at once and was unable to get the load (under status -> realtime graphs) to exceed 3.5%. The traffic on br-lan seems to max out around what iperf3 is indicating.

I USED to be a hardware guy 25 years ago, and have been a dev ever since. So I am capable but no longer knowledgeable enough to figure out whether I can configure my way out of this problem, or whether the hardware (or open-wrt's support for it) just isn't up to snuff.

Any thoughts on what I should look at here? Or how my br-lan should be configured? Currently most settings are straight 'out of the box'.

2

The SoC on the router is not capable of sourcing or sinking packets at anywhere near gigabit rates, even if the router could theoretically route traffic at those speeds. It comes down to the fact that the router's CPU isn't designed for this task -- it is designed to rapidly route traffic through the device, but not to process it locally. Silly analogy -- imagine if the postal service had to open each letter, it obviously couldn't move them through the system as fast as it can by just simply reading the addresses on the envelope.

You need to test through the router -- from WAN to LAN or LAN to WAN in order to accurately determine throughput. The LAN-LAN test you did does prove that the router's built-in switch is capable of gigabit throughput (as would be expected).

2 Likes
3

Thanks for that explanation.
I presume that it would also be insufficient for me to use iperf3 -s running out the router but bound to my wan address..? (for the same reason)
Fwiw that gives me basically the same results.

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  18.5 MBytes   155 Mbits/sec   39    293 KBytes       
[  5]   1.00-2.00   sec  18.5 MBytes   155 Mbits/sec    0    326 KBytes       
[  5]   2.00-3.00   sec  16.4 MBytes   138 Mbits/sec    1    331 KBytes       
[  5]   3.00-4.00   sec  22.7 MBytes   190 Mbits/sec    1    345 KBytes       
[  5]   4.00-5.00   sec  28.4 MBytes   238 Mbits/sec    1    277 KBytes       
[  5]   5.00-6.00   sec  29.6 MBytes   248 Mbits/sec    0    331 KBytes       
[  5]   6.00-7.00   sec  21.3 MBytes   178 Mbits/sec    1    272 KBytes       
[  5]   7.00-8.00   sec  14.1 MBytes   118 Mbits/sec    0    294 KBytes       
[  5]   8.00-9.00   sec  11.0 MBytes  92.0 Mbits/sec    0    309 KBytes       
[  5]   9.00-10.00  sec  11.8 MBytes  99.0 Mbits/sec    0    334 KBytes

Since my ISP owns the modem, I can't test throughput to there; the closest I can get is the speedtest site the ISP hosts. Which confirms my results above -- about 200Mbps max.

So something about my router is definitely limiting the throughput.

4

Correct.

So, if you want to test your router's throughput in a controlled manner, you can actually connect a computer on the WAN side and another computer on the LAN side and then run iPerf between them. To do this, it is easiest (but not required) to put the router under test behind another router with a DHCP server enabled and one of the computers connected to that router. You can do this without another router (just using static IPs), and you can even do it without messing with the WAN side at all by implementing VLANs and running throughput tests between the VLANs.

That said, if 200Mbps is well below your expected/paid-for speed from the ISP, you may need to enable software flow offloading (or hardware offloading) if it is available on your router. Keep in mind that there are some routers that perform better using the vendor firmware because these features are enabled by default... OpenWrt may require you to explicitly enable them (and in some cases, it may not be available due to closed-source drivers and such).

2 Likes
5

WELL!

That makes sense. All I'd need is to put the wan port on the same subnet as the PC on the wan side, or enable a dhcp server on that PC.

But I won't be trying that, because enabling software and hardware offloading was the solution. I'm up to 400Mbps at least.

THANKS!!!

For those coming late to the party, this is what I changed in LuCi:

6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.