new issue on recent builds. connections over ethernet have very bad upload throughput. over wifi the issue doesn't exist and when I try uploading from the router itself, the issue also doesn't happen, only for ethernet clients
hello @Dopam-IT_1987 . I am assuming your colleague also has the same router as you. Isn’t he throttled to ~500mbps using qosify? I’m going to try to turn off packet steering tomorrow and see how it performs.
1 Like
His tests showed about 250 Mbit/s and he reported the same using NAS.
With my WDS setup I see more like 700 Mbit/s so his 250 Mbit/s is surely not merely related to CPU cycles lost running iperf3?
At what point does this issue actually bite then?
I think I'm seeing the same issue, links are doing 400+ mbit but ethernet clients are barely doing 100mbit (Totolink A8000RU device but it uses the same SoC)
I've just tested a build with some commits reverted that I thought might have been the cause, but it turns out they were unrelated.
@daniel you can rule out the 4 recent mediatek ethernet phy/irq changes changes for this performance degradation
My RT3200 is setup as a dumb AP. Only physical connection is the WAN port to a managed switch. When I run iperf3 tests to/from a wired PC also connected to that switch, I am getting good throughputs (~940 Mbps). Is this not the case for you?
Test 1. Listening on RT3200/Sending on PC
RT3200:
# iperf3 -s
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------
Accepted connection from 10.9.8.234, port 44522
[ 5] local 10.9.8.2 port 5201 connected to 10.9.8.234 port 44526
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 112 MBytes 938 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 939 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 939 Mbits/sec
[ 5] 3.00-4.00 sec 112 MBytes 939 Mbits/sec
[ 5] 4.00-5.00 sec 112 MBytes 939 Mbits/sec
[ 5] 5.00-6.00 sec 112 MBytes 939 Mbits/sec
[ 5] 6.00-7.00 sec 112 MBytes 939 Mbits/sec
[ 5] 7.00-8.00 sec 112 MBytes 939 Mbits/sec
[ 5] 8.00-9.00 sec 112 MBytes 939 Mbits/sec
[ 5] 9.00-10.00 sec 112 MBytes 939 Mbits/sec
[ 5] 10.00-10.00 sec 298 KBytes 863 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201 (test #2)
-----------------------------------------------------------
And sending on the PC:
% iperf3 -c 10.9.8.2
Connecting to host 10.9.8.2, port 5201
[ 5] local 10.9.8.234 port 44526 connected to 10.9.8.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 113 MBytes 949 Mbits/sec 26 341 KBytes
[ 5] 1.00-2.00 sec 112 MBytes 939 Mbits/sec 39 284 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 940 Mbits/sec 26 349 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 940 Mbits/sec 39 310 KBytes
[ 5] 4.00-5.00 sec 112 MBytes 939 Mbits/sec 26 369 KBytes
[ 5] 5.00-6.00 sec 112 MBytes 939 Mbits/sec 39 318 KBytes
[ 5] 6.00-7.00 sec 112 MBytes 937 Mbits/sec 26 365 KBytes
[ 5] 7.00-8.00 sec 112 MBytes 939 Mbits/sec 39 338 KBytes
[ 5] 8.00-9.00 sec 112 MBytes 938 Mbits/sec 39 284 KBytes
[ 5] 9.00-10.00 sec 112 MBytes 939 Mbits/sec 26 345 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.09 GBytes 940 Mbits/sec 325 sender
[ 5] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec receiver
iperf Done.
Test 2. Listening on PC/Sending on RT3200
On the PC:
% iperf3 -s
-----------------------------------------------------------
Server listening on 5201 (test #1)
-----------------------------------------------------------
Accepted connection from 10.9.8.2, port 59006
[ 5] local 10.9.8.234 port 5201 connected to 10.9.8.2 port 59008
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 111 MBytes 934 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 939 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 939 Mbits/sec
[ 5] 3.00-4.00 sec 112 MBytes 939 Mbits/sec
[ 5] 4.00-5.00 sec 112 MBytes 939 Mbits/sec
[ 5] 5.00-6.00 sec 112 MBytes 939 Mbits/sec
[ 5] 6.00-7.00 sec 112 MBytes 939 Mbits/sec
[ 5] 7.00-8.00 sec 112 MBytes 939 Mbits/sec
[ 5] 8.00-9.00 sec 112 MBytes 939 Mbits/sec
[ 5] 9.00-10.00 sec 112 MBytes 939 Mbits/sec
[ 5] 10.00-10.01 sec 1.01 MBytes 937 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 1.09 GBytes 939 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201 (test #2)
-----------------------------------------------------------
On the RT3200:
# iperf3 -c 10.9.8.234
Connecting to host 10.9.8.234, port 5201
[ 5] local 10.9.8.2 port 59008 connected to 10.9.8.234 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 0 666 KBytes
[ 5] 1.00-2.00 sec 112 MBytes 942 Mbits/sec 0 666 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 939 Mbits/sec 0 666 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 939 Mbits/sec 0 666 KBytes
[ 5] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 0 666 KBytes
[ 5] 5.00-6.00 sec 112 MBytes 938 Mbits/sec 0 666 KBytes
[ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 0 666 KBytes
[ 5] 7.00-8.00 sec 112 MBytes 940 Mbits/sec 0 697 KBytes
[ 5] 8.00-9.00 sec 112 MBytes 937 Mbits/sec 0 697 KBytes
[ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 0 730 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec 0 sender
[ 5] 0.00-10.01 sec 1.09 GBytes 939 Mbits/sec receiver
iperf Done.
No, that's not the case for me....
Windows 10 <-> Totolink A8000RU <- WDS -> Totolink A8000RU <-> FreeBSD 13
The Totolinks are running: OpenWrt SNAPSHOT, r18710-dc2da6a233 (using OpenSSL instead of wolfSSL)
=== Windows 10 (server) and FreeBSD 13 (client) ===
iperf3 -c 192.168.1.239 -t 60
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 2.95 GBytes 422 Mbits/sec 206 sender
[ 5] 0.00-60.02 sec 2.95 GBytes 422 Mbits/sec receiver
iperf3 -c 192.168.1.239 -t 60 -R
[ ID] Interval Transfer Bitrate
[ 5] 0.00-60.00 sec 1.06 GBytes 152 Mbits/sec sender
[ 5] 0.00-60.00 sec 1.06 GBytes 152 Mbits/sec receiver
=== Totolink (server, where Windows box connects to) and FreeBSD 13 ===
iperf3 -c 192.168.1.253 -t 60
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 3.05 GBytes 436 Mbits/sec 0 sender
[ 5] 0.00-60.02 sec 3.05 GBytes 436 Mbits/sec receiver
iperf3 -c 192.168.1.253 -t 60 -R
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 2.59 GBytes 370 Mbits/sec 28 sender
[ 5] 0.00-60.00 sec 2.59 GBytes 370 Mbits/sec receiver
Can you repeat the tests as I have using iperf3 on the RT3200 directly? Use the same switches I did as well.
The second half is exactly what you're asking for.... I don't mean to pollute this thread bit I'm fairly sure it's a SoC related issue rather than a device specific one.
I cannot reproduce.
On the RT3200:
# iperf3 -s
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 6.51 GBytes 933 Mbits/sec 4611 sender
On the wired PC:
% iperf3 -c 10.9.8.2 -t 60 -R
...
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 6.51 GBytes 933 Mbits/sec 4611 sender
[ 5] 0.00-60.00 sec 6.51 GBytes 932 Mbits/sec receiver
How recent is your OpenWRT image on your RT3200?
Is that using a fairly recent commit? Someone else reported the same thing, Belkin RT3200/Linksys E8450 WiFi AX discussion - #1588 by justinkb which is why I don't think it's device specific... That's being said I'm seeing this only on the far end...
Yes, I built the snapshot on 11-Feb from c391dcd. Try a newer image and report back?
What does "I'm seeing this only on the far end..." mean?
Hmm... I did some more testing, it's seem like in my case at least there's seems to be a NIC compatibility issue. Hooking up another NIC (Intel instead of Apple Thunderbolt adapter (Broadcom)) restores performance using the same cable(s) etc. Worth noting is that the A8000RU uses another switch (Realtek RTL8367S) while the RT3200 uses Mediatek MT7531BE.
For the record, iperf from Ethernet lan device to Ethernet lan device I get 900+ Mbps. It's devices on lan connected via ethernet that get slow upload throughout over wan that's my issue
It could be that the new PHY driver causes this regression, though it seems unlikely given that LAN<->LAN traffic also going through the switch is not affected.
The PHY driver on MT7531 also sets up some TX delay bits which I was surprised to see there (as things were working fine in that regard):
+ /* Set TX Pair delay selection */
+ phy_write_mmd(phydev, MDIO_MMD_VEND1, 0x13, 0x404);
+ phy_write_mmd(phydev, MDIO_MMD_VEND1, 0x14, 0x404);
You could try to drop this part of the patch and see if it resolves the problem.
1 Like
It could be that the new PHY driver causes this regression
but it still occurs on my build with those commits completely reverted, as stated above.
specifically:
Revert "mt7622: linksys-e8450: enable using mt7531 switch irq"
Revert "mt7622: bpi-r64: enable using mt7531 switch irq"
Revert "kernel: backport MediaTek Ethernet PHY driver"
Revert "kernel: backport MT7530 IRQ support
I was suspecting
kernel: backport MediaTek Ethernet PHY driver
could be the cause. If the issue still shows even with this reverted, it must hence be something else.
When trying a build with all this reverted: Did you do a hard power cycle (ie using the physical switch or by disconnecting the power supply) or just software reboot? Because it can be that a soft reboot doesn't reset the registers in the PHY and hence the issue would persist...
1 Like
i'll try that in a bit
No change after power cycle
@justinkb Thanks for trying that. Now we can really rule out that the MT7531-related changes caused any new problems. Which brings us back to square 1: what else could have caused it?
I guess we are down to use git bisect now (given that there is a known-to-work-well commit which still works well when building it now...)
1 Like