I got stuck into a performance issue after upgrading from 17.01 to 18.06.4. My router is a Netgear r7800 and when I run iperf3 on router and desktop, i get this:
So there is no problem from desktop to router (900Mbit/sec), but router to desktop is too slow and fluctuates (150-400 Mbit/sec). I tried several computers/cables/ports, always the same results. I can also confirm that desktop to desktop speed (via r7800) isn't slow.
Can someone point my in de right direction? Samba worked better before
Oh, I have a similar issue with my x86 board Celeron J3355 dual core 2Ghz.
Iperf has never exceeded ~ 600mbps in both directions.
My board's cpu is much stronger than r7800.
What are the actual symptoms of your performance issue? What is iperf like via your router as opposed to from your router? [and as an aside what was the change since you upgraded] Because the scenario you are testing doesn't really correspond to a real use case.
Well, I noticed a slower downloading speed from my router to my desktop using a Samba share. On 17.01 is was 60-80 MB/s, and now it is 15-30 MB/s. Uploading speed is fine. So I investigated the problem and found out it was not the SSD USB harddrive, nor Samba. According to iperf it has to be the ethernet speed.
@stuffie
Jump to 19.X or master/trunk branch, this is where all debugging and enhancements will occur.
If you want a community build for your router this might be of interest: Build for Netgear R7800
@leeandy
Does performance improve if you use another Linux/BSD distribution?
I'm try running Ubuntu 19.04 live-usb on my board, but not improve, similar speed.
I noticed that when using a iperf with cubic setting, the speed was a bit faster ~ 100Mbps than bbr setting. Noticed that the rate of increase and decrease was erratic.
After modifying some things (set cpu scaling_governor to performance, install irqbalance, set bios C state to C6 instead of C10), the speed improved slightly.
Iperf server running on my Ubuntu pc, iperf client run on router board:
root@ASR:~# iperf3 -c 192.168.16.65 -V -C bbr
iperf 3.7
Linux ASR 4.19.57
Control connection MSS 1448
Connecting to host 192.168.16.65, port 5201
Cookie: jdlxgv6o3gdeqavmhxhtl2kmxk7vrs2mfrf3
TCP MSS: 1448 (default)
[ 5] local 192.168.16.1 port 53481 connected to 192.168.16.65 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 842 MBytes 707 Mbits/sec 57544 sender
[ 5] 0.00-10.05 sec 841 MBytes 702 Mbits/sec receiver
CPU Utilization: local/sender 3.9% (0.0%u/3.9%s), remote/receiver 35.4% (3.0%u/32.3%s)
snd_tcp_congestion bbr
rcv_tcp_congestion bbr
root@ASR:~# iperf3 -c 192.168.16.65 -V -C cubic
iperf 3.7
Linux ASR 4.19.57
Control connection MSS 1448
Connecting to host 192.168.16.65, port 5201
Cookie: gwbfl3r2wwezoqwfeffvuihstcpozj6dbmm3
TCP MSS: 1448 (default)
[ 5] local 192.168.16.1 port 53827 connected to 192.168.16.65 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 974 MBytes 817 Mbits/sec 21385 sender
[ 5] 0.00-10.04 sec 971 MBytes 811 Mbits/sec receiver
CPU Utilization: local/sender 4.6% (0.5%u/4.1%s), remote/receiver 33.1% (2.3%u/30.8%s)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
If that's the case I think you can safely assume it's performance issue with the IPQ8***-chipset and/or possibly a faulty cable. There's a "hack/workaround" which makes the SoC go full blast all the time by changing the cpu governor from what I've read but since I don't have any IPQ8-chipset devices I haven't followed the discussion(s) closely.
Thanks for all suggestions. Looking for the best solution, I ran more tests.
First against different versions (clean installs) of LEDE/OpenWRT:
LEDE Reboot 17.01.4 average speed 923 Mbit/sec
LEDE Reboot 17.01.6 average speed 840 Mbit/sec
OpenWRT 18.06.0 average speed 422 Mbit/sec
OpenWRT 18.06.4 average speed 238 Mbit/sec
Those are all download speeds from router to desktop. So, newer versions give worse results. I hope version 19 does not give me 10kbit/sec...
And then I have tested tweaked versions:
stable openwrt-18.06 from https://forum.openwrt.org/t/build-for-netgear-r7800/316:
- average speed 394 Mbit/sec
stable openwrt-19.07 from https://forum.openwrt.org/t/build-for-netgear-r7800/316:
- could not be tested, package iperf3 not available
OpenWRT 18.06.4 with some tweaking (https://forum.openwrt.org/t/netgear-r7800-exploration-ipq8065-qca9984/285/1442):
- average speed 489 Mbit/sec
And tried some other suggestions found on the internet:
Software Flow offloading: no difference
Hardware Flow offloading: not available on r7800
Disable Firewall (iptables -F): slightly better performance (5%)
irqbalance: no change (edited after reply below)
What's next? What do I have to do to get 17.01.4 performance on 18.06.4?
On x86 target, 17.01 branch have ondemand scaling_governor, but it's missing from 18.06 branch, the default scaling_governor is powersave. You can checking by command:
Well, It seems that the problem in the realtek driver on my ubuntu pc with kernel 5.0. After i'm reinstall r8168-dkms driver, the speed > 900mbps on both direction. @stuffie what is the nic card on your computer? What is the nic driver version?
@blinton - did you read the solution from that thread? This is a hardware performance limitation if the packets originate or terminate on the router (the SoC is just not powerful enough to do this).
Yes, thanks a lot for the explanation / solution. Indeed, when using the router as "packet through" and not "originating" from it, I get easily full speed (ie 940 MBit/s) at all time, also for VLANs.
In my case, I do not see any scenario, where I would use the router with packets originating from it (without encryption).
