Slow 5 GHz downloads on R7800

dale · March 30, 2020, 6:55am

I've just got a R7800, and I promptly put OpenWrt 19.07.2 on it. From a 15" 2019 MacBook Pro on 5 GHz I can do an iperf3 test sending data to a wired machine on my LAN at ~750 Mb/s, which is great!

However, going the other direction, from that same wired LAN server to my MacBook Pro on 5 GHz using iperf3, I only get about 330 Mb/s.

When I reflash the R7800 with the stock Netgear firmware, I get easily 700 Mb/s going from wired → wireless under the same conditions—same laptop, all equipment in the same position, same network adapters, same cabling, same software, etc. (Wireless → wired remains fast on the stock firmware, still in the ~750 Mb/s range.)

To confirm my iperf3 LAN findings, I also tested with speedtest.net. I was able to confirm much slower wired → wireless ("download") on OpenWrt: ~300 Mb/s download with OpenWrt, versus 500+ Mb/s download with stock firmware.

Does anyone have any suggestions on how I can get OpenWrt 5 GHz wireless → wired speed to match the stock firmware? This Netgear firmware is scary and I'd much rather be running OpenWrt.

Things I have tried to improve the wired → wireless on 5 GHz:

Using the performance governor as shown (for example) at #2 here (though I think perhaps the switch to 800 MHz then back to 1.7 GHz is unnecessary in 19.07.2)
ethtool changes to tx-usecs and rx-usecs, also as in prior link (I think these are meant to affect latency, not bandwidth, but I could be wrong)
Manually setting smp_affinity for eth0 and eth1 to second CPU
Running irqbalance
Using the latest ath10k CT firmware, firmware-5-ct-full-community-12.bin-lede.017 (replaced firmware-5.bin and rebooted, confirmed expected firmware loaded via dmesg)
Using the latest ath10k CT HTT firmware
Changing the rps_cpus/xps_cpus values for all devices and queues (tried 2 and 3 for all, separately, and did it via SSH after boot, so after /etc/hotplug.d/net/20-smp-tune has run)
Tried hnyman's latest master build

I have run htop while doing my tests, and max CPU usage on any core is typically 50–80% at most. (When testing wireless sending data to wired with iperf3, where I get ~750 Mb/s, I do see one core—just one—getting into the 80%–90% softirq usage range, which I take to mean that the faster wireless → wired transfer is CPU-bound, but wired → wireless is not CPU-bound.)

I did test with another MacBook Pro, a 15" 2013 Retina I had, which was sitting about one foot diagonally left and upwards of the 2019 MBP. This machine was able to get higher speeds, in the range of maybe ~450 Mb/s iperf3 wireless → wired, but of course this is still less than 700 Mb/s. I have no explanation for the difference.

I have configured the R7800 as a "dumb AP":

The firewall, miniupnpd, udhcpd, and dnsmasq services are all turned off.
VLAN is turned off in the switch settings.
Wireless is bridged to wired.
There should be no NAT going on at all, and IIRC iptables was empty, both the filter and nat tables.
The WAN Ethernet port is not connected.
I did turn on software flow offloading, which I don't think matters for bridging.
I turned off legacy data rates for wireless.
I have made sure to log out of LuCI while testing, in case its refreshing would interfere with wireless performance somehow.
Both the 2.4 GHz and 5 GHz wireless radios are turned on, with different ESSIDs, if that matters.
Encryption was set to WPA2-PSK CCMP.

Other than the settings I've mentioned, I left most others at their defaults, such as 80 MHz channel width on 5 GHz. Region is set for device default, which is (AFAIK, at least) US.

The one thing I can think of that I might have screwed up is the 5 GHz wireless channel. I might have left it to "auto" in some cases, and then 36 and/or 48 in other tests. I guess I can go back and test that, but it would be a PITA. (If someone knows "yeah stock is going to be faster than OpenWrt in this case, nothing you can do about it", then I'm grateful for you saving me another hour of reflashing, reconfiguring, testing, etc.)

In case it matters, if I go wired Ethernet on my MacBook, plugged into the R7800, I get near gigabit speeds (~940 Mb/s) in both directions, which is exactly as expected.

mooninite · March 30, 2020, 4:37pm

My R7800 requires certain channels to get fast performance. On 2.4Ghz I have to use channel 11 (20Mhz). On 5Ghz I have to use channel 153 (80Mhz). If I use other channels performance is very low.

I don't have any close neighbors and I only use one AP in my residence.

dale · March 30, 2020, 6:12pm

Thanks for the info. Mine defaulted to 153, so I'm pretty sure I did at least some tests on that channel, but I can go back and check again.

Where my Mac currently sits, it sees twelve 5 GHz networks and twelve 2.4 GHz networks. Only two of those are mine, so that's certainly much more crowded than I'd like.

perceival · March 31, 2020, 10:28am

Try to play with 802.11w and KRACK. In my case when enabled caused strange behavior of IOS devices.

ACwifidude · March 31, 2020, 1:42pm

Anywhere in the 500-600mbps range is great for a wireless AC connection with a 2x2 client. This is an appropriate rate for a 2x2 client once you take in to account wifi overhead and radio physics- here is a good read on wifi 5 and hype vs reality (especially look at the PHY chart): https://www.duckware.com/tech/wifi-in-the-us.html#wifi5

I am running more aggressive ondemand CPU settings, Irqbalance enabled, and my wifi settings are below (prime number beacon interval, 802.11r enabled, software offloading enabled, manually lowered the transmission power to best fit my environment, and psk2+ccmp encryption (force ccmp option on luci, don’t run tkip)). I regularly get mid 400mbps to mid 500mbps via wireless on speedtest (depending on coronavirus neighbor internet usage times, this is a mid-morning speedtest from 20ft away open air with an iphone 2x2 client):

Netgear stock firmware has full use of all proprietary software and the extra two NSS CPUs (full hardware offloading/acceleration). Netgear stock firmware will be a smidge faster on bandwidth benchmark testing (But not on bufferbloat!). This hair of difference is marginal / not significant. OpenWRT features, interface, and superior bufferbloat control give a far better internet experience.

I’m running hynman’s latest master build-

root@OpenWrt:~# uname -a
Linux OpenWrt 4.19.108 #0 SMP Sun Mar 29 18:09:58 2020 armv7l GNU/Linux

Latest CPU tweaks I’ve been trying.


echo 600000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq; echo 600000 > /sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq; echo 25 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold; echo 50 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor

root@OpenWrt:~# cat /etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option hwmode '11a'
        option path 'soc/1b500000.pci/pci0000:00/0000:00:00.0/0000:01:00.0'
        option htmode 'VHT80'
        option channel '161'
        option txpower '22'
        option legacy_rates '0'
        option country 'US'
        option beacon_int '67'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ft_over_ds '1'
        option ssid '********'
        option ft_psk_generate_local '1'
        option key '********'
        option ieee80211r '1'
        option encryption 'psk2+ccmp'

dale · March 31, 2020, 8:09pm

As in, turn them on? I think both are off by default, yes? I left them at their defaults.

dale · March 31, 2020, 8:16pm

Thank you for posting your configs! I'll take a look at these and see if I can glean anything from them.

To be clear, I'm using the R7800 as an access point only, it's not doing any routing or NAT. I'm not sure if bufferbloat applies in the case where I'm just bridging wireless and wired together? I'll have to look into that, and also into what the "NSS CPUs" are.

Like I said, I'd much rather be running OpenWrt just so I know what's running on the device (dear god why is the Netgear firmware intercepting routerlogin.net, even in "AP mode"). But the ~200 Mb/s speed difference is significant for me, and so I'm unfortunately on stock firmware for now.

perceival · April 1, 2020, 5:15am

Indeed. Those options aren't enabled by default.
I would also check if option Allow legacy 802.11b rates is disabled.
Shouldn't cause such a behavior but just in case.
You can also play with log_level and check if anything interesting will appear in the logs.

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11a'
	option path 'soc/1b500000.pci/pci0000:00/0000:00:00.0/0000:01:00.0'
	option legacy_rates '0'
	option country 'PL'
	option htmode 'VHT80'
	option channels '64 116'
	option log_level '0'
	option channel '116'
	option txpower '5'

jam01 · April 15, 2020, 6:17pm

@dale did you find time to dig further into this?

I bought a R7800 about 6 months ago with the full intention of installing OpenWRT eventually. I'm starting to find the time to dive into this, and would be interesting to know what the performance diff between stock and 19.07 others are seeing at this point. Though I will be using it as a router...

dale · April 16, 2020, 12:27am

No, I'm afraid I have not had time to dig back into this.

ACwifidude · April 16, 2020, 1:14am

Stock will probably have the edge on throughput performance due to proprietary hardware acceleration from the NSS CPU cores.

The difference in real world downloads is small. One member did a very large download over nearly 8 minutes and found only a couple seconds difference in download time:

Synthetic benchmarks I’ve seen up to a 100-200mbps difference reported on 5ghz. My speed test of 453 mbps download is above with a 2x2 iphone client in the middle of the morning during these lovely COVID times (I’ve had speeds in the low 500’s during off hours): http://www.dslreports.com/speedtest/61437431

OpenWRT gets you 802.11r, a much better interface, better latency / bufferbloat control, and a variety of other features over stock. I prefer openWRT.

perceival · April 16, 2020, 6:09am

I must admit my impression about OpenWrt is it's getting slower and slower. Before kernel 4.19 I was able to achieve speeds of 55MiBps. Now with 5.4 kernel and v17 of @gearbest CT firmware it's barely reaching 40MiBps. BTW wired speeds are also lower.
For me some improvement was possible with:

Packet Steering enabled (I guess this is new option)
and the following CT firmware version:
firmware ver 10.4b-ct-9984-fH-013-b63cea875 api 5 features mfp,peer-flow-ctrl,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,htt-mgt-CT,set-special-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 435d03f1

mrremo · April 19, 2020, 5:20am

I am using @hnyman latest master build and can't find any problems.
With iperf3 I have determined the following values:

LAN --> 940Mbits/sec
5GHz --> 450-500MBits/sec

These look pretty good to me.

perceival · April 19, 2020, 12:18pm

I have doublechecked and indeed. On 5GHz I am getting:
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 672 MBytes 564 Mbits/sec 0 sender
[ 5] 0.00-10.01 sec 669 MBytes 561 Mbits/sec receiver
But only when laptop acts as iperf3 client. If I am doing the same in the opposite direction the result is far worse:
[ 5] 0.00-10.01 sec 342 MBytes 286 Mbits/sec receiver
Also download of a file is almost 30% slower than the upload.
Any idea?

ACwifidude · April 19, 2020, 1:04pm

The router’s hardware will limit iperf throughput testing. Use two x86 computers (one as the client, one as the server) with the router in between to get accurate results both ways.

The higher throughput from your testing represents reality (the router is passing on traffic ~560 mbps , not generating traffic ~286 mbps).

perceival · April 19, 2020, 4:09pm

I believe it's more complex. Below output of a few tests. Wired is kind of good and stable and generally in terms of WLAN Windows sucks. Other than that I have no idea.
Bold numbers for WLAN. Mbits/s. Intel7265.

	wl w10	eth w10	eth arch	r7800	wl arch
client
wl w10		113	113	134
eth w10	120		883	856	290
eth arch	99	926		909	391
r7800	103	899	859	827	405
wl arch		392	554	563

perceival · April 19, 2020, 4:23pm

What kind of client you have used for the test? Any specific settings on the router/ap?

mrremo · April 19, 2020, 5:12pm

Wlan settings on ap are basicaly default (5GHz, 80MHz).
Client is a XPS13 9350 with Fedora 31, also default settings.
So, no tweaks, just deault 🤷...

_FailSafe · April 19, 2020, 6:08pm

I am experiencing the same thing. Currently running @hnyman's r12927 build.

Test client is a 16" MacBook Pro, so 3x3 AC capable. My MBP indicates a TX rate of between 1170-1300 Mbps. RSSI is -39 to -41 dBm where I'm testing from and noise is -92 dBm.

Wireless is obviously AC 5GHz @ VHT80, channel 153.

I have a beefy pfSense router/firewall <----> r7800 <---> MBP client. My r7800 is operating as a dumb AP in my current setup. When running iperf3 from my MBP to my pfSense host, I see the following via ethernet:

$ iperf3 -c 192.168.45.5 -i1 -V
Connecting to host 192.168.45.5, port 5201
      TCP MSS: 1448 (default)
[  5] local 192.168.45.150 port 64900 connected to 192.168.45.5 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   113 MBytes   944 Mbits/sec
[  5]   1.00-2.00   sec   112 MBytes   942 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   942 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   942 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   941 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   941 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   941 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   942 Mbits/sec
[  5]   8.00-9.00   sec   112 MBytes   942 Mbits/sec
[  5]   9.00-10.00  sec   112 MBytes   942 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec                  sender
[  5]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver
CPU Utilization: local/sender 37.9% (7.0%u/30.9%s), remote/receiver 9.8% (1.5%u/8.3%s)

$ iperf3 -c 192.168.45.5 -i1 -V -R
Connecting to host 192.168.45.5, port 5201
Reverse mode, remote host 192.168.45.5 is sending
      TCP MSS: 1448 (default)
[  5] local 192.168.45.150 port 64894 connected to 192.168.45.5 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   112 MBytes   939 Mbits/sec
[  5]   1.00-2.00   sec   112 MBytes   941 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   936 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   942 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   941 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   941 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   941 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   942 Mbits/sec
[  5]   8.00-9.00   sec   112 MBytes   942 Mbits/sec
[  5]   9.00-10.00  sec   112 MBytes   941 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec  266             sender
[  5]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver

But performing the same test via WiFi on my MBP (from 6 feet away and clear line of site) looks like this:

$ iperf3 -c 192.168.45.5 -i1 -V
Connecting to host 192.168.45.5, port 5201
      TCP MSS: 1448 (default)
[  5] local 192.168.45.145 port 65086 connected to 192.168.45.5 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  44.7 MBytes   375 Mbits/sec
[  5]   1.00-2.00   sec  47.1 MBytes   395 Mbits/sec
[  5]   2.00-3.00   sec  38.1 MBytes   319 Mbits/sec
[  5]   3.00-4.00   sec  40.8 MBytes   342 Mbits/sec
[  5]   4.00-5.00   sec  41.0 MBytes   344 Mbits/sec
[  5]   5.00-6.00   sec  42.4 MBytes   355 Mbits/sec
[  5]   6.00-7.00   sec  45.3 MBytes   380 Mbits/sec
[  5]   7.00-8.00   sec  47.1 MBytes   395 Mbits/sec
[  5]   8.00-9.00   sec  48.5 MBytes   407 Mbits/sec
[  5]   9.00-10.00  sec  47.5 MBytes   399 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   442 MBytes   371 Mbits/sec                  sender
[  5]   0.00-10.01  sec   442 MBytes   371 Mbits/sec                  receiver
CPU Utilization: local/sender 26.0% (4.1%u/21.9%s), remote/receiver 14.9% (2.1%u/12.8%s)

$ iperf3 -c 192.168.45.5 -i1 -V -R
Connecting to host 192.168.45.5, port 5201
Reverse mode, remote host 192.168.45.5 is sending
      TCP MSS: 1448 (default)
[  5] local 192.168.45.145 port 65089 connected to 192.168.45.5 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  25.6 MBytes   215 Mbits/sec
[  5]   1.00-2.00   sec  28.1 MBytes   235 Mbits/sec
[  5]   2.00-3.00   sec  29.8 MBytes   250 Mbits/sec
[  5]   3.00-4.00   sec  30.6 MBytes   256 Mbits/sec
[  5]   4.00-5.00   sec  31.5 MBytes   264 Mbits/sec
[  5]   5.00-6.00   sec  33.4 MBytes   280 Mbits/sec
[  5]   6.00-7.00   sec  34.7 MBytes   291 Mbits/sec
[  5]   7.00-8.00   sec  36.4 MBytes   305 Mbits/sec
[  5]   8.00-9.00   sec  40.1 MBytes   337 Mbits/sec
[  5]   9.00-10.00  sec  41.0 MBytes   344 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.08  sec   332 MBytes   276 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   331 MBytes   278 Mbits/sec                  receiver

FWIW, I know some will look at this and say "oh that's still pretty good WiFi performance for internet usage" and I don't disagree generally. But I use my WiFi for more than internet. I have a NAS that all my laptops use for backups multiple times per day, so when backing up multiple gigabytes of data this speed really starts to feel bogged down.

fantom-x · April 19, 2020, 6:23pm

Your wifi testing results match mine perfectly: max download at a little over 300Mbps and upload is a under 300Mbps. You said that you notice slow downs during the backups: it is my experience and the 300Mbps throughput is combined if uploads and downloads over wifi are happening at the same time. Did you run this test?