R7800 SQM reduce speed to less than half with fiber

Installing and Using OpenWrt
1

Hello,

I have just installed the latest version of OpenWRT for the R7800:
Model : Netgear Nighthawk X4S R7800
Architecture : ARMv7 Processor rev 0 (v7l)
Firmware Version : OpenWrt 19.07.6 r11278-8055e38794 / LuCI openwrt-19.07 branch git-21.018.57536-6ba9740
Kernel Version : 4.14.215

Without SQM I have around 900Mbps Down and 100Mbps Up, with SQM I'm getting the following results:

  1. fq_codel - simplest_tbf --> 400Mbps / 84Mbps
  2. cake - piece_of_cake.qos --> 200Mbps / 84Mbps

I didn't had time to test more combinations at the moment, but in both cases the speed is really low from what I'm getting from my ISP in actual.

Any advice on that issue will be helpful.

Many thanks,

2

You are going to saturate CPU usage with a pipe that fat using SQM. First question is do you need SQM? Try the dslreports bufferbloat test to verify. If so, I recommend that you try a NSS build for the R7800. Pretty amazing results.

That link I posted is probably not the best. Search on my name and NSS or bufferbloat to find me. I do not have time to provide them to you right now. Another but again not what I was looking for. See further down that thread for build instructions and a .config that works as well as how to enable SQM with NSS.

EDIT:
Link to my diffconfig

Here is how I get the NSS code rebased against the current master:

git remote add nss https://github.com/Ansuel/openwrt.git

git fetch nss && git checkout -b my-nss nss/kernel5.4-nss-qsdk10.0 && git revert e168f9106a74bfd33ccca2a5f867652fb9caf073

git checkout master && git checkout -b my-master && git rebase my-nss

git add target/linux/ipq806x/files/include/net/netfilter/nf_conntrack_dscpremark_ext.h target/linux/ipq806x/files/include/uapi/linux/tc_act/tc_nss_mirred.h target/linux/ipq806x/files/net/netfilter/nf_conntrack_dscpremark_ext.c

vim target/linux/ipq806x/files/arch/arm/boot/dts/qcom-ipq8064-c2600.dts

git add target/linux/ipq806x/files/arch/arm/boot/dts/qcom-ipq8064-c2600.dts
git rebase --continue

Note you want to delete everything from HEAD to the end in that last file or else it doesn't compile.

Alternatively, I believe others here are offering rebased code or maybe even unofficial images including NSS.

Once you have the build flashed, you use a modified set of values from my settings here as needed to enable the nssdq_codel. Adjust to your speeds. Also disable the SQM you setup in LuCI since it doesn't work with NSS anyway.

# cat /etc/rc.local

# irq balance between CPUs
[[ -x /usr/sbin/irqbalance ]] && /usr/sbin/irqbalance

modprobe nss-ifb
tc qdisc add dev nssifb root handle 1: nsstbl rate 700Mbit burst 1Mb
tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

#tc qdisc add dev eth0 root handle 1: nsstbl rate 22Mbit burst 1Mb
tc qdisc add dev eth0 root handle 1: nsstbl rate 22Mbit burst 100k
tc qdisc add dev eth0 parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default
ip link set up nssifb

exit 0
3

Thank you @darksky for the detailed response, since I'm pretty new to this I still need go over what you wrote several times :slight_smile:

Regarding your question, I installed OpenWrt because the issues I had when me and my wife are working from home, we have a lot of minor internet lose which are not coming from our ISP. It's like few sec' of no internet and then it's ok again. When using OpenWrt with SQM I don't see this issue.

Actually I should try without SQM and see if maybe the installation of OpenWrt was enough, but I suspect that the device does not handle the internet load when both of us working on the same time - a lot of meetings and cloud work (some is done over VPN).

Note that I'm using a wired connection while my wife is using the wifi and our kids are at home as well watching Netflix mostly :wink:

It's all an assumption and maybe I'm wrong, but I don't know how I can identity the issue.

Edit:
I have disabled SQM.
I'm getting (wired) Overall: A+ | BufferBloat: A+ | Quality: A
While over wifi we are getting: C | C | C

4

I can't remember details but does SQM allow you to select the interface on which to apply it? Do you need to apply it to wireless iface I wonder. Check wiki or see if someone else replies. I disabled the SQM LuCI interface since I am doing it via NSS and there is currently no LuCI module for NSS SQM.

5

Can you post a link to a dslreport wired speed test result without SQM? Set DSLreports settings to use high resolution bufferbloat I will take a look at the detailed report and see how your link is working. You might not need SQM at all, or you might be fine with only SQM on the upload direction.

WiFi is inevitably worse latency without some of the more recent experimental stuff that's only available on the ath10k as far as I know. When doing interactive meetings USE A WIRE it's just a way better solution. Buy a switch if you need more ports.

6

wait a minute.. are you saying that you reach gigabit ("around 900 Mbps") speed on a r7800 with standard openWRT without NSS support? What build are you using, exacly? I had understood that it was not possible.

7

You can't reach line speed (942 Mbps). The best you can hope for a stock image (recent snapshot) is 800 Mbps down and 910 Mbps up. The only tweaks are installation of irqbalance and enabling software flow offload. OTOH, the in-kernel SMB server (ksmbd) does reach line speed when serving files from an SSD connected via USB.

8

  1. @wireless you are right, I probably had in my mind the previous speed before using OpenWRT. I'm getting the following results with OpenWRT and SpeedTest:
    https://www.speedtest.net/result/10840816249

  2. @dlakelan find below my DSL report run with high resolution bufferbloat:
    http://www.dslreports.com/speedtest/67174434

I'm going to read now how to get faster speeds with OpenWRT...

Thank you all!

9

Was that without SQM or with? It's a strange set of periodic spikes in delay.

10

@dlakelan yes both without SQM, basically using the default OpenWrt settings and this startup settings that I was playing with and forgot to remove:

# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor

exit 0
11

I do not believe the performance governor is needed. Either ondemand or schedutil should throttle up upon load.

12

depends. on some of these devices the power consumption is not really any lower with ondemand and the time it takes to throttle up can absolutely adversely affect the latency under load, you can see a couple seconds of bufferbloat at the start of large downloads/uploads, which if you're a gamer would be unpleasant. But for everyday browsing it is probably fine to use ondemand.

13

@dlakelan I removed the performance governor to see if I see any differences after using it the entire last week. Regarding your sentence about gamer, how does it compare to two people using zoom (with video) on the same time, as most of the issues are coming during those video calls?

14

The issue is that when a sudden large download or upload occurs, the flood of packets that comes in can be too much for the slower speed CPU to keep up with. But it can take several hundred milliseconds to ramp up the processor speed to handle the flood. If the flood is something that's periodic, like what you get when you stream a Netflix/YouTube show and every few seconds it grabs a chunk of buffer, then it can be the case that the governor never really gets up to sufficient speed and you're seeing bufferbloat and dropped packets even though when at full speed, your router would handle it fine. If you experience that kind of issue, then use performance governor.

15

Please note that SQM's traffic shaper itself is bursty but often causes not enough sustained load for tge ondemand governor to ramp up. You can either play with ondemand's control paramters to make it more reactive or use the performance governor.

16

that is interesting... last tests I did with an almost stock image - @hynman one - reached 600, 650 Mbps.

17

Following this thread, I have enabled Software flow offloading and I'm getting nearly 800Mbps / 100Mbps without SQM.

Hope it will help somehow, since wife issues with the internet still happens. Seems like streaming Netflix / Playing Xbox while I'm on a Zoom call as well preventing her from getting access - i.e. she have total lost of connection. When she do have a connection she get around 180Mbps/90Mbps but with BufferBloat C on her Macbook Pro 13 2020. I still can't pin the main issue...

I have checked on the same location as she is, with my Lenovo T14 Gen1 Ubuntu 20.04 and I get around 300Mbps / 100Mbps with BufferBloat B.

I'm starting to think the issue is with the Mac NIC...