SQM on Netgear R7800 problem

I would. I have noticed earlier that cake is CPU intensive, so I highlight the CPU issue. High CPU load from qdisc might slow down the other functions of the router.

But in the end it depends on your use case, the download practices and the other load for the router. Just try both for some time and pick the one that feels best...

Personally I use simple/fq_codel, but I have something like 190/19 Mbit, so my SQM needs are rather modest (with normal browsing & office apps).

With a r7800 the best settings for the balance of throughput and latency on (speed test using master 4.19 ~2 months ago) the following:

fq_codel + simplest.qos + software offloading enabled + more aggressive CPU settings:

fq_codel + simplest_tbf.qos + software offloading enabled + more aggressive CPU settings:


root@OpenWrt:~# echo 35 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold;
 echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth0'
        option interface 'eth0.2'
        option qdisc 'fq_codel'
        option qdisc_advanced '0'
        option enabled '1'
        option download '540000'
        option debug_logging '0'
        option verbosity '5'
        option script 'simplest_tbf.qos'
        option linklayer 'none'
        option upload '34000'

I usually run with SQM only on the upload side (Same settings as above - just zero for download):

1 Like

In your last result, I'm pretty sure the bufferbloat test component of the dslreports test has been failing for a lot of people, lately, and I'm not sure why. It could be the link getting slammed so hard when you are not also inbound shaping. I'd be pretty interested in cakes' behavior with ack-filter on on the uplink and no download shaping....


root@OpenWrt:~# uname -a
Linux OpenWrt 5.4.33 #0 SMP Sun Apr 19 07:26:12 2020 armv7l GNU/Linux

r7800, Performance CPU governor + software flow offloading enabled + irqbalance enabled + packet steering enabled + these SQM settings:


root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth0'
        option interface 'eth0.2'
        option enabled '1'
        option debug_logging '0'
        option verbosity '5'
        option linklayer 'none'
        option upload '32500'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option qdisc_advanced '1'
        option squash_dscp '1'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '1'
        option eqdisc_opts 'docsis ack-filter nat'
        option download '0'

r7800, Performance CPU governor + software flow offloading enabled + irqbalance enabled + packet steering enabled + these SQM settings:


root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth0'
        option interface 'eth0.2'
        option enabled '1'
        option debug_logging '0'
        option verbosity '5'
        option linklayer 'none'
        option upload '32500'
        option download '0'
        option qdisc 'fq_codel'
        option script 'simplest_tbf.qos'
        option qdisc_advanced '0'

Analysis: Middle of the afternoon download speed is a little below normal due to usual increased network load. Cake is grouchy about software offloading and/or 0 for download. :man_shrugging:

to me it looks like yer running out of cpu, and should stick to tbf + fq_codel

and one of these damn days we should summon the moxie to inbound shape across cores.

And outbound, pls: some of us are suffering with symmetrical fibre links :wink:

How big is the suffering on symmetrical fiber?

if someone would toss at least $4k/month into https://www.patreon.com/dtaht I'd get back more F/T in fixing bloat issues. Living in yurt got old multiple years back. Terribly chilly in the winters.

1 Like

There was a smile in there. It will cost way less to throw a Xeon based server at the problem :wink:

and good for intel's sales too! :confused:

Another useful feature all these offload engines could would be a programmable completion interrupt for tx. You'd set that to what you would shape to, and completely eliminate the cpu cost of outbound shaping by treating it (with backpressure) as, (for example) a 35Mbit/sec interface.

There are a few ethernet devices that can do this, including one from intel, but I don't rmeember which one.

The use case for a symmetrical line is to work with the cloud (storage in particular): I have to upload/download a lot of data often. With fq_codel/simplest_tbf this router cannot reliably shape more than 200Mbps on the uplink on a good day (download is not shaped), so I am loosing a lot of bulk upload bandwidth in order to be able to have several video calls going through the router at the same time.
Having said that, this router might be the wrong tool for the job to begin with.

What does dsl speedtest look like with 250000 / 250000 with aggressive CPU settings (or performance governor) with zero overhead and no advanced SQM settings?

I will do it later, but keep in mind that it is not a real-life test: it tests either a download or an upload, but not both at the same time.

I am beginning to think that PPPoE and applying VLAN on WAN are taking more resources than I thought.

Not bad. Agreed- Going to need a little more CPU power to get both upload and download consistently at that level.

I have one r7800 acting purely as a wired router and two r7800s as pure APs. Nicely distributes the load it seems. Wired router is running hnyman master build- all features turned off (the default), just CPU tweaks. Even with everything as optimized as I can make it - I can shape a max of 500mbps total (thus why I wanted to see what 250 / 250 did for ya).

Might get a couple more mbps without the extras. :man_shrugging:

SQM: 122550(down); 5700(up); cake; piece_of_cake.qos; Tweaked Ack:

Ok, i tested the following:
SQM: 122550(down); 5700(up); fq_codel; simplest_tbf.qos; software offloading enabled; tweaked CPU, tweaked Ack

SQM: 122550(down); 5700(up); fq_codel; simplest.qos; software offloading enabled; tweaked CPU, tweaked Ack

Looks like tbf +fq_codel gives you the best result. Ack I think is just a cake feature so you can turn it off. Much better bufferbloat compared to your baseline. Enjoy!

Thanks for all that helped. So I use the following setting now:
sqm

config queue 'eth1'
	option debug_logging '0'
	option verbosity '5'
	option qdisc_advanced '0'
	option linklayer 'ethernet'
	option overhead '22'
	option download '122550'
	option upload '5700'
	option interface 'eth0.2'
	option enabled '1'
	option qdisc 'fq_codel'
	option script 'simplest.qos'

/etc/rc.local

echo 35 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold
echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
/usr/sbin/irqbalance

Firewall

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option flow_offloading '1'

Are there any other tweaks that should be used with my setup?

SQM settings look good. You can use these commands to check if your CPU settings are working and that irqbalance is enabled (usually have to restart after putting it in the rc.local file)

cat /sys/devices/system/cpu/cpufreq/ondemand/up_threshold

cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor

cat /etc/config/irqbalance

I run packet steering too to balance the network load:

uci set network.globals.packet_steering=1; uci commit network

You can check the global settings here after you enable it:

cat /etc/config/network

Those are the best tweaks I’ve found for the r7800. I’ve been able to shape up to 450-500mbps total. The next major breakthrough is waking up the NSS CPU cores to get hardware acceleration working, more work needed there but quarky has done great work:

Thanks again. It all seems fine. However, I think irqbalance in not enabled. I put the following into the rc.local section in luci:
/usr/sbin/irqbalance
Rebooted and checked the results as follows:


root@OpenWrt:~# cat /sys/devices/system/cpu/cpufreq/ondemand/up_threshold
35
root@OpenWrt:~# cat /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
10
root@OpenWrt:~# cat /etc/config/irqbalance
config irqbalance 'irqbalance'
        option enabled '0'

        # The default value is 10 seconds
        #option interval '10'

        # List of IRQ's to ignore
        #list banirq '36'
        #list banirq '69'

root@OpenWrt:~# cat /etc/config/network
[...]
config globals 'globals'
        option ula_prefix 'abcd:efgh...:/48'
        option packet_steering '1'
[...]

Run this command and your irqbalance will be enabled:


uci set irqbalance.irqbalance.enabled=1; uci commit

Using uci will survive soft reboots and should balance our your irqs. :sunglasses: