SQM not reaching 90% of the speed (R7800, 400/40mbit)

Installing and Using OpenWrt Network and Wireless Configuration
1

Hi all,

I just tried to configure SQM on my 400/40mbit connection.

So I followed the exact steps from:
https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm

And after doing a speedtest and reaching the 400/40mbit connection I filled in 360000/3600 as 90% of the speed.

When I run a speedtest how I only reach a max of 160mbit download which I dont understand.
The upload is spot on with 36mbit, what I would expect.

Any Ideas what it Could be?
I am using a Netgear R7800 with latest stable build from hnyman.

2

slow CPU

Try other speedtests - cloudflare, waveform, fast.com, fastest is closest to you.
Post waveform result link with all sqm disabled so we know what we are fixing.

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/sqm
cat /etc/config/firewall
3

Thanks for the answer. This is the waveform result without SQM:

And here is the result from the other commands:

{
        "kernel": "5.15.158",
        "hostname": "OpenWrt",
        "system": "ARMv7 Processor rev 0 (v7l)",
        "model": "Netgear Nighthawk X4S R7800",
        "board_name": "netgear,r7800",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "23.05-SNAPSHOT",
                "revision": "r23902-50148a40d2",
                "target": "ipq806x/generic",
                "description": "OpenWrt 23.05-SNAPSHOT r23902-50148a40d2"
        }
}


config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd68:bede:d515::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1.1'
        option ipv6 '0'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option delegate '0'
        list dns '127.0.0.1'
        option dns_metric '20'

config interface 'wan'
        option device 'eth0.2'
        option proto 'dhcp'
        option peerdns '0'
        option hostname '*'
        list dns '1.1.1.1'
        option dns_metric '50'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '1 2 3 4 6t'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '5 0t'

config device
        option name 'eth0.2'
        option type '8021q'
        option ifname 'eth0'
        option vid '2'
        option ipv6 '0'


config wifi-device 'radio0'
        option type 'mac80211'
        option path 'soc/1b500000.pci/pci0000:00/0000:00:00.0/0000:01:00.0'
        option channel '36'
        option band '5g'
        option htmode 'VHT80'
        option cell_density '0'
        option txpower '23'
        option country 'NL'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'Wireless Connection'
        option encryption 'psk2+ccmp'
        option key '*****'

config wifi-device 'radio1'
        option type 'mac80211'
        option path 'soc/1b700000.pci/pci0001:00/0001:00:00.0/0001:01:00.0'
        option channel '6'
        option band '2g'
        option htmode 'HT20'
        option cell_density '0'
        option txpower '20'
        option country 'NL'

config wifi-iface 'default_radio1'
        option device 'radio1'
        option network 'lan'
        option mode 'ap'
        option ssid 'Wireless Connection'
        option encryption 'psk2+ccmp'
        option key '******'


config queue 'eth0'
        option enabled '0'
        option interface 'eth0.2'
        option download '360000'
        option upload '36000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'ethernet'
        option debug_logging '0'
        option verbosity '5'
        option overhead '22'


config defaults
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'lan'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        list network 'wan'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config include 'bcp38'
        option type 'script'
        option path '/usr/lib/bcp38/run.sh'
4

Ok, you are week before 23.05.4 with your snapshot, not critical.
CPU is powerful enough.

Lets adjust your SQM interactively - set download rate to zero and set upload to half of subscribed (20000) and re-run the test - check "best" upload latency you can get, then splice halves up to 40000 (ie 30000, 35000 and so on) and when upload latency grows make step back.
Then repeat process with download (note it is 390 achieved target not 400...)
keep an eye on htop command - if 1 CPU core is steady 100% you may gain from "packet steering" checkbox or irqbalance.

1 Like
5

Hmm.. how can I already be behind? I downloaded this build yesterday and cant find a newer, did I miss something?

Tomorrow I will try as suggested. Strange thing is, how can it be such a big difference? Lets say, the speedtest is 390mbit and I put in 360mbit than the result can never be 160mbit right? Or will the wrong difference mass up things so bad?

6

Probably hnyman needs your feedback if mainstream release is 2x better or 2x worse than his build :wink:

1 Like
7

I am roughly building my old R7800 build once per month nowadays as I currently use MT6000 as main main router. The last 23.05 build is from 2024-06-19, from mid-June, well before 23.05.4 release tag.

I will soon build new builds for main/master and 23.05.
Personally I have been always using the master build in my own router.

But there shouldn't be any major performance related commits since June. (except a problematic mac80211 update and fix for it in 23.05, but your build is from before that.)

And there aren't any special performance tweaks in my build. The irqbalance package is included in the build, but like my docs say, it is initially disabled in config and the user may take it into use if he wants.

The reason might be a slow CPU (for those speeds) like @brada4 already speculated.
Especially if you use the default "cake" qdisc that is rather CPU hungry. You might get better results with SQM options "simple.qos" and "fq-codel" qdisc. It uses much less CPU than cake.

With simple.qos and fq_codel I have reached something like 330/150 Mbit in flent speed testing.

And you might consider if you need to spend any cycles on managing download speed at all...

1 Like
8

Thanks for the answer!

I am a bit confused because I thought CPU would be strong enough as brada mentioned later.

I will play around with the other settings as you suggested to check!

9

ipq8065 is not really fast enough for sqm a 400 MBit/s.

2 Likes
10

have to observe CPU usage, probably simplest of QoS will work, but if CPU hit 100% pre-sqm you have to start with irqbalance/steering (one of)

1 Like
11

Oké. I did a run with download rate to zero and upload to 20000.
https://www.waveform.com/tools/bufferbloat?test-id=ef75291f-b8d5-4f58-9198-3a9fa5a88471

Strange thing is. When I enable SQM my download speed drops, even with download set to zero. I checked CPU with HTOP and max usage was like 3%

Sorry, looked wrong. Indeed 1 core is @ 100% while running the test.. even with download set to zero it still goes to 100% on core 0. Explaining not reaching the 400mbit I think.

Also with SQM completely off, core 0 is reaching about 98/99% running the download speed test..

I have adblock enabled and Stubby using dnsmasq with cloudflare DoT, could that be an issue?

Edit 2: With software flow offloading enabled and SQM disabled core 0 reaches max 50%, but core 1 still does almost nothing. But result is much consistent test reaching 400Mbit

Edit 3: Could it be an idea to switch to the NSS build? And if so, can I just flash the sysupgrade over this version?

12

I made a new 23.05 and tested speeds with my 600/200 fiber with 2 ms latency. (due to low base latency, the possible latency increases are really visible)

Note: irqbalance in use.

23.05 wired SQM: off
B: 612/204 mbit, latency 2 ms / +28 / +15

23.05 wired SQM: 450/180 fq_codel simple.qos
A: 403/178, latency 2 ms / +10 / +0

23.05 wired SQM: 350/180 fq_codel simple.qos
A+: 352/181, latency 2 ms / +0 / +0

23.05 wired SQM: 350/180 piece_of_cake
B: 320/150, latency 2 ms / +2 / +30

23.05 wireless SQM: off
A: 294/178, latency 3 ms / +23 / +18

23.05 wireless SQM: 450/180 fq_codel simple.qos
A: 308/181, latency 3 ms / +13 / +2

23.05 wireless SQM: 350/180 fq_codel simple.qos
A+: 343/180, latency 3 ms / +3 / +1

23.05 wireless SQM: 350/180 piece_of_cake
C: 140/123, latency 3 ms / +6 / +134

23.05 wireless SQM: 300/140 piece_of_cake
A: 140/135, latency 3 ms / +6 / +13

Like earlier, my R7800 likes fq_codel & simple.qos much batter than cake.

That is even more amplified in flent speed testing, when you have simultaneous download/upload.

Ps. main/master is DSA based, while 23.05 is still swconfig based switch config, but there is no major speed difference. If I remeber right DSA is slightly slower.

1 Like
13

hack - Add br-wan so that offload goes to bridge and sqm works on wan adapter.

14

DSA was considerably slower, but ansuel found a way to speed it up considerably (pre-merge), now it's pretty much on par with swconfig.

15

There was (or still is?) a problem if offload-able protocol encap misses the hash then state is dropped to slowpath on every packet, coonntrack -E overrunning socket with one download... Manifesting with pppoe or if dsa parent port is added to zone. Still to guess more appropriate heuristic.

16

Switched to latest stable NSS build to give it a try and these are the results:

CPU cores both stay around 2%, looks much better now. Now I need to get SQM working and check if I can make it even better

17

So it was only cpu

18

Looked like indeed, but strange thing is that it also looked like the CPU didn't balance the work between the two cores. Or is this normal behavior?

Other question left now, I tried to enable SQM again on this NSS build to try, but doesn't change anything in speed so looks like it is not working?

Edit: Think I know why:

Known Issues:

  • Luci SQM and normal SQM config doesn’t work. Have to use custom config.

Now the question is: what is custom config? :slight_smile:

19

No idea, propose to try br-wan.

20

That is normal, unless you enable the irqbalance service that tries to (partially) balance IRQ usage between cores. (Note that Irqbalance is originally from x86 world, and its heuristics classify ARM IRQs only partially. e.g. wifi is not balanced)

NSS and software offload (and hardware offload, not in R7800) all aim to bypass the kernel netfilter processing to which SQM is tied, either via hardware (NSS, hardware offload) or software.