SQM enabled has Capped Download Speeds

Using a NETGEAR Nighthawk X4S R7800 with the 21.02.0-rc4 installed. I have read through R7800 performance - #251 by ACwifidude but from what I gather this mostly has to do with Wifi issues. The whole reason I hooked this router up and updated the firmware to use OpenWrt was to enable SQM as I had another router (TP-Link Archer A20 V3) that was having serious latency issues, especially under load. Well setting up this router was easy enough, so was setting up SQM, however it seems that I am capped at 5-10mpbs download on a 1000mpbs line with no one else using it. I have played around with a few things, but this is all very new to me and I am feeling a bit out of my depth as this is my first crack at ever using any alternative firmware for a router. Thanks in advance.

I disabled SQM and I am not getting 350ish on the download. I am not sure why SQM lowers the download as much as it does.

Did you already read and follow the instructions in the documentation about SQM? (and maybe the additional reading linked from there) https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm

it seems to me you have set a wrong value in "download" and "upload" speeds, and SQM is capping you to that. You should test your speed without SQM, then put your speed values in SQM so it does not cap you anymore.

Then you can test and see if latency is better, and do some more tuning as explained in the article

Afaik SQM isn't a magic thing you can just click on and it works, you need to run tests and tune the numbers, there is no "default settings" that work for everyone.

1 Like

Yes I did. I changed the values multiple times. I made sure I was using the correct values at roughly 90% of the speed. I followed the video link that shows how to install it. I played around with it a bunch but every time I did a speed test it was always 5-10mpbs download. Though it was odd that upload speed was never effected. My guess is that the 21.02.0-rc4 version did not play nice with this router and required much more tweaking than I knew how to do.

You may have applied SQM to the wrong interface. With R7800 the correct wan interface is usually eth0.2 (with the default network settings in 21.02 and current master).

My R7800 does nicely the 200/60 that my ISP gives me.

Please show your /etc/config/sqm file.

Ps. You might try my R7800 community build that has the SQM interface setting set ok by default.

1 Like

Please post the output of the following commands:

1) ifstatus wan

2) tc -s qdisc

3) cat /etc/config/sqm

That might help to better understand the issue.

Do you have a link for that?

Also I was out of town, sorry for the late reply. I ended up using a build from ACwifidude from this post: Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

Since I am using that one I have no tried SQM again, but I would be willing to if I could find the one that Hnyman mentioned.

The build has been running since late 2016: (the second-most read thread in the forum)

ifstatus wan

{
        "up": true,
        "pending": false,
        "available": true,
        "autostart": true,
        "dynamic": false,
        "uptime": 1337,
        "l3_device": "eth0.2",
        "proto": "dhcp",
        "device": "eth0.2",
        "updated": [
                "addresses",
                "routes",
                "data"
        ],
        "metric": 0,
        "dns_metric": 0,
        "delegation": true,
        "ipv4-address": [
                {
                        "address": "96.19.178.237",
                        "mask": 24
                }
        ],
        "ipv6-address": [

        ],
        "ipv6-prefix": [

        ],
        "ipv6-prefix-assignment": [

        ],
        "route": [
                {
                        "target": "0.0.0.0",
                        "mask": 0,
                        "nexthop": "96.19.178.1",
                        "source": "96.19.178.237/32"
                }
        ],
        "dns-server": [
                "24.116.0.53",
                "24.116.2.50"
        ],
        "dns-search": [

        ],
        "neighbors": [

        ],
        "inactive": {
                "ipv4-address": [

                ],
                "ipv6-address": [

                ],
                "route": [

                ],
                "dns-server": [

                ],
                "dns-search": [

                ],
                "neighbors": [

                ]
        },
        "data": {
                "leasetime": 86400,
                "timezone": -25200
        }
}

tc -s qdisc

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 408714500 bytes 611230 pkt (dropped 0, overlimits 0 requeues 6)
 backlog 0b 0p requeues 6
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 408714500 bytes 611230 pkt (dropped 0, overlimits 0 requeues 6)
 backlog 0b 0p requeues 6
  maxpacket 1494 drop_overlimit 0 new_flow_count 11 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root
 Sent 554473541 bytes 502817 pkt (dropped 0, overlimits 0 requeues 383)
 backlog 0b 0p requeues 383
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 554473541 bytes 502817 pkt (dropped 0, overlimits 0 requeues 383)
 backlog 0b 0p requeues 383
  maxpacket 1514 drop_overlimit 0 new_flow_count 244 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8009: dev eth0.2 root refcnt 2 bandwidth 45Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 202258568 bytes 371306 pkt (dropped 305, overlimits 122393 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 290688b of 4Mb
 capacity estimate: 45Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         45Mbit
  target            5ms
  interval        100ms
  pk_delay        394us
  av_delay         27us
  sp_delay          3us
  backlog            0b
  pkts           371611
  bytes       202698256
  way_inds            9
  way_miss          409
  way_cols            0
  drops             305
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len         37150
  quantum          1373

qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
 Sent 467997294 bytes 394515 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 800a: dev ifb4eth0.2 root refcnt 2 bandwidth 1Gbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 451226883 bytes 375624 pkt (dropped 18891, overlimits 2948 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 1587012b of 15140Kb
 capacity estimate: 1Gbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh          1Gbit
  target            5ms
  interval        100ms
  pk_delay         12us
  av_delay          4us
  sp_delay          2us
  backlog            0b
  pkts           394515
  bytes       479821916
  way_inds           23
  way_miss          402
  way_cols            0
  drops           18891
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len         66616
  quantum          1514

cat /etc/config/sqm

config queue 'eth1'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'none'
        option enabled '1'
        option download '1000000'
        option upload '45000'
        option debug_logging '0'
        option verbosity '5'
        option interface 'eth0.2'

Here is the test result, however I do have 1000mbps down and 100mpbs up.

Ah, I do not think your router is capable of shaping at 1 Gbps rates at all. Unless you use the NSS acelleration cores (requiring a special OpenWrt built IIRC, and the use of NSS qdiscs instead of SQM).
On a positive note shaping of egress @45Mbps seems to work as expected....

On stock firmware it would hit just under 1Gbps, the previous firmware as you just mentioned was using NSS and was hitting 970-980. But maybe its unreasonable to think I'd hit those high of speeds while using SQM.

Yes, traffic shaping is quite CPU intensive, and a CPU that happily routes at ~ 1 Gbps often has troubles shaping traffic close to 1 Gbps, especially if traffic shaping is configured to keep queuing latency low. For 1Gbps egress, we could forego the shaper and just relay on BQL to keep queueing latency under control, but for download/ingress traffic that is not really an option (the true bottleneck buffers that tend to be oversized and undermanaged are on the ISP's side of the access link, so the only way to get control back is to establish a traffic shaper that is set below the true bottleneck speed).

1 Like

At least with cake. Cake is much more CPU intensive than e.g. fq_codel in simple.qos

Toi be honest, with 1000 Mbit/s downloads, your main problem is on the upload side, to make sure that you do not choke the upload bandwidth with protocol traffic. I am not sure if it is at all worthwhile to have download QoS on those download speeds. (In real life, there aren't many hosts or ISPs that will enable you to continuously download with a full gigabit speed)

2 Likes

So the costly component for simple.qos ist actually HTB (the hierarchical token bucket filter) fq_codel, like the respective cake FQ scheduler are relatively cheap, CPU-wise, but sure cake, while originally aiming at being more efficient than HTB turned out to be feature-richer and costlier.
Also, for simple.qos SQM scripts allow some twiddling with SHAPER_BURST_DUR_US in /usr/lib/sqm/defaults.sh to allow some level of batching to deal more gracefully with an overloaded CPU, cake does not offer such controls.

This is often quite true, but note that this also means that disabling download shaping now makes a link less robust, sure overload/saturation situations will be rare, but if they occur they can cause transient inconvenient latency under load increases. It is a matter of subjective preference to set the policy for one's own network, I just want to make sure that the trade-off one needs to strike, between a (permanent) rate "sacrifice" and a expectedly much rarer latency-under-load spikes, is clear.

Side-note: personally, I tend to opt for low latency under load more than for maximum rate, and had my 116/35 link shaped to 49/33 as that was what my router at the time could achieve at robust and reliable bi-directional gross shaping rates*; but that was my subjective decision and the policy I set for my network, and every network admin needs to find their own decision based on local preferences.

*) I since upgraded my router and now am shaping to 95/36 on an effective 100/37 sync, because if latency under load is not compromised I enjoy a faster link more than a slower one...

The low test results of OP are a bit baffling.
R7800 does much better also with cake.
I have only a 200/60 line, and prefer fq_codel, but in any case, below is the test done with my R7800 with piece_of_cake with 190/55 limits.
cake reaches 181/53 (despite possibly unoptimized SQM link-layer settings, which possibly cause the slight underperform. And I am not using irqbalance etc. that could distribute CPU load more evenly to multiple cores...)
That is roughly the same as fq_codel with the same limits.

http://www.dslreports.com/speedtest/69495780

OP's result of 114/44 seems strangely low.

1 Like

Without a properly configured overhead, SQM uses the kernel's default of 14 bytes (which really is just the size of those ethernet header fields the kernel maintains itself and hence knows the size of), so for your 190/55 limits you can expect at best:

190 * ((1500-20-20)/(1500+14)) = 183.22 Mbps
55 * ((1500-20-20)/(1500+14)) = 53.04 Mbps

So, your speedtest looks spot-on for a non-CPU limited SQM instance.

For the OP we would get:

1000 * ((1500-20-20)/(1500+14)) = 964.33 (note, this is above the goodput possible over a 1Gbps ethernet link*)
45 * ((1500-20-20)/(1500+14)) = 43.39 Mbps

So the uplink is in the right ball park, but the downlink is off by a factor of ~8... which seems odd...

*) 1000 * ((1500-20-20)/(1500+38)) = 949.28 Mbps

1 Like

So if I turn off SQM, well disable it here:

Download goes up to this:

That looks rather odd as well, only 302.7/47.6 Mbps Down/Up seems rather too little for a 1000/100 plan...
Which ISP and which plan is this again?

BTW, if you know how to log into your router via SSH you can use:
/etc/init.d/sqm stop
and
/etc/init.d/sqm start
to stop and start SQM from the command line. And the use tc -s qdisc to confirm that SQM's shaper instances (either cake or HTB) are gone or appear...
If you suspect SQM problems try:
SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm stop
and
SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start
and post the output to this thread...

root@OpenWrt:~# SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start
/usr/lib/sqm/run.sh: line 57: can't create : nonexistent directory
SQM: Starting SQM script: piece_of_cake.qos on eth0.2, in: 1000000 Kbps, out: 45000 Kbps
SQM: fn_exists: function candidate name: sqm_start
SQM: fn_exists: TYPE_OUTPUT: sqm_start: not found
SQM: fn_exists: return value: 1
SQM: Using generic sqm_start_default function.
SQM: fn_exists: function candidate name: sqm_prepare_script
SQM: fn_exists: TYPE_OUTPUT: sqm_prepare_script is a function
SQM: fn_exists: return value: 0
SQM: sqm_start_default: starting sqm_prepare_script
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_61d0d type ifb
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc replace dev SQM_IFB_61d0d root cake
SQM: QDISC cake is useable.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_61d0d down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_61d0d type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_2071a type ifb
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc replace dev SQM_IFB_2071a root cake
SQM: QDISC cake is useable.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_2071a down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_2071a type ifb
SQM: sqm_start_default: Starting piece_of_cake.qos
SQM: ifb associated with interface eth0.2:
SQM: Currently no ifb is associated with eth0.2, this is normal during starting of the sqm system.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name ifb4eth0.2 type ifb
SQM: fn_exists: function candidate name: egress
SQM: fn_exists: TYPE_OUTPUT: egress is a function
SQM: fn_exists: return value: 0
SQM: egress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or acceptable.
SQM: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev eth0.2 root
SQM: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
SQM: LLA: default link layer adjustment method for cake is cake
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev eth0.2 root cake bandwidth 45000kbit besteffort
SQM: sqm_start_default: egress shaping activated
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_641e6 type ifb
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc replace dev SQM_IFB_641e6 ingress
SQM: QDISC ingress is useable.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_641e6 down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_641e6 type ifb
SQM: fn_exists: function candidate name: ingress
SQM: fn_exists: TYPE_OUTPUT: ingress is a function
SQM: fn_exists: return value: 0
SQM: ingress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or acceptable.
SQM: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev eth0.2 handle ffff: ingress
SQM: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: Invalid argument
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev eth0.2 handle ffff: ingress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or acceptable.
SQM: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev ifb4eth0.2 root
SQM: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
SQM: LLA: default link layer adjustment method for cake is cake
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb4eth0.2 root cake bandwidth 1000000kbit besteffort wash
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev ifb4eth0.2 up
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc filter add dev eth0.2 parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth0.2
SQM: sqm_start_default: ingress shaping activated
SQM: piece_of_cake.qos was started on eth0.2 successfully