SQM and R7800 router

Installing and Using OpenWrt Network and Wireless Configuration
1

hi,

I have the build r15401, 5.4.85 and I try to make a good sqm configuration.
I use speedtest on dslreport and even if I enable sqm or not, even if I change my download speed, my speed is still the same, about 127mbps, which is very good (I have 120/10). what do you think?

and not sure if sqm works good, here my config:

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'none'
        option enabled '1'
        option debug_logging '0'
        option verbosity '5'
        option qdisc_advanced '1'
        option squash_dscp '0'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'nat dual-dsthost diffserv3 docsis mpu 0 besteffort'
        option eqdisc_opts 'nat dual-srchost diffserv3 docsis ack-filter mpu 0'
        option interface 'eth0.2'
        option download '117300'
        option upload '9500'

root@OpenWrt:~# ifstatus wan
{
        "up": true,
        "pending": false,
        "available": true,
        "autostart": true,
        "dynamic": false,
        "uptime": 21124,
        "l3_device": "eth0.2",
        "proto": "dhcp",
        "device": "eth0.2",
        "updated": [
                "addresses",
                "routes",
                "data"
        ],
        "metric": 0,
        "dns_metric": 0,
        "delegation": true,
        "ipv4-address": [
                {
                        "address": "XXXXXXX",
                        "mask": 24
                }
        ],
        "ipv6-address": [

        ],
        "ipv6-prefix": [

        ],
        "ipv6-prefix-assignment": [

        ],
        "route": [
                {
                        "target": "0.0.0.0",
                        "mask": 0,
                        "nexthop": "XXXXXXX",
                        "source": "XXXXXXX/32"
                }
        ],
        "dns-server": [
                "1.1.1.1"
        ],
        "dns-search": [

        ],
        "neighbors": [

        ],
        "inactive": {
                "ipv4-address": [

                ],
                "ipv6-address": [

                ],
                "route": [

                ],
                "dns-server": [
                        "205.151.67.34",
                        "205.151.67.2",
                        "205.151.67.6"
                ],
                "dns-search": [
                        "telus.ca"
                ],
                "neighbors": [

                ]
        },
        "data": {
                "leasetime": 57060
        }
}
root@OpenWrt:~# tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta                                                                                                                     rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta                                                                                                                     rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth1.1 root refcnt 2
qdisc cake 8015: dev eth0.2 root refcnt 2 bandwidth 9500Kbit diffserv3 dual-srch                                                                                                                     ost nat nowash ack-filter split-gso rtt 100ms noatm overhead 18
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
qdisc noqueue 0: dev wg0 root refcnt 2
qdisc noqueue 0: dev wlan0 root refcnt 2
qdisc noqueue 0: dev wlan1 root refcnt 2
qdisc cake 8016: dev ifb4eth0.2 root refcnt 2 bandwidth 117300Kbit besteffort du                                                                                                                     al-dsthost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta                                                                                                                     rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 173655357 bytes 324440 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 571 drop_overlimit 0 new_flow_count 21 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta                                                                                                                     rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 145940 bytes 1106 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 535 drop_overlimit 0 new_flow_count 3 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8015: dev eth0.2 root refcnt 2 bandwidth 9500Kbit diffserv3 dual-srch                                                                                                                     ost nat nowash ack-filter split-gso rtt 100ms noatm overhead 18
 Sent 279162 bytes 1713 pkt (dropped 0, overlimits 193 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 10496b of 4Mb
 capacity estimate: 9500Kbit
 min/max network layer size:           28 /    1452
 min/max overhead-adjusted size:       46 /    1470
 average network hdr offset:           14

                   Bulk  Best Effort        Voice
  thresh      593744bit     9500Kbit     2375Kbit
  target         30.7ms          5ms       7.67ms
  interval        126ms        100ms        103ms
  pk_delay          0us        958us         22us
  av_delay          0us         63us          2us
  sp_delay          0us          8us          2us
  backlog            0b           0b           0b
  pkts                0         1681           32
  bytes               0       277554         1608
  way_inds            0            1            0
  way_miss            0          310            2
  way_cols            0            0            0
  drops               0            0            0
  marks               0            0            0
  ack_drop            0            0            0
  sp_flows            0            6            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1466           54
  quantum           300          300          300

qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
 Sent 1174295 bytes 2417 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wg0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8016: dev ifb4eth0.2 root refcnt 2 bandwidth 117300Kbit besteffort du                                                                                                                     al-dsthost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18
 Sent 1197535 bytes 2410 pkt (dropped 7, overlimits 237 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 187200b of 5865000b
 capacity estimate: 117300Kbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       64 /    1518
 average network hdr offset:           14

                  Tin 0
  thresh     117300Kbit
  target            5ms
  interval        100ms
  pk_delay        174us
  av_delay         35us
  sp_delay          6us
  backlog            0b
  pkts             2417
  bytes         1208133
  way_inds            3
  way_miss          268
  way_cols            0
  drops               7
  marks               0
  ack_drop            0
  sp_flows            6
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

root@OpenWrt:~# SQM_DEBUG=1 SQM_VERBOSITY_MAX=11 /etc/init.d/sqm stop ; SQM_DEBU
G=1 SQM_VERBOSITY_MAX=11 /etc/init.d/sqm start
SQM: Acquired run lock
/usr/lib/sqm/run.sh: line 57: can't create : nonexistent directory
SQM: Stopping SQM on eth0.2
SQM: Acquired run lock
/usr/lib/sqm/run.sh: line 57: can't create : nonexistent directory
SQM:
SQM: Sat Jan  2 13:28:21 EST 2021: Starting.
SQM: Starting SQM script: piece_of_cake.qos on eth0.2, in: 117300 Kbps, out: 950                                                                                                                     0 Kbps
SQM: fn_exists: function candidate name: sqm_start
SQM: fn_exists: TYPE_OUTPUT: sqm_start: not found
SQM: fn_exists: return value: 1
SQM: Using generic sqm_start_default function.
SQM: fn_exists: function candidate name: sqm_prepare_script
SQM: fn_exists: TYPE_OUTPUT: sqm_prepare_script is a function
SQM: fn_exists: return value: 0
SQM: sqm_start_default: starting sqm_prepare_script
SQM: cmd_wrapper: COMMAND: /sbin/ip link add name SQM_IFB_414f7 type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_414f7 type ifb
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc replace dev SQM_IFB_414f7 root cak                                                                                                                     e
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc qdisc replace dev SQM_IFB_414f7 root                                                                                                                      cake
SQM: QDISC cake is useable.
SQM: cmd_wrapper: COMMAND: /sbin/ip link set dev SQM_IFB_414f7 down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_414f7 down
SQM: cmd_wrapper: COMMAND: /sbin/ip link delete SQM_IFB_414f7 type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_414f7 type ifb
SQM: cmd_wrapper: COMMAND: /sbin/ip link add name SQM_IFB_569d8 type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_569d8 type ifb
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc replace dev SQM_IFB_569d8 root cak                                                                                                                     e
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc qdisc replace dev SQM_IFB_569d8 root                                                                                                                      cake
SQM: QDISC cake is useable.
SQM: cmd_wrapper: COMMAND: /sbin/ip link set dev SQM_IFB_569d8 down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_569d8 down
SQM: cmd_wrapper: COMMAND: /sbin/ip link delete SQM_IFB_569d8 type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_569d8 type ifb
SQM: sqm_start_default: Starting piece_of_cake.qos
SQM: ifb associated with interface eth0.2:
SQM: Currently no ifb is associated with eth0.2, this is normal during starting                                                                                                                      of the sqm system.
SQM: cmd_wrapper: COMMAND: /sbin/ip link add name ifb4eth0.2 type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name ifb4eth0.2 type ifb
SQM: fn_exists: function candidate name: egress
SQM: fn_exists: TYPE_OUTPUT: egress is a function
SQM: fn_exists: return value: 0
SQM: egress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or                                                                                                                      acceptable.
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc del dev eth0.2 root
SQM: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev eth0.2 root
SQM: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
SQM: LLA: default link layer adjustment method for cake is cake
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc add dev eth0.2 root cake bandwidth                                                                                                                      9500kbit besteffort nat dual-srchost diffserv3 docsis ack-filter mpu 0
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc qdisc add dev eth0.2 root cake bandw                                                                                                                     idth 9500kbit besteffort nat dual-srchost diffserv3 docsis ack-filter mpu 0
SQM: sqm_start_default: egress shaping activated
SQM: cmd_wrapper: COMMAND: /sbin/ip link add name SQM_IFB_f8cdc type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_f8cdc type ifb
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc replace dev SQM_IFB_f8cdc ingress
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc qdisc replace dev SQM_IFB_f8cdc ingr                                                                                                                     ess
SQM: QDISC ingress is useable.
SQM: cmd_wrapper: COMMAND: /sbin/ip link set dev SQM_IFB_f8cdc down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_f8cdc down
SQM: cmd_wrapper: COMMAND: /sbin/ip link delete SQM_IFB_f8cdc type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_f8cdc type ifb
SQM: fn_exists: function candidate name: ingress
SQM: fn_exists: TYPE_OUTPUT: ingress is a function
SQM: fn_exists: return value: 0
SQM: ingress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or                                                                                                                      acceptable.
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc del dev eth0.2 handle ffff: ingres                                                                                                                     s
SQM: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev eth0.2 handle ffff                                                                                                                     : ingress
SQM: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: Invalid argument
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc add dev eth0.2 handle ffff: ingres                                                                                                                     s
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc qdisc add dev eth0.2 handle ffff: in                                                                                                                     gress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or                                                                                                                      acceptable.
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc del dev ifb4eth0.2 root
SQM: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev ifb4eth0.2 root
SQM: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
SQM: LLA: default link layer adjustment method for cake is cake
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc qdisc add dev ifb4eth0.2 root cake bandw                                                                                                                     idth 117300kbit besteffort nat dual-dsthost diffserv3 docsis mpu 0 besteffort
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc qdisc add dev ifb4eth0.2 root cake b                                                                                                                     andwidth 117300kbit besteffort nat dual-dsthost diffserv3 docsis mpu 0 besteffor                                                                                                                     t
SQM: cmd_wrapper: COMMAND: /sbin/ip link set dev ifb4eth0.2 up
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev ifb4eth0.2 up
SQM: cmd_wrapper: COMMAND: /usr/sbin/tc filter add dev eth0.2 parent ffff: proto                                                                                                                     col all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev i                                                                                                                     fb4eth0.2
SQM: cmd_wrapper: tc: SUCCESS: /usr/sbin/tc filter add dev eth0.2 parent ffff: p                                                                                                                     rotocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect d                                                                                                                     ev ifb4eth0.2
2

On this hardware I used slightly different settings: SQM/QoS can saturate the CPU/is this expected or can the code be improved?

I am now using the NSS patched code and getting much better results. If your connection is 120/10 you should not need that. Just look at the link I posted and try those settings.

3

I tried to change for simple.qos and fq_codel but I have a lot of errors when I restart sqm, usualy not

root@OpenWrt:~# /etc/init.d/sqm restart
SQM: Stopping SQM on eth0.2
SQM: Starting SQM script: simple.qos on eth0.2, in: 116300 Kbps, out: 9400 Kbps
SQM: Using generic sqm_start_default function.
SQM: get_burst (by duration): the calculated burst/quantum size of 1175 bytes wa                                                                                                                     s below the minimum of 1749 bytes.
SQM: get_burst (by duration): the calculated burst/quantum size of 1175 bytes wa                                                                                                                     s below the minimum of 1749 bytes.
SQM: ERROR: cmd_wrapper: tc: FAILURE (1): /usr/sbin/tc qdisc add dev eth0.2 pare                                                                                                                     nt 1:11 handle 110: fq_codel limit 1001 target 5000us interval 100000us noecn qu                                                                                                                     antum 300 flows 1024 nat dual-srchost diffserv3 docsis ack-filter mpu 0
SQM: ERROR: cmd_wrapper: tc: LAST ERROR: What is "nat"?
Usage: ... fq_codel     [ limit PACKETS ] [ flows NUMBER ]
[ memory_limit BYTES ]
[ target TIME ] [ interval TIME ]
[ quantum BYTES ] [ [no]ecn ]
[ ce_threshold TIME ]
[ drop_batch SIZE ]
SQM: ERROR: cmd_wrapper: tc: FAILURE (1): /usr/sbin/tc qdisc add dev eth0.2 pare                                                                                                                     nt 1:12 handle 120: fq_codel limit 1001 target 5000us interval 100000us noecn qu                                                                                                                     antum 300 flows 1024 nat dual-srchost diffserv3 docsis ack-filter mpu 0
SQM: ERROR: cmd_wrapper: tc: LAST ERROR: What is "nat"?
Usage: ... fq_codel     [ limit PACKETS ] [ flows NUMBER ]
[ memory_limit BYTES ]
[ target TIME ] [ interval TIME ]
[ quantum BYTES ] [ [no]ecn ]
[ ce_threshold TIME ]
[ drop_batch SIZE ]
SQM: ERROR: cmd_wrapper: tc: FAILURE (1): /usr/sbin/tc qdisc add dev eth0.2 pare                                                                                                                     nt 1:13 handle 130: fq_codel limit 1001 target 5000us interval 100000us noecn qu                                                                                                                     antum 300 flows 1024 nat dual-srchost diffserv3 docsis ack-filter mpu 0
SQM: ERROR: cmd_wrapper: tc: LAST ERROR: What is "nat"?
Usage: ... fq_codel     [ limit PACKETS ] [ flows NUMBER ]
[ memory_limit BYTES ]
[ target TIME ] [ interval TIME ]
[ quantum BYTES ] [ [no]ecn ]
[ ce_threshold TIME ]
[ drop_batch SIZE ]
SQM: ERROR: cmd_wrapper: tc: FAILURE (1): /usr/sbin/tc qdisc add dev ifb4eth0.2                                                                                                                      parent 1:10 handle 110: fq_codel limit 1001 target 5000us interval 100000us ecn                                                                                                                      flows 1024 nat dual-dsthost diffserv3 docsis mpu 0 besteffort
SQM: ERROR: cmd_wrapper: tc: LAST ERROR: What is "nat"?
Usage: ... fq_codel     [ limit PACKETS ] [ flows NUMBER ]
[ memory_limit BYTES ]
[ target TIME ] [ interval TIME ]
[ quantum BYTES ] [ [no]ecn ]
[ ce_threshold TIME ]
[ drop_batch SIZE ]
SQM: simple.qos was started on eth0.2 successfully
4

And don't know why my speed always remains the same even if I change the download and upload between 85 and 95%

5

These options are only supported by cake, so for simple.qos and fq_codel you need to empty both iqdisc_opts and eqdisc_opts.

1 Like
6

btw, what is the most important data to check in the test about latency during my tests.?

33333

222
222584Ă—555 66.7 KB
1111

7

The chart at the bottom with the miliseconds charts is the most important. Consider turning down your upload (8500 or 9000) - that should improve your results.

You want the average (the colored bar to be in the white shaded area of the chart and the range (red bar) to be in that white shaded area as well (out of the “red” and “yellow” ranges, the less latency the better).

This is good bufferbloat control (with a r7800 too):

image
image1214Ă—1052 106 KB

1 Like
8

ok thanks, I would like to get an ''A+'' in bufferbloat, the best I have is ''A''
and I tried a lot of setting

9

Try a slightly lower upload speed. That should get you there.

10

By all means, have a look at the detailed bufferbloat graphs (by e.g. clicking the links below the three bars in the default view to expand either idle, downloading or uploading), these tell a richer story than the one letter grade...

11

on my wrt1900 with 19.07.5 build, when I use sqm and I change upload and download limit speed in config, my speed results change in dslreport, and usualy I have 10-15 mbps less.. but here with my r7800 with NSS build, my speed down and upload are always the same....
33333

12

you mean that ; ?

4243
4243517Ă—611 8.48 KB

1 Like
13

Yes, as you can see both Download and Upload look mostly okay, but each have a nasty spike. Worth researching where that spike comes from (like testing with different browsers). Also the Upload seems more variable even without that spike...

14

about upload, I am not able to stabilize it

15

I wonder if I chose the right one?
I use interface ''switch vlan eth0.2

4243
4243536Ă—579 34.1 KB

16

If you are running a NSS build - NSS builds bypass SQM :grin:

To run SQM you’ll have to use NSS fq_codel. Put this in your start up script, reset your device and see what you think:


modprobe nss-ifb

ip link set up nssifb

# Shape ingress traffic to 115 Mbit with chained NSSFQ_CODEL
tc qdisc add dev nssifb root handle 1: nsstbl rate 115Mbit burst 1Mb
tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

# Shape egress traffic to 9 Mbit with chained NSSFQ_CODEL
tc qdisc add dev eth0 root handle 1: nsstbl rate 9Mbit burst 1Mb
tc qdisc add dev eth0 parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default
17

reset your device????
you mean?

18

ok, Ive put the script, reboot and restart sqm, I got the NSS fq_codel, do I need to setup anything else in advance or I can keep what I had?

option iqdisc_opts 'nat dual-dsthost diffserv3 docsis mpu 0 besteffort ingress'
option eqdisc_opts 'nat dual-srchost diffserv3 docsis ack-filter mpu 0'

and to be sure, I need to choose ''eth0.2'' right?

Sans titre
Sans titre536Ă—645 39.5 KB

19

All your settings are already set by the script. No need to tweak anything in luci (nothing on the luci sqm page will do anything).

Make small tweaks to the speed (usually lower it just a bit) to get your bufferbloat right. CPU utilization should be minimal with NSS offloading. :sunglasses:

20

are you telling me that I have to disable sqm in Luci?