New to SQM / looking for help

Just for reference, I'm currently running OpenWrt/ SQM on an eight year old ivy-bridge era dual-core Celeron 1037u (4 GB RAM, 500 GB SSHD, 2* Intel 82574L (e1000e) ethernet), which can route/ NAT/ firewall and do SQM easily at full 1 GBit/s wirespeed (~53% CPU load on one core, the other idles around 14%-20%). It's totally bored handling my real WAN connection (400/200 MBit/s) and rarely even clocks up beyond 800 MHz; ~15 watts idle, 22-25 watts under full load (kernel compile/ ffmpeg).

Only because I've done it like a dozen times (and felt suitably stupid afterward)....I see you are running SQM on your eth4 interface. This (eth4) is your WAN interface, correct? If it is not, try selecting your WAN interface, whichever one that might be. Works much better that way :wink:

Yeah that's why I am stomped, You will think my hardware is overkill...

Yeah i looked in to that many times. I actually changed it to eth0, because someone mentioned somewhere else the scripts were hardcoded with eth0. not sure if its true

I re-enabled cake and this is the system logs, not sure what the error is....


Here is the CPU frequency when running waveform bufferbloat test

and here is the result with cake enabled...that's very bad

same test with fqcodel simple qos

system logs with fqcodel...

I think i am giving gup lol!

Don't give up! I know this is frustrating in the present moment, but seriously, CAKE is so good when it's working.

As to your act_ipt question, it doesn't sound like that's the issue here. I'm basing that on this post:

Would you mind posting your /etc/config/sqm file again in its entirety?

Also, are you building your OpenWrt image yourself or downloading a pre-built image? (Sorry if I missed this previously)

I am using an image, I choose the latest available...

Openwrt version generic-ext4-combined-efi.img.gz

here is the info

root@Willthetech_Home:~# cat /etc/config/sqm

config queue 'eth1'
        option debug_logging '0'
        option verbosity '5'
        option enabled '1'
        option interface 'eth0'
        option linklayer 'none'
        option download '600000'
        option upload '40000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'

root@Willthetech_Home:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8009: dev eth0 root refcnt 9 bandwidth 40Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 175330683 bytes 819991 pkt (dropped 373, overlimits 1316669 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 183100b of 4Mb
 capacity estimate: 40Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         40Mbit
  target            5ms
  interval        100ms
  pk_delay       1.46ms
  av_delay        207us
  sp_delay          2us
  backlog            0b
  pkts           820364
  bytes       175870869
  way_inds         1141
  way_miss         1195
  way_cols            0
  drops             373
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len         21810
  quantum          1220

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 2085107885 bytes 1493796 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth1 root
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth2 root
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth2 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth3 root
 Sent 2142684821 bytes 1468975 pkt (dropped 0, overlimits 0 requeues 78)
 backlog 0b 0p requeues 78
qdisc fq_codel 0: dev eth3 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 1493812 bytes 16842 pkt (dropped 0, overlimits 0 requeues 4)
 backlog 0b 0p requeues 4
  maxpacket 1514 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 483665269 bytes 328119 pkt (dropped 0, overlimits 0 requeues 9)
 backlog 0b 0p requeues 9
  maxpacket 1514 drop_overlimit 0 new_flow_count 3 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 339703052 bytes 225926 pkt (dropped 0, overlimits 0 requeues 18)
 backlog 0b 0p requeues 18
  maxpacket 1514 drop_overlimit 0 new_flow_count 11 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 213609624 bytes 149979 pkt (dropped 0, overlimits 0 requeues 9)
 backlog 0b 0p requeues 9
  maxpacket 1514 drop_overlimit 0 new_flow_count 3 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 337643403 bytes 223769 pkt (dropped 0, overlimits 0 requeues 12)
 backlog 0b 0p requeues 12
  maxpacket 1514 drop_overlimit 0 new_flow_count 5 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 450300751 bytes 306134 pkt (dropped 0, overlimits 0 requeues 13)
 backlog 0b 0p requeues 13
  maxpacket 1514 drop_overlimit 0 new_flow_count 5 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 153653553 bytes 102481 pkt (dropped 0, overlimits 0 requeues 5)
 backlog 0b 0p requeues 5
  maxpacket 107 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 162615357 bytes 115725 pkt (dropped 0, overlimits 0 requeues 8)
 backlog 0b 0p requeues 8
  maxpacket 1514 drop_overlimit 0 new_flow_count 2 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev docker0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 800a: dev ifb4eth0 root refcnt 2 bandwidth 600Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 2144263891 bytes 1493013 pkt (dropped 783, overlimits 1673299 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 13816701b of 15140Kb
 capacity estimate: 600Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        600Mbit
  target            5ms
  interval        100ms
  pk_delay         15us
  av_delay          3us
  sp_delay          0us
  backlog            0b
  pkts          1493796
  bytes      2145446613
  way_inds          325
  way_miss         1107
  way_cols            0
  drops             783
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len         27252
  quantum          1514

root@Willthetech_Home:~# ifstatus wan
Interface wan not found



I also installed your NETDATA SQM stuff...not sure if it will help!

1 Like

Could you please post the following:

  1. tc -s qdisc show dev eth0 ; echo " " ; tc -s qdisc show dev ifb4eth0

  2. run a wafeform speedtest and post the "Share Your Results:" link here and immediately after run (again):

  3. tc -s qdisc show dev eth0 ; echo " " ; tc -s qdisc show dev ifb4eth0

I am interested in what pk_delay we will see.


root@Willthetech_Home:~# tc -s qdisc show dev eth0 ; echo " " ; tc -s qdisc show
 dev ifb4eth0
qdisc cake 8009: root refcnt 9 bandwidth 40Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 281015215 bytes 1664785 pkt (dropped 386, overlimits 1442336 requeues 5)
 backlog 0b 0p requeues 5
 memory used: 326Kb of 4Mb
 capacity estimate: 40Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         40Mbit
  target            5ms
  interval        100ms
  pk_delay       1.42ms
  av_delay        118us
  sp_delay          1us
  backlog            0b
  pkts          1665171
  bytes       281574897
  way_inds        20044
  way_miss        14547
  way_cols           27
  drops             386
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len         21810
  quantum          1220

qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 5331062625 bytes 4326685 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

qdisc cake 800a: root refcnt 2 bandwidth 600Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 5485700216 bytes 4325899 pkt (dropped 787, overlimits 3001226 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 13816701b of 15140Kb
 capacity estimate: 600Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        600Mbit
  target            5ms
  interval        100ms
  pk_delay         16us
  av_delay          4us
  sp_delay          0us
  backlog            0b
  pkts          4326686
  bytes      5486888739
  way_inds       126700
  way_miss        14172
  way_cols            0
  drops             787
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len         33308
  quantum          1514

root@Willthetech_Home:~# tc -s qdisc show dev eth0 ; echo " " ; tc -s qdisc show
 dev ifb4eth0
qdisc cake 8009: root refcnt 9 bandwidth 40Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 460497515 bytes 2457170 pkt (dropped 785, overlimits 2668535 requeues 5)
 backlog 0b 0p requeues 5
 memory used: 326Kb of 4Mb
 capacity estimate: 40Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         40Mbit
  target            5ms
  interval        100ms
  pk_delay       1.88ms
  av_delay       1.14ms
  sp_delay          1us
  backlog            0b
  pkts          2457955
  bytes       461631514
  way_inds        20075
  way_miss        15076
  way_cols           27
  drops             785
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len         21810
  quantum          1220

qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 6994534794 bytes 5521497 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

qdisc cake 800a: root refcnt 2 bandwidth 600Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 7191556252 bytes 5519395 pkt (dropped 2102, overlimits 4402889 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 13816701b of 15140Kb
 capacity estimate: 600Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        600Mbit
  target            5ms
  interval        100ms
  pk_delay         14us
  av_delay          2us
  sp_delay          0us
  backlog            0b
  pkts          5521497
  bytes      7194720896
  way_inds       138456
  way_miss        14685
  way_cols            0
  drops            2102
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len         33308
  quantum          1514



here is this too...

Odd thing is if I set my bandwidth to 150mbits or below it seems to work great, but anything above that I get some really bad latency. if I change to FQ codel / Simple QOS and I set my bandwidth to 600mbits. I get about 60ms latency, still not as good as I want way better than Cake is giving me at the same speed setting.

Mmmh, so that shows that on ingress/download where you see the attrocious bufferbloat in the waveform speedtest, cake also reports matching high internal sojourntimes... that again makes my think about getting access to CPU issues....

BTW, your netdata plots are more helpful than the cake statistics, simply because the peak delay ages out too quickly (this comes fro the download and the subsequent upload test, basically resets the download peak counter due to the reverse ACK traffic and whatnot else).

I think seeing the output of:

cat /proc/interrupts

might be interesting....

root@Willthetech_Home:~# cat /proc/interrupts
            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7
   0:         28          0          0          0          0          0          0          0   IO-APIC    2-edge      timer
   1:          0          8          0          0          0          0          0          0   IO-APIC    1-edge      i8042
   4:          0          0          0         15          0          0          0          0   IO-APIC    4-edge      ttyS0
   8:          0          0          1          0          0          0          0          0   IO-APIC    8-edge      rtc0
   9:          0          5          0          0          0          0          0          0   IO-APIC    9-fasteoi   acpi
  12:          9          0          0          0          0          0          0          0   IO-APIC   12-edge      i8042
 123:          0          0          0          0          0         45          0          0   PCI-MSI 32768-edge      i915
 124:          0        206          0          0          0          0       2022          0   PCI-MSI 376832-edge      ahci[0000:00:17.0]
 125:          0          0          0          0          0          0          0          0   PCI-MSI 327680-edge      xhci_hcd
 126:          1          0          0          0          0          0          0          0   PCI-MSI 2097152-edge      eth0
 127:          0        845          0          0    1112058          0          0          0   PCI-MSI 2097153-edge      eth0-TxRx-0
 128:          0          0        148          0          0          0          0     309415   PCI-MSI 2097154-edge      eth0-TxRx-1
 129:          0          0          0     324732          0          0          0          0   PCI-MSI 2097155-edge      eth0-TxRx-2
 130:          0     433138          0          0        141          0          0          0   PCI-MSI 2097156-edge      eth0-TxRx-3
 131:          0          0          0          0          0     593524          0          0   PCI-MSI 2097157-edge      eth0-TxRx-4
 132:          0    1235432          0          0          0          0         95          0   PCI-MSI 2097158-edge      eth0-TxRx-5
 133:          0          0          0          0          0          0     655743         59   PCI-MSI 2097159-edge      eth0-TxRx-6
 134:         73          0     296428          0          0          0          0          0   PCI-MSI 2097160-edge      eth0-TxRx-7
 135:          0          0          0          0          0          0          0          0   PCI-MSI 2099200-edge      eth1
 136:          0       3998          0          0          0          0          7          0   PCI-MSI 2099201-edge      eth1-TxRx-0
 137:          0          0       3998          0          0          0          0          7   PCI-MSI 2099202-edge      eth1-TxRx-1
 138:          7          0          0          0          0          0          0       3998   PCI-MSI 2099203-edge      eth1-TxRx-2
 139:       3998          7          0          0          0          0          0          0   PCI-MSI 2099204-edge      eth1-TxRx-3
 140:          0          0          7          0          0          0          0       3998   PCI-MSI 2099205-edge      eth1-TxRx-4
 141:          0          0          0          7          0       3998          0          0   PCI-MSI 2099206-edge      eth1-TxRx-5
 142:          0          0          0       3998          7          0          0          0   PCI-MSI 2099207-edge      eth1-TxRx-6
 143:          0          0          0          0          0          7       3998          0   PCI-MSI 2099208-edge      eth1-TxRx-7
 144:          0          0          0          0          0          0          0          0   PCI-MSI 2101248-edge      eth2
 145:          0          0          0          0          0       3998          0          7   PCI-MSI 2101249-edge      eth2-TxRx-0
 146:          7          0       3998          0          0          0          0          0   PCI-MSI 2101250-edge      eth2-TxRx-1
 147:          0          7          0          0          0          0       3998          0   PCI-MSI 2101251-edge      eth2-TxRx-2
 148:       3998          0          7          0          0          0          0          0   PCI-MSI 2101252-edge      eth2-TxRx-3
 149:          0          0          0          7       3998          0          0          0   PCI-MSI 2101253-edge      eth2-TxRx-4
 150:          0          0          0       3998          7          0          0          0   PCI-MSI 2101254-edge      eth2-TxRx-5
 151:          0          0          0          0          0          7          0       3998   PCI-MSI 2101255-edge      eth2-TxRx-6
 152:          0          0       3998          0          0          0          7          0   PCI-MSI 2101256-edge      eth2-TxRx-7
 153:          0          0          0          0          0          0          0          1   PCI-MSI 2103296-edge      eth3
 154:        194          0          0          0    1140012          0          0          0   PCI-MSI 2103297-edge      eth3-TxRx-0
 155:          0        202          0     389242          0          0          0          0   PCI-MSI 2103298-edge      eth3-TxRx-1
 156:          0          0         66          0          0          0          0     395008   PCI-MSI 2103299-edge      eth3-TxRx-2
 157:          0     362142          0         83          0          0          0          0   PCI-MSI 2103300-edge      eth3-TxRx-3
 158:          0          0          0          0         92     322859          0          0   PCI-MSI 2103301-edge      eth3-TxRx-4
 159:          0          0          0          0          0     314669          0          0   PCI-MSI 2103302-edge      eth3-TxRx-5
 160:          0          0          0          0          0          0     528970          0   PCI-MSI 2103303-edge      eth3-TxRx-6
 161:          0          0          0          0          0          0    1071147        111   PCI-MSI 2103304-edge      eth3-TxRx-7
 NMI:          0          0          0          0          0          0          0          0   Non-maskable interrupts
 LOC:    3128146    2513398    2687286    2649625    2502015    2488208    2792236    2592385   Local timer interrupts
 SPU:          0          0          0          0          0          0          0          0   Spurious interrupts
 PMI:          0          0          0          0          0          0          0          0   Performance monitoring interrupts
 IWI:          0          0          0          0          0          0          0          0   IRQ work interrupts
 RTR:          5          0          0          0          0          0          0          0   APIC ICR read retries
 RES:       3002       2670       1826       2934       2664       2977       3349       2110   Rescheduling interrupts
 CAL:    1229458    1100057     638168     655226     682061     832889     680377     578411   Function call interrupts
 TLB:        188        178        212        153        279        165        294        133   TLB shootdowns
 TRM:          0          0          0          0          0          0          0          0   Thermal event interrupts
 THR:          0          0          0          0          0          0          0          0   Threshold APIC interrupts
 DFR:          0          0          0          0          0          0          0          0   Deferred Error APIC interrupts
 MCE:          0          0          0          0          0          0          0          0   Machine check exceptions
 MCP:         26         27         27         27         27         27         27         27   Machine check polls
 ERR:          4
 MIS:          0
 PIN:          0          0          0          0          0          0          0          0   Posted-interrupt notification event
 NPI:          0          0          0          0          0          0          0          0   Nested posted-interrupt event
 PIW:          0          0          0          0          0          0          0          0   Posted-interrupt wakeup event

@WilR I don’t know if this will help, but for reference I have a 400/20mbps cable connection. I actually get more like 480/24mbps. Regardless, here is the SQM config I use:

root@OpenWrt:~# cat /etc/config/sqm

config queue
        option interface 'eth0'
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'cake'
        option linklayer 'none'
        option qdisc_advanced '1'
        option squash_dscp '0'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'docsis dual-dsthost nat ingress'
        option eqdisc_opts 'docsis dual-srchost nat ack-filter'
        option enabled '1'
        option upload '24500'
        option download '462500'
        option script 'ctinfo_4layercake.qos'


I wonder if you might be willing to drop in this config and give it a test. Obviously you’ll want to modify the upload, download, and interface to match your setup. But I’m just curious what your experience will be with this known good config.

Notice that my linklayer option is “none”, however that’s because I specify “docsis” in the advanced config options.

You can create of backup of your current SQM config in case this doesn’t give positive results.

done

 OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
root@Willthetech_Home:~# cat /etc/config/sqm

config queue
        option interface 'eth0'
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'cake'
        option linklayer 'none'
        option qdisc_advanced '1'
        option squash_dscp '0'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'docsis dual-dsthost nat ingress'
        option eqdisc_opts 'docsis dual-srchost nat ack-filter'
        option enabled '1'
        option upload '40000'
        option download '600000'
        option script 'ctinfo_4layercake.qos'

root@Willthetech_Home:~#


It seems like is not limiting my speeds..also my upload got worse.

Would you mind dropping the output from your tc -s qdisc show dev eth0 ; echo " " ; tc -s qdisc show dev ifb4eth0 command here again?

1 Like

Just to test install "Qosify"

I recommend that you use the Online Image Builder to build your custom OpenWrt firmware (with packages that you want already installed in the firmware, along with some commands to run on first boot):

Example:

  1. On "Custom package selection" ADD to the end of the last package, these packages:

    • qosify auc luci-app-attendedsysupgrade
  2. On "Custom script to run on first boot (optional)" ADD:

# Qosify
BANDWIDTH_UP="40mbit"
BANDWIDTH_DOWN="600mbit"
OVERHEAD_TYPE="docsis"
INGRESS_OPTIONS=""
EGRESS_OPTIONS="wash ack-filter"
INGRESS_EGRESS_OPTIONS="ether-vlan"

######################################
#   JESUS CHRIST IS THE LORD! AMEN   #
######################################

# Qosify
# Clear default config
uci del qosify.@defaults[0].dscp_prio
uci del qosify.@defaults[0].prio_max_avg_pkt_len
uci del qosify.besteffort
uci del qosify.bulk
uci del qosify.video
uci del qosify.voice
# ICMP (ping) and default class for "unmarked traffic"
uci set qosify.@defaults[0].dscp_icmp="+besteffort"
uci set qosify.@defaults[0].dscp_default_tcp="unmarked_traffic"
uci set qosify.@defaults[0].dscp_default_udp="unmarked_traffic"
# unmarked_traffic class
uci set qosify.unmarked_traffic=class
uci set qosify.unmarked_traffic.ingress="CS1"
uci set qosify.unmarked_traffic.egress="CS1"
uci set qosify.unmarked_traffic.prio_max_avg_pkt_len="1256"
uci set qosify.unmarked_traffic.dscp_prio="CS4"
uci set qosify.unmarked_traffic.bulk_trigger_pps="250"
uci set qosify.unmarked_traffic.bulk_trigger_timeout="10"
uci set qosify.unmarked_traffic.dscp_bulk="CS1"
# browsing class
uci set qosify.browsing=class
uci set qosify.browsing.ingress="CS0"
uci set qosify.browsing.egress="CS0"
uci set qosify.browsing.prio_max_avg_pkt_len="575"
uci set qosify.browsing.dscp_prio="AF41"
uci set qosify.browsing.bulk_trigger_pps="1000"
uci set qosify.browsing.bulk_trigger_timeout="10"
uci set qosify.browsing.dscp_bulk="CS1"
# bulk class
uci set qosify.bulk=class
uci set qosify.bulk.ingress="CS1"
uci set qosify.bulk.egress="CS1"
# besteffort class
uci set qosify.besteffort=class
uci set qosify.besteffort.ingress="CS0"
uci set qosify.besteffort.egress="CS0"
# network_services class
uci set qosify.network_services=class
uci set qosify.network_services.ingress="CS2"
uci set qosify.network_services.egress="CS2"
# broadcast_video class
uci set qosify.broadcast_video=class
uci set qosify.broadcast_video.ingress="CS3"
uci set qosify.broadcast_video.egress="CS3"
# gaming class
uci set qosify.gaming=class
uci set qosify.gaming.ingress="CS4"
uci set qosify.gaming.egress="CS4"
# multimedia_conferencing class
uci set qosify.multimedia_conferencing=class
uci set qosify.multimedia_conferencing.ingress="AF42"
uci set qosify.multimedia_conferencing.egress="AF42"
uci set qosify.multimedia_conferencing.prio_max_avg_pkt_len="575"
uci set qosify.multimedia_conferencing.dscp_prio="AF41"
# telephony class
uci set qosify.telephony=class
uci set qosify.telephony.ingress="EF"
uci set qosify.telephony.egress="EF"
# Reorder classes
uci reorder qosify.unmarked_traffic=1
uci reorder qosify.browsing=2
uci reorder qosify.bulk=3
uci reorder qosify.besteffort=4
uci reorder qosify.network_services=5
uci reorder qosify.broadcast_video=6
uci reorder qosify.gaming=7
uci reorder qosify.multimedia_conferencing=8
uci reorder qosify.telephony=9
# CAKE
uci set qosify.wan.name="wan"
uci set qosify.wan.disabled="0"
uci set qosify.wan.bandwidth_up="$BANDWIDTH_UP"
uci set qosify.wan.bandwidth_down="$BANDWIDTH_DOWN"
uci set qosify.wan.overhead_type="$OVERHEAD_TYPE"
uci set qosify.wan.autorate_ingress="0"
uci set qosify.wan.ingress_options="$INGRESS_OPTIONS"
uci set qosify.wan.egress_options="$EGRESS_OPTIONS"
uci set qosify.wan.options="$INGRESS_EGRESS_OPTIONS"
# Rules
cat << "EOF" > /etc/qosify/00-defaults.conf
# SSH
tcp:22    network_services

# NTP
udp:123   network_services

# DNS
tcp:53    network_services
tcp:5353  network_services
udp:53    network_services
udp:5353  network_services

# DNS over TLS (DoT)
tcp:853   multimedia_conferencing
udp:853   multimedia_conferencing

# HTTP/HTTPS/QUIC
tcp:80    browsing
tcp:443   browsing
udp:80    browsing
udp:443   browsing

# Microsoft (Download)
dns:*1drv*                 bulk
dns:*backblaze*            bulk
dns:*backblazeb2*          bulk
dns:*ms-acdc.office*       bulk
dns:*onedrive*             bulk
dns:*sharepoint*           bulk
dns:*update.microsoft*     bulk
dns:*windowsupdate*        bulk

# MEGA (Download)
dns:*mega*                 bulk

# Dropbox (Download)
dns:*dropboxusercontent*   bulk

# Google (Download)
dns:*drive.google*         bulk
dns:*googleusercontent*    bulk

# Steam (Download)
dns:*steamcontent*         bulk

# Epic Games (Download)
dns:*download.epicgames*   bulk
dns:*download2.epicgames*  bulk
dns:*download3.epicgames*  bulk
dns:*download4.epicgames*  bulk
dns:*epicgames-download1*  bulk

# YouTube
dns:*googlevideo*   besteffort

# Facebook
dns:*fbcdn*         besteffort

# Twitch
dns:*ttvnw*         besteffort

# TikTok
dns:*tiktok*        besteffort

# Netflix
dns:*nflxvideo*     besteffort

# Amazon Prime Video
dns:*aiv-cdn*       besteffort
dns:*aiv-delivery*  besteffort
dns:*pv-cdn*        besteffort

# Disney Plus
dns:*disney*        besteffort
dns:*dssott*        besteffort

# HBO
dns:*hbo*           besteffort
dns:*hbomaxcdn*     besteffort

# BitTorrent
tcp:6881-7000    bulk
tcp:51413        bulk
udp:6771         bulk
udp:6881-7000    bulk
udp:51413        bulk

# Usenet
tcp:119          bulk
tcp:563          bulk

# Live Streaming to YouTube Live, Twitch, Vimeo and LinkedIn Live
tcp:1935-1936    broadcast_video
tcp:2396         broadcast_video
tcp:2935         broadcast_video

# Xbox
tcp:3074         gaming
udp:88           gaming
#udp:500         gaming # UDP port already used in "VoWiFi" rules
udp:3074         gaming
udp:3544         gaming
#udp:4500        gaming # UDP port already used in "VoWiFi" rules

# PlayStation
tcp:3478-3480    gaming
#udp:3478-3479   gaming # UDP ports already used in "Zoom" rules

# Call of Duty
#tcp:3074        gaming # TCP port already used in "Xbox" rules
tcp:3075-3076    gaming
#udp:3074        gaming # UDP port already used in "Xbox" rules
udp:3075-3079    gaming
udp:3658         gaming

# FIFA
tcp:3659         gaming
udp:3659         gaming

# Minecraft
tcp:25565        gaming
udp:19132-19133  gaming
udp:25565        gaming

# Supercell Games
tcp:9339         gaming
udp:9339         gaming

# Zoom, Microsoft Teams, Skype and FaceTime (they use these same ports)
udp:3478-3497    multimedia_conferencing

# Zoom
dns:*zoom*       multimedia_conferencing
tcp:8801-8802    multimedia_conferencing
udp:8801-8810    multimedia_conferencing

# Skype
dns:*skype*      multimedia_conferencing

# FaceTime
udp:16384-16387  multimedia_conferencing
udp:16393-16402  multimedia_conferencing

# GoToMeeting
udp:1853         multimedia_conferencing
udp:8200         multimedia_conferencing

# Webex Meeting
tcp:5004         multimedia_conferencing
udp:9000         multimedia_conferencing

# Jitsi Meet
tcp:5349         multimedia_conferencing
udp:10000        multimedia_conferencing

# Google Meet
udp:19302-19309  multimedia_conferencing

# TeamViewer
tcp:5938         multimedia_conferencing
udp:5938         multimedia_conferencing

# Voice over Internet Protocol (VoIP)
tcp:5060-5061    telephony
udp:5060-5061    telephony

# Voice over WiFi or WiFi Calling (VoWiFi)
udp:500          telephony
udp:4500         telephony
EOF

# Saving modified values
uci commit qosify
/etc/init.d/qosify restart

  1. Request Build, wait and download the firmware, then flash the firmware and configure the router from scratch (don't use a backup).

  2. To confirm if the traffic is marked correctly you have to use this command:

    qosify-status
    
  3. Test:

P.S. Don't delete the packages that appear in "Custom package selection", they are necessary or you will brick the router, but you can replace those packages if you want.

Information about my Qosify configuration:

1 Like

Mmmh this looks unsuspicious to me, thanks. Next stop powertop to see what the core frequencies do.

1 Like

I understand that you like qosify over sqm (which is fine) but here, both really are just tools to instantiate cake on ingress and/or egress, after they are done setting things up all that is left is identical. So I am not sure how switching to qosify is going to change anything substantial?

3 Likes