SQM, cake and piece_of_cake.qos - High CPU usage

Installing and Using OpenWrt Network and Wireless Configuration
#1

Hey,

I've set up SQM in my Archer C7 v4 with cake and piece_of_cake.qos. I've tested the bufferbloat in it seems fine. The CPU usage during the test is pretty high, getting spikes between ~80%-~90% , but the performance of the internet is good. When using a torrent client, the CPU usage is always on 100% and I encounter lag spikes and bad performance.

Is there a way to decrease the CPU usage?
I've used before QoS on WR1024ND with DD-WRT, and I've never noticed any problem with it. Should I use QoS instead of SQM in Archer C7?

1 Like
#2

Are you running the torrent client on the router or do you see this problem if you torent to a computer in your internal network?
Also, what bandwidth to the internet do you have?

#3

What bandwidth are you trying to shape?

An Archer C7, like most MIPS-based routers with similar single-core SoCs, without NAT offloading isn't good for more than a couple hundred mbps, even before you get to "fancy" QoS/SQM management.

#4

@moeller0 I'm using the torrent client on my PC, connected via Ethernet to the router.

@jeff Sorry, I forgot to mention the bandwidth. I'm trying to shape only ~50Mbps.

#5

Ah, okay that looks a bit odd,
Could you please post the output of the following commands:
cat /etc/config/sqm
tc -d qdisc
tc -s qdisc

then do some torrenting and do

tc -s qdisc

maybe this will reveal a smoking gun...

#6

I also see fairly high CPU usage on the C7 v4 with Cake, and that's on a 12Mbps DSL line.

tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth0.1 root refcnt 2
qdisc noqueue 0: dev eth0.2 root refcnt 2
qdisc cake 8004: dev pppoe-wan root refcnt 2 bandwidth 647Kbit diffserv3 dual-srchost nat ack-filter split-gso rtt 100.0ms atm overhead 48 mpu 64
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 8005: dev ifb4pppoe-wan root refcnt 2 bandwidth 10227Kbit diffserv3 dual-dsthost nat split-gso rtt 100.0ms atm overhead 48 mpu 64


tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 80824865485 bytes 167066140 pkt (dropped 0, overlimits 0 requeues 5)
 backlog 0b 0p requeues 5
  maxpacket 1214 drop_overlimit 0 new_flow_count 10 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.2 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8004: dev pppoe-wan root refcnt 2 bandwidth 647Kbit diffserv3 dual-srchost nat ack-filter split-gso rtt 100.0ms atm overhead 48 mpu 64
 Sent 18749208818 bytes 73838506 pkt (dropped 6580682, overlimits 89503310 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 2319296b of 4Mb
 capacity estimate: 647Kbit
 min/max network layer size:           28 /    1492
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       40432bit      647Kbit    161744bit
  target        450.5ms       28.2ms      112.6ms
  interval      901.1ms      123.2ms      225.2ms
  pk_delay          0us       62.2ms       12.8ms
  av_delay          0us        7.1ms        1.5ms
  sp_delay          0us         11us         28us
  backlog            0b           0b           0b
  pkts                0     80173003       246185
  bytes               0  19465302637     30145446
  way_inds            0     12803981        44565
  way_miss            0      1283072        29800
  way_cols            0            1            0
  drops               0       550685            0
  marks               0        66631            0
  ack_drop            0      6029997            0
  sp_flows            0            3            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1492          576
  quantum           300          300          300

qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
 Sent 57453625302 bytes 86538406 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8005: dev ifb4pppoe-wan root refcnt 2 bandwidth 10227Kbit diffserv3 dual-dsthost nat split-gso rtt 100.0ms atm overhead 48 mpu 64
 Sent 56971329613 bytes 86229152 pkt (dropped 328239, overlimits 76126053 requeues 0)
 backlog 10150b 7p requeues 0
 memory used: 242048b of 4Mb
 capacity estimate: 10227Kbit
 min/max network layer size:           28 /    1492
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh      639184bit    10227Kbit     2556Kbit
  target         28.4ms        5.0ms        7.1ms
  interval      123.4ms      100.0ms      102.1ms
  pk_delay          0us        7.4ms        745us
  av_delay          0us        3.1ms         38us
  sp_delay          0us        1.7ms         23us
  backlog            0b       10150b           0b
  pkts                0     85139485      1417913
  bytes               0  57217136958    238197624
  way_inds            0     11768904            0
  way_miss            0      1348805           11
  way_cols            0            0            0
  drops               0       328239            0
  marks               0        69254            0
  ack_drop            0            0            0
  sp_flows            0            4            2
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1492          168
  quantum           300          312          300

Unit has been running for a week, so byte counts are high. Line has 40+ devices on it and stays pretty busy. Performance is fine, but anytime I look at top or load averages, they are high.

#7

Thanks @JonFo this might indicate that the C7 v4 might simply not be up to the task of shaping the OPs 50 Mbps under a real demanding/multi-flow load like bit torrent, which is a bit sad, as I believer we used to be able to shape up to around a combined 60-70 Mbps on similar MIPS hardware. I guess It also depends on what else your router needs to do. Since you connect via ethernet could you for a quick test disable all radios in the router and reprat the turrent test? (I have a 500MHz Lantiq Xway MIPS 34Kc @500 MHz that can definitely not handle sqm/layer_cake, NAT, PPPoE-termination, firewalling duties and wifi at the same time, so it might be simply to much load for the hardware (then again a 750MHz MIPS 74K device with 128 MB ram should better than the old 34Kc...). Are you using PPPoE or ds-lite by any chance?

#8

In my case, the radios are disabled already. I use separate AP's.

Yes, I'm using PPPoE as the modem is a dumb DSL bridge modem, and my ISP requires PPPoE.

I do have a friend on 250Mbps Cable that has the same hardware and firmware levels as mine, and he can shape up to around 150Mbps effective. His user experience is great, but the CPU load while running the tests does hit 100% and idle goes to 0% and sirq is high. So it can definitely handle 50, but maybe the high connection count caused by torrenting is a heavier CPU load than traditional usage patterns.

#9

wow. I note that the ack-filtering code is fairly high overhead, also. But, wow, look at how effective it is being! Do you "feel" a difference in normal use with ack-filtering on?

#10

Also 40 users? on a 10mbit/647kbit dsl link? wow, twice.

pleasing also to see tons of marks on inbound (do you have osx clients? explicitly enabled ecn on linux clients?)

#11

On the torrent front, one thing I keep hoping will happen is that torrent clients will start marking their traffic CS1, which pushes torrent traffic to the cake background queue (6.5% bandwidth minimum) It's a one liner in transmission's settings.json. Similarly, I see no use of the "bulk" traffic class on the inbound so you could try matching on the port number to send stuff that way.

per-host fq as cake does "solves" this problem pretty well without classification however.

#12

Dave, one thing I've noticed is that some low-cost managed switches, such as TP-link ones use the DSCP values as if they were a monotone increasing priority. So CS1 actually gets more priority in the switch than CS0. :smirk: Because of this I've been tagging all my ingress as CS2 by default and then up or down shifting priority from there.

prioritization/QoS is hard, and poorly standardized. If you have different managed switches they'll do different things for example. I have a mix of Zyxel and TP-Link switches, and I can't control the TP-Link so I've customized the QoS in the Zyxel to match the TP-link, with that, and appropriate DSCP tagging within the router, squid proxy, and on my VOIP phones etc, I can get excellent responsiveness for all my real-time stuff (VOIP and Games), as well as my moderate priority stuff (Video streaming), but the effort that's gone into getting it right end-to-end across my network is quite high.

EDIT: incidentally, the reason it even matters in the switches is that I run home directories off an NFS server that has a bonded NIC, and the switches use LAG group trunks, so there can be large high-bandwidth LAN transfers just from everyday usage (several machines hitting the file server at once) so prioritizing certain traffic in the switches is necessary to ensure VOIP or game traffic is delivered in a timely manner between the phone and the router even if you're opening up a multi-gig data file or copying a video to the fileserver.

#13

sigh. Is per-host fq helping on the voip front?

Does TOS4 (the old TOS "IMM" bit) or CS6/CS7 help on the voip front? At least the latter two should get through your switches sanely.

#14

@dtaht yes CS6 works great for VOIP, better than the usual EF because by default the wifi drivers put EF into the VIDEO WMM queue whereas CS6 makes it into VOICE.

The issue with CS1 was more that I was tagging large streams like torrents or whatever CS1 and it was slowing my local NFS file transfers tagged CS0 :wink: now that everything starts at CS2, CS1 for torrents or whatever does work fine.

It's great how every tech issue has an appropriate xkcd to link to :laughing:

#15

Here is the output from my router, hope it'll help. Though it seems that the hardware is too weak.

cat /etc/config/sqm 
cat /etc/config/sqm

config queue 'eth1'
        option qdisc_advanced '0'
        option debug_logging '0'
        option verbosity '0'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option download '50000'
        option upload '2450'
        option enabled '1'
        option linklayer 'ethernet'
        option overhead '8'
        option interface 'eth0.2'

config queue
        option debug_logging '0'
        option linklayer 'none'
        option enabled '1'
        option verbosity '0'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option qdisc_advanced '0'
        option interface 'br-guest_lan'
        option download '500'
        option upload '5000'
tc -d qdisc 
tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc hfsc 1: dev ifb0 root refcnt 2 default 30
qdisc fq_codel 400: dev ifb0 parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc fq_codel 100: dev ifb0 parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc fq_codel 200: dev ifb0 parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc fq_codel 300: dev ifb0 parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc cake 8022: dev eth0.2 root refcnt 2 bandwidth 2450Kbit besteffort triple-isolate split-gso rtt 100.0ms noatm overhead 8
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
qdisc hfsc 1: dev pppoe-wan root refcnt 2 default 30
qdisc fq_codel 400: dev pppoe-wan parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc fq_codel 100: dev pppoe-wan parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc fq_codel 200: dev pppoe-wan parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc fq_codel 300: dev pppoe-wan parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 8025: dev br-guest_lan root refcnt 2 bandwidth 5Mbit besteffort triple-isolate split-gso rtt 100.0ms raw overhead 0
qdisc ingress ffff: dev br-guest_lan parent ffff:fff1 ----------------
qdisc noqueue 0: dev wlan1 root refcnt 2
qdisc noqueue 0: dev wlan1-1 root refcnt 2
qdisc cake 8023: dev ifb4eth0.2 root refcnt 2 bandwidth 50Mbit besteffort triple-isolate wash split-gso rtt 100.0ms noatm overhead 8
qdisc cake 8026: dev ifb4r-guest_lan root refcnt 2 bandwidth 500Kbit besteffort triple-isolate wash split-gso rtt 100.0ms raw overhead 0
qdisc noqueue 0: dev eth0.3 root refcnt 2
qdisc noqueue 0: dev eth0.1 root refcnt 2
tc -s qdisc 
tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 35663823532 bytes 52325566 pkt (dropped 0, overlimits 0 requeues 25)
 backlog 0b 0p requeues 25
  maxpacket 1514 drop_overlimit 0 new_flow_count 186 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc hfsc 1: dev ifb0 root refcnt 2 default 30
 Sent 38659177611 bytes 32234466 pkt (dropped 49938, overlimits 36320698 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 400: dev ifb0 parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 493590892 bytes 479683 pkt (dropped 1252, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 59131 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc fq_codel 100: dev ifb0 parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 4276635 bytes 44813 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1470 drop_overlimit 0 new_flow_count 8292 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc fq_codel 200: dev ifb0 parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 29864256107 bytes 23175660 pkt (dropped 45715, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 2853873 ecn_mark 0
  new_flows_len 1 old_flows_len 1
qdisc fq_codel 300: dev ifb0 parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 8297053977 bytes 8534310 pkt (dropped 2971, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 2061176 ecn_mark 0
  new_flows_len 1 old_flows_len 1
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8022: dev eth0.2 root refcnt 2 bandwidth 2450Kbit besteffort triple-isolate split-gso rtt 100.0ms noatm overhead 8
 Sent 2038731096 bytes 20079893 pkt (dropped 221230, overlimits 11774255 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 999744b of 4Mb
 capacity estimate: 2450Kbit
 min/max network layer size:           16 /    1500
 min/max overhead-adjusted size:       24 /    1508
 average network hdr offset:           14

                  Tin 0
  thresh       2450Kbit
  target          7.4ms
  interval      102.4ms
  pk_delay        854us
  av_delay        171us
  sp_delay         11us
  backlog            0b
  pkts         20301123
  bytes      2056464705
  way_inds      1152918
  way_miss       506554
  way_cols            0
  drops          221230
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
 Sent 34437160227 bytes 29375788 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc hfsc 1: dev pppoe-wan root refcnt 2 default 30
 Sent 2336422020 bytes 22230565 pkt (dropped 21353, overlimits 9207025 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 400: dev pppoe-wan parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 64787256 bytes 379798 pkt (dropped 1874, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 105462 ecn_mark 0
  new_flows_len 0 old_flows_len 2
qdisc fq_codel 100: dev pppoe-wan parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 6297047 bytes 54208 pkt (dropped 11, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 576 drop_overlimit 0 new_flow_count 11519 ecn_mark 0
  new_flows_len 1 old_flows_len 2
qdisc fq_codel 200: dev pppoe-wan parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 1741302740 bytes 16005322 pkt (dropped 3674, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 6234000 ecn_mark 0
  new_flows_len 1 old_flows_len 1
qdisc fq_codel 300: dev pppoe-wan parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 524034977 bytes 5791237 pkt (dropped 15794, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 1597820 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
 Sent 38732922571 bytes 32284404 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8025: dev br-guest_lan root refcnt 2 bandwidth 5Mbit besteffort triple-isolate split-gso rtt 100.0ms raw overhead 0
 Sent 242512060 bytes 236985 pkt (dropped 5319, overlimits 333338 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 369024b of 4Mb
 capacity estimate: 5Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh          5Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay       11.5ms
  av_delay        2.0ms
  sp_delay          6us
  backlog            0b
  pkts           242304
  bytes       250095210
  way_inds         2367
  way_miss        14431
  way_cols            2
  drops            5319
  marks              10
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev br-guest_lan parent ffff:fff1 ----------------
 Sent 35414143 bytes 194874 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1-1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8023: dev ifb4eth0.2 root refcnt 2 bandwidth 50Mbit besteffort triple-isolate wash split-gso rtt 100.0ms noatm overhead 8
 Sent 34372204964 bytes 29059322 pkt (dropped 316466, overlimits 37251919 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 716224b of 4Mb
 capacity estimate: 50Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       54 /    1508
 average network hdr offset:           14

                  Tin 0
  thresh         50Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay        2.9ms
  av_delay        453us
  sp_delay          7us
  backlog            0b
  pkts         29375788
  bytes     34848421259
  way_inds      1727051
  way_miss       472935
  way_cols            0
  drops          316466
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

qdisc cake 8026: dev ifb4r-guest_lan root refcnt 2 bandwidth 500Kbit besteffort triple-isolate wash split-gso rtt 100.0ms raw overhead 0
 Sent 36243899 bytes 193114 pkt (dropped 1760, overlimits 208221 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 300608b of 4Mb
 capacity estimate: 500Kbit
 min/max network layer size:           20 /    1514
 min/max overhead-adjusted size:       20 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        500Kbit
  target         36.3ms
  interval      131.3ms
  pk_delay       34.0ms
  av_delay        2.8ms
  sp_delay         13us
  backlog            0b
  pkts           194874
  bytes        38142365
  way_inds         2550
  way_miss        15107
  way_cols            0
  drops            1760
  marks              45
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc noqueue 0: dev eth0.3 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
tc -s qdisc - While torrenting 
tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 36573568031 bytes 53776180 pkt (dropped 0, overlimits 0 requeues 26)
 backlog 0b 0p requeues 26
  maxpacket 1514 drop_overlimit 0 new_flow_count 192 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc hfsc 1: dev ifb0 root refcnt 2 default 30
 Sent 39507616304 bytes 33094807 pkt (dropped 50304, overlimits 37175362 requeues 0)
 backlog 17906b 15p requeues 0
qdisc fq_codel 400: dev ifb0 parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 493590892 bytes 479683 pkt (dropped 1252, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 59131 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc fq_codel 100: dev ifb0 parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 4276635 bytes 44813 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1470 drop_overlimit 0 new_flow_count 8292 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc fq_codel 200: dev ifb0 parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 29864256107 bytes 23175660 pkt (dropped 45715, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 2853873 ecn_mark 0
  new_flows_len 1 old_flows_len 1
qdisc fq_codel 300: dev ifb0 parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 9145492670 bytes 9394651 pkt (dropped 3337, overlimits 0 requeues 0)
 backlog 17906b 15p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 2428674 ecn_mark 0 memory_used 29760
  new_flows_len 1 old_flows_len 32
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8022: dev eth0.2 root refcnt 2 bandwidth 2450Kbit besteffort triple-isolate split-gso rtt 100.0ms noatm overhead 8
 Sent 2091089968 bytes 20734845 pkt (dropped 298924, overlimits 12683545 requeues 0)
 backlog 13133b 168p requeues 0
 memory used: 999744b of 4Mb
 capacity estimate: 2450Kbit
 min/max network layer size:           16 /    1500
 min/max overhead-adjusted size:       24 /    1508
 average network hdr offset:           14

                  Tin 0
  thresh       2450Kbit
  target          7.4ms
  interval      102.4ms
  pk_delay      171.1ms
  av_delay       33.9ms
  sp_delay        2.7ms
  backlog        13133b
  pkts         21033937
  bytes      2114786990
  way_inds      1165320
  way_miss       508197
  way_cols            0
  drops          298924
  marks               0
  ack_drop            0
  sp_flows           65
  bk_flows           35
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
 Sent 35295854241 bytes 30238602 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc hfsc 1: dev pppoe-wan root refcnt 2 default 30
 Sent 2378619549 bytes 22963023 pkt (dropped 22897, overlimits 9955896 requeues 0)
 backlog 465b 7p requeues 0
qdisc fq_codel 400: dev pppoe-wan parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 64787256 bytes 379798 pkt (dropped 1874, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 105462 ecn_mark 0
  new_flows_len 0 old_flows_len 2
qdisc fq_codel 100: dev pppoe-wan parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 6297047 bytes 54208 pkt (dropped 11, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 576 drop_overlimit 0 new_flow_count 11519 ecn_mark 0
  new_flows_len 1 old_flows_len 2
qdisc fq_codel 200: dev pppoe-wan parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 1741302740 bytes 16005322 pkt (dropped 3674, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 6234000 ecn_mark 0
  new_flows_len 1 old_flows_len 1
qdisc fq_codel 300: dev pppoe-wan parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb
 Sent 566232506 bytes 6523695 pkt (dropped 17338, overlimits 0 requeues 0)
 backlog 465b 7p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 1864879 ecn_mark 0 memory_used 13888
  new_flows_len 4 old_flows_len 19
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
 Sent 39581846118 bytes 33145126 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8025: dev br-guest_lan root refcnt 2 bandwidth 5Mbit besteffort triple-isolate split-gso rtt 100.0ms raw overhead 0
 Sent 242512060 bytes 236985 pkt (dropped 5319, overlimits 333338 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 369024b of 4Mb
 capacity estimate: 5Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh          5Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay       11.5ms
  av_delay        2.0ms
  sp_delay          6us
  backlog            0b
  pkts           242304
  bytes       250095210
  way_inds         2367
  way_miss        14431
  way_cols            2
  drops            5319
  marks              10
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev br-guest_lan parent ffff:fff1 ----------------
 Sent 35414143 bytes 194874 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1-1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8023: dev ifb4eth0.2 root refcnt 2 bandwidth 50Mbit besteffort triple-isolate wash split-gso rtt 100.0ms noatm overhead 8
 Sent 35240745902 bytes 29920405 pkt (dropped 318167, overlimits 38224816 requeues 0)
 backlog 40237b 30p requeues 0
 memory used: 716224b of 4Mb
 capacity estimate: 50Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       54 /    1508
 average network hdr offset:           14

                  Tin 0
  thresh         50Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay       10.2ms
  av_delay        2.4ms
  sp_delay        338us
  backlog        40237b
  pkts         30238602
  bytes     35719194669
  way_inds      1796459
  way_miss       474170
  way_cols            0
  drops          318167
  marks               0
  ack_drop            0
  sp_flows           72
  bk_flows           16
  un_flows            0
  max_len          1514
  quantum          1514

qdisc cake 8026: dev ifb4r-guest_lan root refcnt 2 bandwidth 500Kbit besteffort triple-isolate wash split-gso rtt 100.0ms raw overhead 0
 Sent 36243899 bytes 193114 pkt (dropped 1760, overlimits 208221 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 300608b of 4Mb
 capacity estimate: 500Kbit
 min/max network layer size:           20 /    1514
 min/max overhead-adjusted size:       20 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        500Kbit
  target         36.3ms
  interval      131.3ms
  pk_delay       34.0ms
  av_delay        2.8ms
  sp_delay         13us
  backlog            0b
  pkts           194874
  bytes        38142365
  way_inds         2550
  way_miss        15107
  way_cols            0
  drops            1760
  marks              45
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc noqueue 0: dev eth0.3 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
#16

Mmh, no smoking gun, but I can almost guarantee that overhead 8 ist wrong. if this is a cable/docsis plan try setting the overhead to 18.
Are you by any chance able to seoarate the torrenting out from the computer yu use for your interactive browsing? If yes, have a look at the "Making cake sing and dance, on a tight rope without a safety net" section of https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details to learn how to configure per internal IP fairness. That has the promise to localize the bad latency spikes to that computer that actually overloads the network by torrenting while other computers might continue to work well for interactive use. If that helps it might be time to figure out hoe to configure your torrent client to behave better.

Best Regards

#17

I thought it was lightning. Or does Apple count. Make that 16 standards;-)

#18

I'm using PPPoE and set it to 8 by the SQM guide in OpenWRT wiki. Should I change the value to something else?
Unfortunately, I can't separate the torrent client from my computer. Should I use luci-app-qos instead of SQM? As I mentioned before, I used DD-WRT's QoS on TL-WR1043ND. It worked without any problems, on the same bandwidth shape.

#19

What kind of plan do you have? The bandwidth reminds me of cable/Docsis, but PPPoE would rather indicate XDSL/Fiber? Anyway 8 is almost always the wrong Overhead :wink:

Well, actually all you would need to do is use a different interface on the same computer an make the torrent client bind to that interface, but I guess that is not much easier than re-locating the torrent client to a different machine.

That is your call, if you can use port based rules to identify the torrent traffic qos-scripts/luci-app-qos might be better suited to reduce the impact of the torrents on the rest of your network (but so might using the throttling that the torrent client probably offers in its configuration).
So the per-IP fairness cake offers seems to be a much easier to use method compared to setting up complicated QOS rules (to make up for the fact that torrent clients/servers tend to be overly optimistic in their congestion measurements when faces with a non-latency-inducing AQM like cake/fq_codel), but the cost of that simplicity is that you will need to use a separate machine (heck a virtual machine should be enough). I am sorry that this design is not optimal for your use-case.

Best Regards

#20

I'm still getting used to the terms :slight_smile:
The plan is VDSL2 with maximum speed of 100Mbps, but I only get around 55Mbps (poor infrastructure).
Is 8 still wrong?

Doesn't sound easy, but creating virtual interface might help. I'll try that.

I've managed to get stable connection with QoS (again, in DD-WRT) by prioritizing all the packets but ACK with fq_codel and HTB (if I remember correctly).

Thanks for your help!