I tried sqm (cake) but whenever I use SQM I feel a little hitreg/animation delay in FPS games even when ping around 12ms constant. The whole game feels little sluggish. I've been a gamer for years and as soon as I turn off SQM in the middle of the match I get instant kills like I don't have to try hard. The gameplay feels perfect.
So what I'm thinking is that I mark gaming packets to either CS6 or CS7 and based on DSCP tags I want to allocate small bandwidth just for the games. For the rest of the traffic I don't care how much bandwidth it needs. (Or based on ports rather than DSCP tags)
I'm on Raspberry Pi4, my net plan is 10mbps both download & upload. But I get 100mbps for some sites and google services like YouTube, Drive etc. So capping bandwidth with SQM seems inutile.
So help is really appreciated thank you!
That sounds weird, especially if the router is a pi4 and you only have a 10 Mbps plan...
How does you /etc/config/sqm look lie and what is the output of tc -s qdisc? (Only answer this if you are still interested in figuring out why SQM fails).
This is not really how this works, for egress strict prioritization might actually work (the rest of the non-gaming traffic might se unpleasant delays, but you might be willing to accept that), but for the incoming traffic this is not going to work, you need a traffic shaper otherwise the oversized and under-managed queues on the ISP's side are going to severely impact gaming latency as well as all other packets.
Burt for that traffic shaper you need to find a rate at which the upstream buffers/queues do not overfill.
Well 10Mbps is certainly not terrible, but also not great either....
Unfortunately that is the only chance you have to control upstream bufferbloat, you need to throttle the traffic such that the slowest link is not at the ISP anymore, but in your router so that cake actually has a chance of better controlling the delays.
Thanks for the quick response. And yes so badly I wanna figure that out 
config queue 'eth1'
option qdisc 'cake'
option linklayer 'none'
option enabled '1'
option interface 'pppoe-wan'
option download '0'
option debug_logging '0'
option verbosity '5'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '1'
option ingress_ecn 'NOECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option script 'layer_cake.qos'
option eqdisc_opts 'diffserv8 nat ack-filter'
option upload '9800'
config queue
option enabled '1'
option interface 'br-lan'
option download '0'
option debug_logging '0'
option verbosity '5'
option qdisc 'cake'
option script 'layer_cake.qos'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '1'
option ingress_ecn 'NOECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option linklayer 'none'
option eqdisc_opts 'diffserv8 nat'
option upload '9800'
I agree but get this when I disable SQM the gameplay feels heaven. But I love how SQM shapes the whole traffic. I don't understand why I get the delay with SQM on?
This is never really ideal, but I guess on the Pi4 this really is just a single ethernet interface?
This also seems optimistic, splitting 10 Mbps in 8 tiers means each individual tier is pretty narrow. I would go back to diffserv4. Also make sure you only put your gaming traffic into either the Video or Voice class, keep almost everything else on Best Effort or Bulk.
Prioritization is a mostly a zero-sum game, for every packet that gets lower latency/faster service under load some other packet(s) will need to be delayed. If there are no packets to delay (because all data is in the Vide or Voice tin) others can also not be expedited. So yes prioritization can work well, but only if used sparingly/judiciously.
Try something:
- post the result of
tc -s qdisc - start playing a game
- play for a while
- while the game is still on, run
tc -s qdiscagain and post the results (later) - play some more, then stop
- wait say 5 minutes with low network load
- run
tc -s qdiscagain and post the results
The goal is to see whether cake's statistics reveal any smoking gun what might be happening.
Yes it works I was first skeptical too lol
Ahhh good catch I didn't think of that
Okay it'll take some time but I'll do it
No hurry, I will log out for the day soon anyway....
That's alright mate I've to go eat now and gotta take a break so I'll come back after an hour
Yes I've tried the script it works and gives similar result at least to me in gaming. Played around with it for days but I am sticking with simple SQM with cake for now. Later I'll try again 
1 Like
Game packets marked as CS7
Before the game:
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 317838399 bytes 530231 pkt (dropped 0, overlimits 0 requeues 2661)
backlog 0b 0p requeues 2661
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 35138799 bytes 82963 pkt (dropped 0, overlimits 0 requeues 186)
backlog 0b 0p requeues 186
maxpacket 1506 drop_overlimit 0 new_flow_count 28 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 138444256 bytes 160459 pkt (dropped 0, overlimits 0 requeues 2347)
backlog 0b 0p requeues 2347
maxpacket 1494 drop_overlimit 0 new_flow_count 72 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 48345777 bytes 84252 pkt (dropped 0, overlimits 0 requeues 111)
backlog 0b 0p requeues 111
maxpacket 1506 drop_overlimit 0 new_flow_count 49 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 63892827 bytes 116456 pkt (dropped 0, overlimits 0 requeues 17)
backlog 0b 0p requeues 17
maxpacket 1506 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 32016740 bytes 86101 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 0: dev eth1 root refcnt 2 bandwidth unlimited diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 665059932 bytes 577309 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 4352b of 15140Kb
capacity estimate: 0bit
min/max network layer size: 30 / 1514
min/max overhead-adjusted size: 30 / 1514
average network hdr offset: 14
Bulk Best Effort Voice
thresh 0bit 0bit 0bit
target 5ms 5ms 5ms
interval 100ms 100ms 100ms
pk_delay 0us 3us 0us
av_delay 0us 2us 0us
sp_delay 0us 1us 0us
backlog 0b 0b 0b
pkts 0 577309 0
bytes 0 665059932 0
way_inds 0 2233 0
way_miss 0 2098 0
way_cols 0 0 0
drops 0 0 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 3 0
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 1514 0
quantum 1514 1514 1514
qdisc cake 8032: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv8 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 6245 bytes 32 pkt (dropped 0, overlimits 6 requeues 0)
backlog 0b 0p requeues 0
memory used: 4800b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 54 / 1486
min/max overhead-adjusted size: 54 / 1486
average network hdr offset: 2
Tin 0 Tin 1 Tin 2 Tin 3 Tin 4 Tin 5 Tin 6 Tin 7
thresh 9800Kbit 8575Kbit 7503Kbit 6565Kbit 5744Kbit 5026Kbit 4398Kbit 3848Kbit
target 5ms 5ms 5ms 5ms 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms 100ms 100ms 100ms 100ms
pk_delay 0us 2us 383us 579us 0us 0us 1us 0us
av_delay 0us 0us 6us 10us 0us 0us 0us 0us
sp_delay 0us 0us 4us 9us 0us 0us 0us 0us
backlog 0b 0b 0b 0b 0b 0b 0b 0b
pkts 0 2 18 11 0 0 1 0
bytes 0 189 4728 1254 0 0 74 0
way_inds 0 0 0 0 0 0 0 0
way_miss 0 1 17 4 0 0 1 0
way_cols 0 0 0 0 0 0 0 0
drops 0 0 0 0 0 0 0 0
marks 0 0 0 0 0 0 0 0
ack_drop 0 0 0 0 0 0 0 0
sp_flows 0 1 8 1 0 0 1 0
bk_flows 0 0 0 0 0 0 0 0
un_flows 0 0 0 0 0 0 0 0
max_len 0 123 1486 432 0 0 74 0
quantum 300 300 300 300 300 300 300 300
qdisc cake 802f: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv8 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 1699 bytes 13 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 2752b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 40 / 557
min/max overhead-adjusted size: 40 / 557
average network hdr offset: 0
Tin 0 Tin 1 Tin 2 Tin 3 Tin 4 Tin 5 Tin 6 Tin 7
thresh 9800Kbit 8575Kbit 7503Kbit 6565Kbit 5744Kbit 5026Kbit 4398Kbit 3848Kbit
target 5ms 5ms 5ms 5ms 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms 100ms 100ms 100ms 100ms
pk_delay 0us 2us 3us 4us 0us 0us 1us 0us
av_delay 0us 0us 0us 0us 0us 0us 0us 0us
sp_delay 0us 0us 0us 0us 0us 0us 0us 0us
backlog 0b 0b 0b 0b 0b 0b 0b 0b
pkts 0 2 2 8 0 0 1 0
bytes 0 161 1114 364 0 0 60 0
way_inds 0 0 0 0 0 0 0 0
way_miss 0 1 1 3 0 0 1 0
way_cols 0 0 0 0 0 0 0 0
drops 0 0 0 0 0 0 0 0
marks 0 0 0 0 0 0 0 0
ack_drop 0 0 0 0 0 0 0 0
sp_flows 0 1 0 1 0 0 1 0
bk_flows 0 0 0 0 0 0 0 0
un_flows 0 0 0 0 0 0 0 0
max_len 0 109 557 60 0 0 60 0
quantum 300 300 300 300 300 300 300 300
While playing game:
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 321604747 bytes 538969 pkt (dropped 0, overlimits 0 requeues 2661)
backlog 0b 0p requeues 2661
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 36401609 bytes 88282 pkt (dropped 0, overlimits 0 requeues 186)
backlog 0b 0p requeues 186
maxpacket 1506 drop_overlimit 0 new_flow_count 28 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 138872996 bytes 161413 pkt (dropped 0, overlimits 0 requeues 2347)
backlog 0b 0p requeues 2347
maxpacket 1494 drop_overlimit 0 new_flow_count 72 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 49560445 bytes 85344 pkt (dropped 0, overlimits 0 requeues 111)
backlog 0b 0p requeues 111
maxpacket 1506 drop_overlimit 0 new_flow_count 49 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 64109402 bytes 117000 pkt (dropped 0, overlimits 0 requeues 17)
backlog 0b 0p requeues 17
maxpacket 1506 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 32660295 bytes 86930 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 0: dev eth1 root refcnt 2 bandwidth unlimited diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 666609284 bytes 586511 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 4352b of 15140Kb
capacity estimate: 0bit
min/max network layer size: 30 / 1514
min/max overhead-adjusted size: 30 / 1514
average network hdr offset: 14
Bulk Best Effort Voice
thresh 0bit 0bit 0bit
target 5ms 5ms 5ms
interval 100ms 100ms 100ms
pk_delay 0us 2us 0us
av_delay 0us 2us 0us
sp_delay 0us 1us 0us
backlog 0b 0b 0b
pkts 0 586511 0
bytes 0 666609284 0
way_inds 0 2295 0
way_miss 0 2305 0
way_cols 0 0 0
drops 0 0 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 1 0
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 1514 0
quantum 1514 1514 1514
qdisc cake 8032: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv8 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 3772593 bytes 8770 pkt (dropped 28, overlimits 5408 requeues 0)
backlog 0b 0p requeues 0
memory used: 167552b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 42 / 1506
min/max overhead-adjusted size: 42 / 1506
average network hdr offset: 14
Tin 0 Tin 1 Tin 2 Tin 3 Tin 4 Tin 5 Tin 6 Tin 7
thresh 9800Kbit 8575Kbit 7503Kbit 6565Kbit 5744Kbit 5026Kbit 4398Kbit 3848Kbit
target 5ms 5ms 5ms 5ms 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms 100ms 100ms 100ms 100ms
pk_delay 0us 23.1ms 27.3ms 367us 1.4ms 0us 263us 9.33ms
av_delay 0us 16.7ms 8.56ms 29us 38us 0us 6us 1.54ms
sp_delay 0us 41us 49us 4us 5us 0us 4us 5us
backlog 0b 0b 0b 0b 0b 0b 0b 0b
pkts 0 731 757 6617 71 0 143 479
bytes 0 1005205 377068 2288288 19173 0 10614 107472
way_inds 0 0 8 0 0 0 0 0
way_miss 0 9 252 103 13 0 3 86
way_cols 0 0 0 0 0 0 0 0
drops 0 12 11 5 0 0 0 0
marks 0 0 0 0 0 0 0 0
ack_drop 0 0 0 0 0 0 0 0
sp_flows 0 0 4 3 1 0 1 2
bk_flows 0 0 0 1 0 0 0 0
un_flows 0 0 0 0 0 0 0 0
max_len 0 1506 1506 1506 1484 0 90 567
quantum 300 300 300 300 300 300 300 300
qdisc cake 802f: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv8 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 1348607 bytes 9215 pkt (dropped 3, overlimits 824 requeues 0)
backlog 0b 0p requeues 0
memory used: 24768b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 29 / 1492
min/max overhead-adjusted size: 29 / 1492
average network hdr offset: 0
Tin 0 Tin 1 Tin 2 Tin 3 Tin 4 Tin 5 Tin 6 Tin 7
thresh 9800Kbit 8575Kbit 7503Kbit 6565Kbit 5744Kbit 5026Kbit 4398Kbit 3848Kbit
target 5ms 5ms 5ms 5ms 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms 100ms 100ms 100ms 100ms
pk_delay 0us 201us 3.25ms 398us 0us 0us 6us 132us
av_delay 0us 13us 293us 18us 0us 0us 2us 8us
sp_delay 0us 3us 7us 4us 0us 0us 2us 4us
backlog 0b 0b 0b 0b 0b 0b 0b 0b
pkts 0 659 496 7417 0 0 146 500
bytes 0 44771 706058 554717 0 0 8760 35897
way_inds 0 0 0 0 0 0 0 0
way_miss 0 4 20 120 0 0 5 89
way_cols 0 0 0 0 0 0 0 0
drops 0 0 1 0 0 0 0 0
marks 0 0 0 0 0 0 0 0
ack_drop 0 2 0 0 0 0 0 0
sp_flows 0 0 0 1 0 0 1 1
bk_flows 0 0 1 0 0 0 0 0
un_flows 0 0 0 0 0 0 0 0
max_len 0 438 1480 1492 0 0 64 376
quantum 300 300 300 300 300 300 300 300
Closed the game and waited 5 mins:
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 331614766 bytes 550646 pkt (dropped 0, overlimits 0 requeues 2661)
backlog 0b 0p requeues 2661
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 37520048 bytes 90940 pkt (dropped 0, overlimits 0 requeues 186)
backlog 0b 0p requeues 186
maxpacket 1506 drop_overlimit 0 new_flow_count 28 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 139570467 bytes 162761 pkt (dropped 0, overlimits 0 requeues 2347)
backlog 0b 0p requeues 2347
maxpacket 1494 drop_overlimit 0 new_flow_count 72 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 52648762 bytes 88093 pkt (dropped 0, overlimits 0 requeues 111)
backlog 0b 0p requeues 111
maxpacket 1506 drop_overlimit 0 new_flow_count 49 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 64835011 bytes 118361 pkt (dropped 0, overlimits 0 requeues 17)
backlog 0b 0p requeues 17
maxpacket 1506 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 37040478 bytes 90491 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 0: dev eth1 root refcnt 2 bandwidth unlimited diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 667809836 bytes 595847 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 4352b of 15140Kb
capacity estimate: 0bit
min/max network layer size: 30 / 1514
min/max overhead-adjusted size: 30 / 1514
average network hdr offset: 14
Bulk Best Effort Voice
thresh 0bit 0bit 0bit
target 5ms 5ms 5ms
interval 100ms 100ms 100ms
pk_delay 0us 5us 0us
av_delay 0us 2us 0us
sp_delay 0us 1us 0us
backlog 0b 0b 0b
pkts 0 595847 0
bytes 0 667809836 0
way_inds 0 2515 0
way_miss 0 2510 0
way_cols 0 0 0
drops 0 0 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 10 0
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 1514 0
quantum 1514 1514 1514
qdisc cake 8032: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv8 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 13782612 bytes 20447 pkt (dropped 130, overlimits 20301 requeues 0)
backlog 0b 0p requeues 0
memory used: 223616b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 42 / 1506
min/max overhead-adjusted size: 42 / 1506
average network hdr offset: 14
Tin 0 Tin 1 Tin 2 Tin 3 Tin 4 Tin 5 Tin 6 Tin 7
thresh 9800Kbit 8575Kbit 7503Kbit 6565Kbit 5744Kbit 5026Kbit 4398Kbit 3848Kbit
target 5ms 5ms 5ms 5ms 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms 100ms 100ms 100ms 100ms
pk_delay 0us 24.5ms 22.5ms 38.7ms 21.7ms 0us 297us 10.5ms
av_delay 0us 12ms 9.1ms 13.8ms 3.38ms 0us 12us 3.28ms
sp_delay 0us 756us 1.87ms 4us 5us 0us 4us 120us
backlog 0b 0b 0b 0b 0b 0b 0b 0b
pkts 0 1214 2084 15304 463 0 482 1030
bytes 0 1638400 928350 10799925 272538 0 35750 279913
way_inds 0 0 123 0 0 0 0 0
way_miss 0 12 557 223 27 0 7 119
way_cols 0 0 0 0 0 0 0 0
drops 0 15 12 101 2 0 0 0
marks 0 0 0 0 0 0 0 0
ack_drop 0 0 0 0 0 0 0 0
sp_flows 0 0 0 8 4 0 1 0
bk_flows 0 0 0 0 0 0 0 0
un_flows 0 0 0 0 0 0 0 0
max_len 0 1506 1506 1506 1484 0 90 622
quantum 300 300 300 300 300 300 300 300
qdisc cake 802f: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv8 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 2343719 bytes 18545 pkt (dropped 5, overlimits 1655 requeues 0)
backlog 0b 0p requeues 0
memory used: 24768b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 29 / 1492
min/max overhead-adjusted size: 29 / 1492
average network hdr offset: 0
Tin 0 Tin 1 Tin 2 Tin 3 Tin 4 Tin 5 Tin 6 Tin 7
thresh 9800Kbit 8575Kbit 7503Kbit 6565Kbit 5744Kbit 5026Kbit 4398Kbit 3848Kbit
target 5ms 5ms 5ms 5ms 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms 100ms 100ms 100ms 100ms
pk_delay 0us 376us 2.4ms 1.47ms 0us 0us 121us 59us
av_delay 0us 15us 249us 116us 0us 0us 9us 7us
sp_delay 0us 3us 4us 3us 0us 0us 5us 4us
backlog 0b 0b 0b 0b 0b 0b 0b 0b
pkts 0 3861 577 12753 0 0 514 845
bytes 0 261608 813211 1182603 0 0 31440 56581
way_inds 0 0 0 6 0 0 0 0
way_miss 0 8 25 272 0 0 13 132
way_cols 0 0 0 0 0 0 0 0
drops 0 0 1 0 0 0 0 0
marks 0 0 0 0 0 0 0 0
ack_drop 0 4 0 0 0 0 0 0
sp_flows 0 0 1 10 0 0 1 1
bk_flows 0 0 0 1 0 0 0 0
un_flows 0 0 0 0 0 0 0 0
max_len 0 1259 1480 1492 0 0 104 823
quantum 300 300 300 300 300 300 300 300
I think while testing I've had Hardware flow offloading enabled.
You seem to have a spurious/superflous cake instance on eth1, unless that is there on purpose think about removing that...
On br-lan and pppoe-wan you use diffserv8, unless you really have that many priority levels you need, consider switching back to diffserv4 or diffserv3. Keep in mind that prioritisation is a zero-sum game, for every packet getting better service (lower delay and jitter) some other packets will get worse service, if there are not enough packets available to "take that hit" other packets can not really be expedited. So "less-is-more" in prioritization if lowering bufferbloat is the goal.
This is quite some peak delay, not suure it affects your game though.
that seems to persist even after the game.
I have zero experience with that, so can not offer an informed opinion on whether that helps or hurts. Maybe try without it?
Observed DSCP markings don't work with Hardware flow offloading. All packets were marked CS0. I'll test again and post the results soon
Yes I'm considering to keep diffserv4
I think this could be it. Do you think I should consider using Qosify?
It's bcz of net.core.default_qdisc=cake in sysctl.conf I removed that line and now it shows:
root@OpenWrt:~# tc qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev eth0 root
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc cake 800b: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
qdisc cake 800e: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Great that explains it, btw I think fq_codel is the better generic qdisc, cake really is taylored for more specific use cases and can eat up way more CPU cycles.
1 Like
Me and my friend figured out why ping was getting high from 4 ms to 86ms at load is bcz of I was stress testing with 16 connections in IDM. I changed it's default max connection to 2 and ping is now around 30 ms rather being above 80ms with no SQM. With SQM enabled at load pages are opening faster and even everything is working as it should. Not sure about hitreg in games yet. I'm still testing.
What is IDM?
Please report back on potential effects on the game
creates multiple connections to download the file and assembles downloaded parts into one in the end
Thanks, question with SQM active do you still see the effect of IDM with 16 connections on the ping measured on other devices?
It doesn't affect ping but browsing is wayyy slow like 2G speed on the same device meanwhile on other devices speed is okay-ish like it takes 2 seconds to load every image on bbc news site
Here's the result without offload
Before the game:
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 95974400 bytes 119939 pkt (dropped 0, overlimits 0 requeues 21)
backlog 0b 0p requeues 21
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 4324030 bytes 11708 pkt (dropped 0, overlimits 0 requeues 2)
backlog 0b 0p requeues 2
maxpacket 814 drop_overlimit 0 new_flow_count 2 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 12221471 bytes 17319 pkt (dropped 0, overlimits 0 requeues 11)
backlog 0b 0p requeues 11
maxpacket 1392 drop_overlimit 0 new_flow_count 3 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 1889435 bytes 6019 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 814 drop_overlimit 0 new_flow_count 1 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 9492009 bytes 25639 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 68047455 bytes 59254 pkt (dropped 0, overlimits 0 requeues 6)
backlog 0b 0p requeues 6
maxpacket 1392 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 8312588 bytes 54578 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 8014: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 2685 bytes 18 pkt (dropped 0, overlimits 6 requeues 0)
backlog 0b 0p requeues 0
memory used: 1536b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 42 / 503
min/max overhead-adjusted size: 42 / 503
average network hdr offset: 1
Bulk Best Effort Video Voice
thresh 612496bit 9800Kbit 4900Kbit 2450Kbit
target 29.7ms 5ms 5ms 7.42ms
interval 125ms 100ms 100ms 102ms
pk_delay 0us 0us 34us 3us
av_delay 0us 0us 0us 0us
sp_delay 0us 0us 0us 0us
backlog 0b 0b 0b 0b
pkts 0 0 15 3
bytes 0 0 2523 162
way_inds 0 0 0 0
way_miss 0 0 5 3
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 1 1
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 0 0 503 66
quantum 300 300 300 300
qdisc cake 8011: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 3933 bytes 20 pkt (dropped 0, overlimits 2 requeues 0)
backlog 0b 0p requeues 0
memory used: 2752b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 40 / 720
min/max overhead-adjusted size: 40 / 720
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 612496bit 9800Kbit 4900Kbit 2450Kbit
target 29.7ms 5ms 5ms 7.42ms
interval 125ms 100ms 100ms 102ms
pk_delay 0us 1us 108us 5us
av_delay 0us 0us 1us 0us
sp_delay 0us 0us 1us 0us
backlog 0b 0b 0b 0b
pkts 0 1 14 5
bytes 0 557 3056 320
way_inds 0 0 0 0
way_miss 0 1 4 4
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 1 1 1
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 0 557 720 83
quantum 300 300 300 300
While in the game:
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 109718214 bytes 135015 pkt (dropped 0, overlimits 0 requeues 21)
backlog 0b 0p requeues 21
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 4600998 bytes 12251 pkt (dropped 0, overlimits 0 requeues 2)
backlog 0b 0p requeues 2
maxpacket 814 drop_overlimit 0 new_flow_count 2 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 15841797 bytes 20491 pkt (dropped 0, overlimits 0 requeues 11)
backlog 0b 0p requeues 11
maxpacket 1392 drop_overlimit 0 new_flow_count 3 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 1962100 bytes 6361 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 814 drop_overlimit 0 new_flow_count 1 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 16442644 bytes 31076 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 70870675 bytes 64836 pkt (dropped 0, overlimits 0 requeues 6)
backlog 0b 0p requeues 6
maxpacket 1392 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 9931314 bytes 65619 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 8014: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 13746499 bytes 15094 pkt (dropped 94, overlimits 20819 requeues 0)
backlog 0b 0p requeues 0
memory used: 262080b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 42 / 1506
min/max overhead-adjusted size: 42 / 1506
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 612496bit 9800Kbit 4900Kbit 2450Kbit
target 29.7ms 5ms 5ms 7.42ms
interval 125ms 100ms 100ms 102ms
pk_delay 17.6ms 43.9ms 14.4ms 14.1ms
av_delay 615us 13.8ms 7.74ms 2.28ms
sp_delay 8us 661us 4us 9us
backlog 0b 0b 0b 0b
pkts 66 574 14005 543
bytes 31349 417164 13297756 125977
way_inds 0 0 0 0
way_miss 8 29 215 141
way_cols 0 0 0 0
drops 0 4 90 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 3 1 1
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 1484 1506 1596 562
quantum 300 300 300 300
qdisc cake 8011: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 1379749 bytes 11060 pkt (dropped 0, overlimits 2050 requeues 0)
backlog 0b 0p requeues 0
memory used: 24768b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 29 / 1492
min/max overhead-adjusted size: 29 / 1492
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 612496bit 9800Kbit 4900Kbit 2450Kbit
target 29.7ms 5ms 5ms 7.42ms
interval 125ms 100ms 100ms 102ms
pk_delay 0us 1.68ms 105us 306us
av_delay 0us 240us 7us 13us
sp_delay 0us 10us 3us 4us
backlog 0b 0b 0b 0b
pkts 0 222 10423 415
bytes 0 246083 1101378 32288
way_inds 0 0 3 4
way_miss 0 23 239 104
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 2 0 1
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 0 1480 1492 576
quantum 300 300 300 300
Closed the game waited for 5 mins:
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 205175973 bytes 218676 pkt (dropped 0, overlimits 0 requeues 21)
backlog 0b 0p requeues 21
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 5038978 bytes 13389 pkt (dropped 0, overlimits 0 requeues 2)
backlog 0b 0p requeues 2
maxpacket 814 drop_overlimit 0 new_flow_count 2 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 43972623 bytes 41545 pkt (dropped 0, overlimits 0 requeues 11)
backlog 0b 0p requeues 11
maxpacket 1392 drop_overlimit 0 new_flow_count 3 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 5551531 bytes 9236 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 814 drop_overlimit 0 new_flow_count 1 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 75037364 bytes 74161 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 75575477 bytes 80345 pkt (dropped 0, overlimits 0 requeues 6)
backlog 0b 0p requeues 6
maxpacket 1392 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 13607120 bytes 99924 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 8014: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
Sent 109204258 bytes 98755 pkt (dropped 473, overlimits 177647 requeues 0)
backlog 2784b 2p requeues 0
memory used: 347776b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 42 / 1506
min/max overhead-adjusted size: 42 / 1506
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 612496bit 9800Kbit 4900Kbit 2450Kbit
target 29.7ms 5ms 5ms 7.42ms
interval 125ms 100ms 100ms 102ms
pk_delay 13ms 35.9ms 59.9ms 16.2ms
av_delay 552us 13.6ms 50.6ms 4.37ms
sp_delay 6us 1.09ms 102us 92us
backlog 0b 2784b 0b 0b
pkts 145 3460 94604 1021
bytes 50870 3709410 105822536 274251
way_inds 0 0 0 0
way_miss 14 65 279 200
way_cols 0 0 0 0
drops 0 38 435 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 1 2 4 1
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 1484 1506 3096 562
quantum 300 300 300 300
qdisc cake 8011: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
Sent 4300709 bytes 45348 pkt (dropped 2, overlimits 5989 requeues 0)
backlog 0b 0p requeues 0
memory used: 48640b of 4Mb
capacity estimate: 9800Kbit
min/max network layer size: 29 / 1492
min/max overhead-adjusted size: 29 / 1492
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 612496bit 9800Kbit 4900Kbit 2450Kbit
target 29.7ms 5ms 5ms 7.42ms
interval 125ms 100ms 100ms 102ms
pk_delay 0us 3.75ms 675us 137us
av_delay 0us 628us 25us 9us
sp_delay 0us 16us 6us 4us
backlog 0b 0b 0b 0b
pkts 0 950 43773 627
bytes 0 949713 3310008 43744
way_inds 0 0 3 22
way_miss 0 50 315 136
way_cols 0 0 0 0
drops 0 2 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 3 0
bk_flows 0 2 0 0
un_flows 0 0 0 0
max_len 0 1480 1600 576
quantum 300 300 300 300
Also figured out why pk_delay is higher bcz of two SQM instances. This time I only applied SQM on pppoe-wan(eth1) and the delay reduced to 97us instead 15ms
root@OpenWrt:~# tc qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev eth0 root
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc cake 8020: dev pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash ack-filter split-gso rtt 100ms raw overhead 0
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 8021: dev ifb4pppoe-wan root refcnt 2 bandwidth 9800Kbit diffserv4 triple-isolate nat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
So why is that?