Yes I'm aware shouldn't use bridge interface in sqm but it works for me. You can give it a shot.
2 Likes
In the old config before dsa was good to do sqm on wan up and eth0 up but with dsa config only wan is necessary in all of my trials gets best results... Also i prefer diffserv4 no need for 8 tins but all personal preference 
2 Likes
Also could you try with and without "datacenter" in Advanced option string to pass to the egress queueing disciplines on both instances? Do you get better gameplay with that parameter?
I will add it to my sqm on wan and report back... I will also share my config file when i get home... Solidus1983 taught me to keep my network simple...
1 Like
i will test with data center
but if modifiy with ef not CS6 latency doesnt work like this
#!/bin/ash -x
IPT="iptables"
iptmark="iptables"
##ipset for streming sites.they are bening filled by dnsmasq
ipset create streaming hash:ip
ipset create usrcdn hash:ip
ipset create bulk hash:ip
ipset create latsens hash:ip
#flush mangle table
$IPT -t mangle -F PREROUTING
#Clear interface dscp marks, we don't trust ISP marks(also to use our own marks).
$IPT -t mangle -A PREROUTING -i wan -j DSCP --set-dscp 0
########################################
#Latency Sensitive (gaming/voip)
########################################
##Latency Sensitive
$IPT -t mangle -A PREROUTING -p icmp -j DSCP --set-dscp-class EF ##dscp tag for ping packets
$IPT -t mangle -A PREROUTING -m set --match-set latsens src -j DSCP --set-dscp-class EF ## set dscp tag for our Latency Sensitive (latsens) ipset
$IPT -t mangle -A PREROUTING -p tcp -m conntrack --ctorigsrc 192.168.2.160 -m multiport ! --ports 80,443,8080 -j DSCP --set-dscp-class EF -m comment --comment "PS5" #i don't have PS4, change the ip according to your ip setting.
####Latency Sensitive Ports####
sport00="3074"
sport4="9305:9308"
sport16="3478:3479"
dport00="3074:3075,3478:3479,3658:3659,9305:9308"
$IPT -t mangle -A PREROUTING -p udp -m multiport --sport $sport00 -j DSCP --set-dscp-class EF
$IPT -t mangle -A PREROUTING -p udp -m multiport --sport $sport4 -j DSCP --set-dscp-class AF41
$IPT -t mangle -A PREROUTING -p udp -m multiport --sport $sport16 -j DSCP --set-dscp-class AF41
$IPT -t mangle -A PREROUTING -p udp -m multiport --dport $dport00 -j DSCP --set-dscp-class EF
$IPT -t mangle -N dscp_mark > /dev/null 2>&1
$IPT -t mangle -F dscp_mark
#check if POSTROUTING already exits then jumps to our tables if not, add them
$IPT -t mangle -L POSTROUTING -n | grep dscp_mark || $IPT -t mangle -A POSTROUTING -o br-lan -j dscp_mark
iptmark() {
$IPT -t mangle -A dscp_mark "$@"
}
#A robust 2 rules to detect realtime traffic
iptmark -p udp -m hashlimit --hashlimit-name udp_high_prio --hashlimit-above 100/sec --hashlimit-burst 50 --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55 -m comment --comment "connmark for udp"
#unmarked UDP streams with small packets get CS6
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 22,25,53,67,68,123,143,161,162,514,80,443,8080 -m connbytes --connbytes 0:940 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6 -m comment --comment "small udp connection gets CS6"
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 137,442,444,445,446,447 -m connbytes --connbytes 0:940 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6 -m comment --comment "small udp connection gets CS6"
#large udp streams like video call get AF41
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 22,25,53,67,68,123,143,161,162,514,80,443,8080 -m connbytes --connbytes 940:1500 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class AF21 -m comment --comment "large udp connection gets AF21"
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 137,442,444,445,446,447 -m connbytes --connbytes 940:1500 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6 -m comment --comment "small udp connection gets CS6"
#DNS traffic both udp and tcp
iptmark -p udp -m multiport --port 53 -j DSCP --set-dscp-class CS6 -m comment --comment "DNS udp"
#NTP
iptmark -p udp -m multiport --port 123 -j DSCP --set-dscp-class CS6 -m comment --comment "NTP udp"
#High priority ipset, i use for pubgM
iptmark ! -p udp -m set --match-set latsens src,dst -j DSCP --set-dscp-class EF -m comment --comment "latency sensitive ipset" ## set dscp tag for Latency Sensitive (latsens) ipset,udp
iptmark -p tcp -m set --match-set latsens src,dst -j DSCP --set-dscp-class EF -m comment --comment "latency sensitive ipset" ## set dscp tag for Latency Sensitive (latsens) ipset
###########
##Browsing
###########
#medium priority for browsing
iptmark -p tcp -m multiport --ports 80,443 -j DSCP --set-dscp-class CS3 -m comment --comment "Browsing at CS3"
##################
#TCP SYN,ACK flows
##################
#Make sure ACK,SYN packets get priority (to avoid upload speed limiting our download speed)
iptmark -p tcp --tcp-flags ALL ACK -m length --length :128 -j DSCP --set-dscp-class CS3
iptmark -p tcp --tcp-flags ALL SYN -m length --length :666 -j DSCP --set-dscp-class CS3
#Small packet is probably interactive or flow control
iptmark -m dscp ! --dscp 24 -m dscp ! --dscp 18 -m dscp ! --dscp 34 -m dscp ! --dscp 40 -m dscp ! --dscp 48 -m length --length 0:500 -j DSCP --set-dscp-class CS3
#Small packet connections: multi purpose (don't harm since not maxed out)
iptmark -m dscp ! --dscp 24 -m dscp ! --dscp 18 -m dscp ! --dscp 34 -m dscp ! --dscp 40 -m dscp ! --dscp 48 -m connbytes --connbytes 0:250 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS3
########################################
#Streaming Media (videos/audios)
########################################
#Known video streams sites like netflix
iptmark -m set --match-set streaming src,dst -j DSCP --set-dscp-class AF41 -m comment --comment "video audio stream ipset"
#known usrcdn like google or akamai
iptmark -m set --match-set usrcdn src,dst -j DSCP --set-dscp-class AF21 -m comment --comment "usrcdn ipset"
#########################################
#Background Traffic (Bulk/file transfer)
#########################################
#bulk traffic ipset, like windows udates and steam updates/downloads
iptmark -p tcp -m set --match-set bulk src,dst -j DSCP --set-dscp-class CS1 -m comment --comment "bulk traffic ipset"
iptmark -p udp -m set --match-set bulk src,dst -j DSCP --set-dscp-class CS1 -m comment --comment "bulk traffic ipset"
iptmark -p tcp -m connbytes --connbytes 350000: --connbytes-dir both --connbytes-mode bytes -m dscp --dscp-class CS3 -j DSCP --set-dscp-class CS1 -m comment --comment "Downgrade CS3 to CS1 for bulk tcp traffic"
1 Like
I read that for games if you really wanna use higher DCSP then use EF for lan or else CS5,CS6 or CS7 (I use CS7 it's overkill but meh) for wifi games
2 Likes
I only go up to cs6 any higher doesnt work but if you play wired EF is what i read that gets good success
1 Like
I play wired yes 
So datacentre will set a considerably smaller target and interval, 5 µs and 100µs compared to the default 5ms and 100ms. Unless your servers are really close by (same rack) this will most likely just lead to excessive dropping/marking (if the gaming flows ever actually queue up packets that is).... Now empirical data always wins against theoretical considerations (as networking is not amenable to closed form mathematical solutions), but at least also try metro (500µs/1000µs) and/or regional (1.5ms/30ms)...
3 Likes
I very much suggest starting with regional. Even devices on my home network can't ping each other through a few switches in 100us.
1 Like
Back then I tried regional and setting RTT to 40ms (since 40 is what I get max ping on COD). The gameplay felt a bit better than default 100ms rtt. So I took drastic approach putting datacenter and gameplay got much better not just the hitreg but the number of times enemies jumping in front of you and killing you before you could react significantly reduced. Idk how cake algorithm works but it almost feels like by default cake waits till the target or until the assigned queues are filled with the packets. Even at idle load it made difference to me in FPS games. I'll look into algo later but correct me please if I'm wrong.
3 Likes
I agree added nat datacentre last night played a few rounds definitely felt exactly what you described good work Alan!!!
2 Likes
i confirm too ! good work
1 Like
Hello everyone,
I have been following this thread and a few others older threads in order to try understanding the use of DSCP marks with Cake SQM. I'm normally able to decipher most of what I read, but in this case I'm afraid it all feels like this involves some kind of black magic...
I have posted above my config (post 194), asking for help of more experienced users, but got no answers do far. So I went ahead and added another queue referring to my lan and inverted the download with the upload speeds. I can still surf the web and everything seems to work, but don't know how to verify that my setup is indeed doing its job.
Once again, here is my setup:
config queue 'eth1'
option interface 'eth1'
option qdisc 'cake'
option ingress_ecn 'ECN'
option itarget 'auto'
option etarget 'auto'
option enabled '1'
option download '200000'
option upload '100000'
option script 'layer_cake.qos'
option linklayer 'ethernet'
option overhead '44'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option iqdisc_opts 'diffserv4 nat dual-dsthost ingress'
option eqdisc_opts 'diffserv4 nat dual-srchost'
config queue
option interface 'eth5.210'
option enabled '1'
option download '200000'
option upload '60000'
option script 'layer_cake.qos'
option qdisc 'cake'
option itarget 'auto'
option etarget 'auto'
option linklayer 'ethernet'
option overhead '44'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option ingress_ecn 'ECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option iqdisc_opts 'diffserv4 nat dual-dsthost ingress'
option eqdisc_opts 'diffserv4 nat dual-srchost'
config queue
option interface 'br-lan'
option enabled '1'
option download '100000'
option upload '200000'
option script 'layer_cake.qos'
option qdisc 'cake'
option itarget 'auto'
option etarget 'auto'
option linklayer 'ethernet'
option overhead '44'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option ingress_ecn 'ECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option iqdisc_opts 'diffserv4 nat dual-dsthost ingress'
option eqdisc_opts 'diffserv4 nat dual-srchost'
I have seen people using the command tc -s qdisc above, but I must admit I don't know how to interpret it's output:
# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
Sent 1287367373 bytes 1060996 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 8015: dev eth1 root refcnt 2 bandwidth 100Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
Sent 317391430 bytes 789202 pkt (dropped 1, overlimits 734084 requeues 0)
backlog 0b 0p requeues 0
memory used: 162528b of 5000000b
capacity estimate: 100Mbit
min/max network layer size: 28 / 1500
min/max overhead-adjusted size: 72 / 1544
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 6250Kbit 100Mbit 50Mbit 25Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 90us 154us 35us 6us
av_delay 7us 31us 3us 2us
sp_delay 2us 2us 2us 1us
backlog 0b 0b 0b 0b
pkts 467257 240540 66462 14944
bytes 207825914 96524539 10721249 2321222
way_inds 0 2097 450 99
way_miss 105 1453 2078 523
way_cols 0 0 0 0
drops 0 1 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 1 0 1 1
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 1506 1514 1506 1406
quantum 300 1514 1514 762
qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
Sent 1259946102 bytes 1041447 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth5 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
Sent 129729 bytes 1473 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc cake 8011: dev br-lan root refcnt 2 bandwidth 200Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
Sent 1287367415 bytes 1060997 pkt (dropped 0, overlimits 1273446 requeues 0)
backlog 0b 0p requeues 0
memory used: 188232b of 10000000b
capacity estimate: 200Mbit
min/max network layer size: 28 / 1500
min/max overhead-adjusted size: 72 / 1544
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 12500Kbit 200Mbit 100Mbit 50Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 75us 66us 30us 6us
av_delay 30us 33us 4us 3us
sp_delay 1us 3us 2us 2us
backlog 0b 0b 0b 0b
pkts 570441 402288 71556 16712
bytes 722307017 507954393 54009141 3096864
way_inds 18484 2646 1615 17
way_miss 107 4158 6186 3688
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 13 1
bk_flows 1 0 1 0
un_flows 0 0 0 0
max_len 63588 59046 13626 1406
quantum 381 1514 1514 1514
qdisc ingress ffff: dev br-lan parent ffff:fff1 ----------------
Sent 295306811 bytes 826956 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8019: dev eth5.210 root refcnt 2 bandwidth 60Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
Sent 128703 bytes 1470 pkt (dropped 0, overlimits 34 requeues 0)
backlog 0b 0p requeues 0
memory used: 1536b of 4Mb
capacity estimate: 60Mbit
min/max network layer size: 28 / 576
min/max overhead-adjusted size: 72 / 620
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 3750Kbit 60Mbit 30Mbit 15Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 0us 10us 7us 10us
av_delay 0us 3us 1us 5us
sp_delay 0us 1us 1us 1us
backlog 0b 0b 0b 0b
pkts 0 330 176 964
bytes 0 29244 10340 89119
way_inds 0 0 0 0
way_miss 0 153 175 248
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 1 1
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 0 167 342 590
quantum 300 1514 915 457
qdisc ingress ffff: dev eth5.210 parent ffff:fff1 ----------------
Sent 58904 bytes 896 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8012: dev ifb4br-lan root refcnt 2 bandwidth 100Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 44
Sent 309988677 bytes 826139 pkt (dropped 817, overlimits 678071 requeues 0)
backlog 0b 0p requeues 0
memory used: 213760b of 5000000b
capacity estimate: 100Mbit
min/max network layer size: 32 / 1500
min/max overhead-adjusted size: 76 / 1544
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 6250Kbit 100Mbit 50Mbit 25Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 168us 208us 27us 12us
av_delay 22us 23us 10us 6us
sp_delay 1us 10us 3us 3us
backlog 0b 0b 0b 0b
pkts 870 814950 1511 9625
bytes 52200 310456922 119712 589957
way_inds 24 28252 0 0
way_miss 55 17069 375 14
way_cols 0 0 0 0
drops 0 817 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 1 2 1 0
bk_flows 0 1 0 0
un_flows 0 0 0 0
max_len 60 34178 1406 590
quantum 300 1514 1514 762
qdisc noqueue 0: dev wg0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8016: dev ifb4eth1 root refcnt 2 bandwidth 200Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 44
Sent 1284305218 bytes 1039498 pkt (dropped 1949, overlimits 1243344 requeues 0)
backlog 0b 0p requeues 0
memory used: 2384000b of 10000000b
capacity estimate: 200Mbit
min/max network layer size: 46 / 1492
min/max overhead-adjusted size: 90 / 1536
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 12500Kbit 200Mbit 100Mbit 50Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 1.65ms 82us 0us 25us
av_delay 1.02ms 20us 0us 14us
sp_delay 114us 6us 0us 10us
backlog 0b 0b 0b 0b
pkts 2585 1034157 0 4705
bytes 3845759 1282481267 0 644930
way_inds 0 3468 0 0
way_miss 24 3463 0 23
way_cols 0 0 0 0
drops 2 1947 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 14 0 1
bk_flows 0 2 0 0
un_flows 0 0 0 0
max_len 60240 66516 0 1406
quantum 381 1514 1514 1514
qdisc cake 801a: dev ifb4eth5.210 root refcnt 2 bandwidth 200Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 44
Sent 71448 bytes 896 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 2304b of 10000000b
capacity estimate: 200Mbit
min/max network layer size: 42 / 564
min/max overhead-adjusted size: 86 / 608
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 12500Kbit 200Mbit 100Mbit 50Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 0us 21us 0us 9us
av_delay 0us 12us 0us 2us
sp_delay 0us 8us 0us 2us
backlog 0b 0b 0b 0b
pkts 0 791 0 105
bytes 0 64846 0 6602
way_inds 0 0 0 0
way_miss 0 259 0 2
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 1 0 1
bk_flows 0 1 0 0
un_flows 0 0 0 0
max_len 0 578 0 362
quantum 381 1514 1514 1514
So I kindly ask once again for someone to help validating my config. Thanks in advance!
If more information is needed, pls kindly advise and I will post here ASAP.
Cesar
1 Like
Hi you play wireless or wired? The dscp is cs6 I think for wireless and wired good ef seems good I can send my config if you want
1 Like
Wired. Thanks for answering.
Do you think my config is correct?
What is your wan interface!!, you need to use a sqm on wan egress and lan egress.
config queue ''
option interface 'your wan'
option qdisc 'cake'
option ingress_ecn 'ECN'
option itarget 'auto'
option etarget 'auto'
option enabled '1'
option download '0'
option upload 'you uploud intenet speed by 85%'
option script 'layer_cake.qos'
option linklayer 'ethernet'
option overhead '44'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option iqdisc_opts
option eqdisc_opts 'diffserv4 nat dual-srchost'
config queue
option interface 'your lan'
option enabled '1'
option download '0'
option upload 'your donwload intenet speed by 85%'
option script 'layer_cake.qos'
option qdisc 'cake'
option itarget 'auto'
option etarget 'auto'
option linklayer 'ethernet'
option overhead '44'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option ingress_ecn 'ECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option iqdisc_opts
option eqdisc_opts 'diffserv4 nat dual-dsthost ingress'
1 Like
Hey @EXREYFOX , thanks for stepping in.
In fact, as mentioned in my original post (post 194), I have two WAN interfaces: wan and wanb. Access to both is managed by mwan3 in a 60% wan / 40% wanb share ratio.
The first two queues you see in my config above, refer to the two wan interfaces, first is wan, second is wanb. The third queue refer to the br-lan bridge.
If I follow your post correcty, I must change my two wan queues to contemplate ECN only on ingress, NOECN on outgress. Then set download to 0 and set upload to 85% of my upload speed (which equates to 85000). Finally, remove the contents of iqdisc-opts and set eqdisc_opts to diffserv4 nat dual-srchost.
For the lan queue do the same thing, but use 85% of the download speed (equates to 170000) as the upload figure and set eqdisc_opts to diffserv4 nat dual-dsthost ingress.
Is this correct? If so, I must admit I would have never figured this out myself without your help...
1 Like
In a scenario of playing with dscp tags the sqm config needs to be like that ( many post talk about dscp have that type)
You can try .
2 Likes
Thanks once again for your valuable help. The problem I faced was that I read so many posts, from different authors (posted in different times and with varying goals), that I got really confused could not put my finger on the exact thing to do. I will try you suggestions in a few moments and post back with the results.
TIA
2 Likes