Ultimate SQM settings: Layer_cake + DSCP marks

I grabbed your second packet capture and looked, what I see is a bimodal distribution of packet lengths. I think the data and some control information are traveling in the same UDP stream. Or perhaps the large packets are some kind of compression reset (like sending a new set of parameters or something). I isolated one conversation (based on ip address and ports) and found that there's a bunch of packets around 200-300 bytes or less and another bunch around 1000 bytes. Something like 1:640 bytes would capture about 60% of the stream. I'd suggest something like that.

Looking in your first packet capture (game), the maximum packet size is 555 bytes

So try out something like UDP packets with --length 1:700. It's also the case that 700 bytes takes about 5.6 ms to send at 1Mbps so that's not an enormous delay in case you have a false positive.

1 Like

Oh, nice analysis.
i will try 1:700 length,is it better to insert conntrack match with the same rule?
like this:

Conntrack match is probably much more intensive, as it has to keep track of statistics on the whole flow, but it will ensure the whole flow is prioritized not just the short packets, so it's up to you. Maybe try it with UDP conntrack avgpacket between 1:700 both directions... See how it works in a packet capture.

Edit: suggested rule:

iptables -t mangle -A OUTPUT -p udp -m connbytes --connbytes 0:700 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6
1 Like

Mmh, Not sure about marking different packets from the same flow with different dscps, this can lead to packet reordering at the receiver, which in turn might have some side-effects. Admitted, this is UDP not TCP but still, reordering seems better avoided.

So what is the right way to do it.
Cause port/ip is hard to guess and there's no available traffic identifier ?!

i will see if there's a noticeable side effects!
@dlakelan

my plan is to prioritize the whole flow not just the short packets, but as you know it's not possible to match
port/ip cause it's change on every session!

i think it should be prerouting instead of output ?!

iptables -t mangle -A PREROUTING -p udp -m multiport ! --ports 53,80,443,60887 -m connbytes --connbytes 0:700 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6

Yes makes sense for a router

1 Like

Yes the reordering issue is a good reason to use the conntrack match, because it will match the whole flow.

1 Like

when i used some rules to prioritize small tcp flags, now browsing is much faster when download or uploading.
even with full saturated up/down link will not affect game or browsing!

But do I still need to use length with conntrack!?, something like this:
$IPT -t mangle -A PREROUTING -p udp -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m length --length 0:700 -j DSCP --set-dscp-class CS6

No do the connbytes match I suggested with connbytes average btw 0-700 you can add the port exclusion if you like, like you did in your modified command from Ultimate SQM settings: Layer_cake + DSCP marks

Briefly for the first few ms to a sec or so the connection might go back and forth until the avg size stabilizes, after that the whole flow will be prioritized so long as it stays below 700 bytes. You might drop that to more like 550 or so, it's still catch your example packet capture.

Like this ?
$IPT -t mangle -A PREROUTING -p udp -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m connbytes --connbytes 0:700 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6
port exclusion is important, you know that HTTP and HTTPS are bulky, also i'm already have a rules for them!

Yes that should work fine, the port exclusion will handle ignoring QUIC too so that's good.

1 Like

Another thing you could add would be a rateest match, not only match small packets but also no more than say 300kbps for the flow... But I'm not sure how to do that, rateest match.

1 Like

Man page:
http://ipset.netfilter.org/iptables-extensions.man.html#lbBU

I think the problem is in iptables the rate estimate is not on a per flow basis, but in nft it could be done because nft meters can be per flow

Avgpacket size is probably enough once you exclude port 53,80,443 etc

I see.
For now, Let me see how will conntrack work.

Hashlimit is what you want. After tagging based on connbytes, do another rule, hashlimit match based on IP and port both dirs for rates over say 300kbit/s change the tag back to cs0 or something else. You can match on dscp too so it only affects the high priority flows. I'm on my phone so can't really write the example for you

I think, i can make it in one rule?!
$IPT -t mangle -A PREROUTING -p udp -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m connbytes --connbytes 0:700 --connbytes-dir both --connbytes-mode avgpkt -m hashlimit --hashlimit 54/sec --hashlimit-mode srcip,srcport,dstip,dstport -j DSCP --set-dscp-class CS6
based on packet length=700

I think the problem with doing it all in one go is that the hashlimit will allow you to tag the first 54 packets in a given sec, then it will not allow further tagging... so imagine you have some kind of high bandwidth flow using small packets, it will tag 54 packets CS6 and then the rest of the packets in that second won't get tagged cs6.

I think what you want is use a hashlimit over rule to tag the entire connection "high bandwidth", and then in your connbytes rule match only connections not tagged high bandwidth.

For example, if I use 0x55 as a connmark meaning "bulk flow" (you can also do this with connlabel or other techniques)


$IPT -t mangle -A PREROUTING -p udp -m hashlimit --hashlimit-above 150/sec --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55

$IPT -t mangle -A PREROUTING -p udp -m connmark ! 0x55 -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m length --length 0:700 -j DSCP --set-dscp-class CS6

Then only flows sending fewer than 150 packets per second (total, both directions, so about 75 packets per second each direction) are eligible to be boosted based on their small packet size.

1 Like