Ultimate SQM settings: Layer_cake + DSCP marks

dlakelan · December 26, 2018, 6:38pm

I think the problem is in iptables the rate estimate is not on a per flow basis, but in nft it could be done because nft meters can be per flow

Avgpacket size is probably enough once you exclude port 53,80,443 etc

hisham2630 · December 26, 2018, 6:39pm

I see.
For now, Let me see how will conntrack work.

dlakelan · December 26, 2018, 6:52pm

Hashlimit is what you want. After tagging based on connbytes, do another rule, hashlimit match based on IP and port both dirs for rates over say 300kbit/s change the tag back to cs0 or something else. You can match on dscp too so it only affects the high priority flows. I'm on my phone so can't really write the example for you

hisham2630 · December 26, 2018, 7:11pm

I think, i can make it in one rule?!
$IPT -t mangle -A PREROUTING -p udp -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m connbytes --connbytes 0:700 --connbytes-dir both --connbytes-mode avgpkt -m hashlimit --hashlimit 54/sec --hashlimit-mode srcip,srcport,dstip,dstport -j DSCP --set-dscp-class CS6
based on packet length=700

dlakelan · December 26, 2018, 7:23pm

I think the problem with doing it all in one go is that the hashlimit will allow you to tag the first 54 packets in a given sec, then it will not allow further tagging... so imagine you have some kind of high bandwidth flow using small packets, it will tag 54 packets CS6 and then the rest of the packets in that second won't get tagged cs6.

I think what you want is use a hashlimit over rule to tag the entire connection "high bandwidth", and then in your connbytes rule match only connections not tagged high bandwidth.

For example, if I use 0x55 as a connmark meaning "bulk flow" (you can also do this with connlabel or other techniques)


$IPT -t mangle -A PREROUTING -p udp -m hashlimit --hashlimit-above 150/sec --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55

$IPT -t mangle -A PREROUTING -p udp -m connmark ! 0x55 -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m length --length 0:700 -j DSCP --set-dscp-class CS6

Then only flows sending fewer than 150 packets per second (total, both directions, so about 75 packets per second each direction) are eligible to be boosted based on their small packet size.

hisham2630 · December 26, 2018, 7:27pm

So how do you know that 0x55 is bulk based on what?
Or you jut assume it's a bulk based on hash limit!

dlakelan · December 26, 2018, 7:31pm

Yes if it sends more than 75 packets each way each second then I tag it 0x55 and it can't be eligible to upgrade to cs6

EDIT: now that I think about it the hashlimit will split each direction so the number should probably be something like 75 not 150, because each direction will get its own hashlimit.

The thing about this technique is it's a latch... as soon as hashlimit hits, the whole connection is marked, and it becomes ineligible for the rest of the duration of the connection, so there's no oscillating back and forth between a few CS6 and the rest unmarked etc.

hisham2630 · December 26, 2018, 7:58pm

I will check and see, also i changed it to 75, many thanks.
I think you are the only active member here, others are lazy!
EDIT:
iptables error:
Bad argument0x55'``
EDIT:
First you should set hashlimit name:
$IPT -t mangle -A PREROUTING -p udp -m hashlimit --hashlimit-name udp_high_prio --hashlimit-above 75/sec --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55
second:
you should add --mark:
$IPT -t mangle -A PREROUTING -p udp -m connmark ! --mark 0x55 -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,80,443,60887 -m length --length 0:700 -j DSCP --set-dscp-class CS6
now there's no error!
EDIT:
New capture: https://a.uguu.se/GpDPSLsrH0Bw_mi8-new1.pcap for the same game.
Good night!

dlakelan · December 26, 2018, 8:34pm

I'm glad you are making progress, and I may use some of these ideas myself, but let's just give thanks not say uncharitable things about others. I will look at your capture, I hope it is working well.

I see that all the packets from the game server are tagged, I assume you're capturing on LAN so it hasn't tagged your LAN packets yet. It looks like your rules are working.

hisham2630 · December 27, 2018, 1:14pm

Thanks, let me test for a few days then let you know how it's work.
sure they deserve more than thank you, without them we couldn't see OPENWRT present until now!

Yeah i'm capturing on br-lan, i will check the tags on pppoe-wan as well.
i will reorder the rules to make script smaller and more readable, then i will share it here so others
can benefit from it.
Also i can say this QOS is working even with wrong bandwidth set for download and upload.

dlakelan · December 27, 2018, 4:09pm

Well if it's too far off things shouldn't work well but if it's close, the prioritization of the important packets can help a lot, and if the ISP actually supports the tags then it can be a huge benefit, since you have a wireless ISP it might be that their equipment uses something like wmm and so it works on their equipment too.

hisham2630 · December 27, 2018, 4:20pm

I'm on Fiber ISP for 3 months, they use EPON to access the service!

dlakelan · December 27, 2018, 4:27pm

Oh right you said that. But in any case they may honor the tags, and especially they may do so over some bottleneck link.

I'm particularly interested in how your UDP rule works for things like your VoIP apps and video chat etc. A rule that does a good job of catching real-time communications based on basic bandwidth and packet size is great if it works.

hisham2630 · December 27, 2018, 4:37pm

I think so.

This is will be really great and shared for first time on the net!, i did a lot of searching and can't find something
similar to our idea, let test for a few days!, you deserve a more than thank you!!!!
Also i will optimize the script to include more services and i will abandon some rules; like those have dst and
src , i can use:
--ports instead of --sports and --dports
I was reading about MIKROTIK QOS in forms and blogs, you can find a useful rules!
but their queue is not easy to understand and use
EDIT:
Is it better to add udp port exception for the hashlimit rule?
SOS:
it's work for outbound: https://a.uguu.se/DUjZzFsaXDZt_mi8-new-o.pcap game ip start with 49.xx.xx.xx

dlakelan · December 27, 2018, 9:22pm

I don't think it makes a difference, but you could check the connmark as the first thing in that rule, there's no need to remark once it's marked.

hisham2630 · December 28, 2018, 3:25pm

I think prioritize based on port is not needed any more!!!
let me test for a few days then let you know!
I want an advice, let's assume the following:

I have a voip phone connected to lan.
I have a game and voip calls via skype or whatsapp.
in this case is it better to tag the voip phone with CS6 and nothing else.
And tag udp packets (using our method) for games and voip like skype and whatsapp with CS5 ?

dlakelan · December 28, 2018, 3:43pm

I do think tagging the VoIP phone higher priority than the UDP size based one makes sense, then false positives don't affect your known VoIP stuff.

Only issue is cake doesn't make a distinction when using layer cake Ultimate SQM settings: Layer_cake + DSCP marks

I think cake has an even finer grained tier system but I'm not sure.

You might want to try my hfsc based system, it has one real-time VoIP queue, one real-time game etc queue and 3 linkshare queues for high normal and background.

hisham2630 · December 28, 2018, 3:53pm

It would be nice if we can change cake/layer_cake parameters.

Do you have a link for it ?
I think i need to disable sqm to use your HFSC based system!

dlakelan · December 28, 2018, 3:57pm

I can put it on my blog, I had an older version there, it used a lot of TC commands, but there are iptables commands or nftables that work fine instead, and easier to understand. I'm not home though so it will be Sunday before I can do that.

The iptables CLASSIFY match eliminates the need for lots of TC commands.

http://models.street-artists.org/2018/01/16/understanding-hfsc-in-linux-qos/

That shows how to set up the hfsc classes. Then you need classify rules in iptables to put in class 1:10 Voip
1:20 Games etc
1:30 Streaming video or other high priority non realtime
1:40 normal
1:50 bulk

hisham2630 · December 28, 2018, 4:13pm

I'm not in rush, take your time.
BTW: i don't like those TC commands, but no problem.
I think there's some limitations to use NFT on OPENWRT, just not sure!

I agree.
I want to have a look at cake diffserv8 i think it's better in terms of priorities!

static int cake_config_diffserv4(struct Qdisc *sch)
{
/*  Further pruned list of traffic classes for four-class system:
 *
 *          Latency Sensitive  (CS7, CS6, EF, VA, CS5, CS4)
 *          Streaming Media    (AF4x, AF3x, CS3, AF2x, TOS4, CS2, TOS1)
 *          Best Effort        (CS0, AF1x, TOS2, and those not specified)
 *          Background Traffic (CS1)
 *
 *              Total 4 traffic classes.

if you look at those classes:

Latency sensitive (CS7,CS6) so it mean's there's no difference in priority between CS6 and CS7, so that's
whats you want to tell me?
and the rest of classes do the same?!

Looking at DIFFSERV8:

+static int cake_config_diffserv8(struct Qdisc *sch)
+{
+/*	Pruned list of traffic classes for typical applications:
+ *
+ *		Network Control          (CS6, CS7)
+ *		Minimum Latency          (EF, VA, CS5, CS4)
+ *		Interactive Shell        (CS2, TOS1)
+ *		Low Latency Transactions (AF2x, TOS4)
+ *		Video Streaming          (AF4x, AF3x, CS3)
+ *		Bog Standard             (CS0 etc.)
+ *		High Throughput          (AF1x, TOS2)
+ *		Background Traffic       (CS1)
+ *
+ *		Total 8 traffic classes.
+ */

So i think it's make a sense, but i don't know much bandwidth is divided between those classes!