Ultimate SQM settings: Layer_cake + DSCP marks

I hope someone here could create a telegram group with anti spammer bot, so i can easily asset you.
lets just post problems and solutions here!
if any one do it, then let me know!

I'm sorry to reply out of your conversation but I just noticed something with the script..

I was checking if playing smash on switch online was marking CS6 correctly and noticed that all the packets were being marked as CS3 instead.

After looking at the qos.sh I think that these 2 rules are not working correctly @hisham2630

# unmarked UDP streams with small packets get CS6
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 22,25,53,67,68,123,143,161,162,514,5353,80,443,8080,60001 -m connbytes --connbytes 0:300 --connbytes-dir both --connbytes-mode bytes -j DSCP --set-dscp-class CS6 -m comment --comment "small udp connection gets CS6"

and

#Small packet is probably interactive or flow control
iptmark -m dscp ! --dscp  24 -m dscp ! --dscp  18 -m dscp ! --dscp  34 -m dscp ! --dscp  40 -m dscp ! --dscp  48 -m length --length 0:500 -j DSCP --set-dscp-class CS3 -m comment --comment "Small Flow Control"

If I disable the flow control rule, the packets are CS0, they are not marked by the first rule as CS6. I tried enabling it and changing it to CS6 and it works, the switch packets are CS6 while playing online.

maybe something with the length setting?

A word of caution, on default wifi WMM, CS6 and CS7 two map into AC_VO, which has the highest priority (as desired), but also does not allow aggregation and will noticeably reduce the achievable throughput over wifi if used at a significant rate, so make sure traffic in AC_VO stays sparse. (AC_VO will leave a bit of room for AC_VI, but will completely starve AC_BE).
But that is basically true for all priority systems...

Well the sqm team started the pathfinding first for me :slight_smile: My involvement with Linux has mostly been from being told "you can't do that" and me saying "hmm, why not?". I'm not sure if that was because of "no one has thought of it yet" or "no one has coded it yet" (or both). CAKE's de-natting host fairness is a good example "host fairness doesn't work with NAT because it doesn't know the internal addresses... why not? Surely this must be possible because Linux knows how to translate the packets so they end up in the right place, so if it is doing the NATting, it must know the original addresses. Mostly my solutions involve layer violations that seem to upset people but if there's a need to do something....

I'd like to start the nftables implementation, however nftables is completely unknown to me. What CONNMARK --savedscp-mark does is pretty simple:

Obtain packet DSCP field. Bit shift this so it maps into a 6 bit long mask specified as a parameter eg. 0xfc000000
Logically OR that with a 'DSCP is stored' parameter eg: 0x02000000
Store the result in conntrack's connmark field. That's it, job done.

I have absolutely no idea as to a possible nftables command syntax to aim for that. A fictitious 'set connmark from dscp shift left 'n' or 'm'

Perhaps a nftables guru can tell me which bits exist and which bits need to be written.

2 Likes

Well, this is where I typically stop :wink: So kudos to you to get $hit done!

One man's layer violation is another man's layering violation (aka the existing layers might simply be sup-optimal, like when btrfs started to hoist typical block layer features like raid modes into a file system).

Nah, my (too) clever plan its getting your iptables solution some more play with actual users in the field, and then we can approach the kernel team again, so that they recruit an nf guru to help out...
I have a hunch that nf was tasked with creating tools/methods to translate valid iptables commands into the nftables equivalents, so for a demanded feature we might be able to get the required "air support" to put this into the nftables developers plate. Obviously, I am overly generous to myself here when talking about we, you did all the heavy lifting :wink:

P.S.: Maybe we should move this topic into its own thread?

There are so many infos in this thread and I have to admit that I still don't understand most of it. So I haven't done anything yet as I'm still trying to understand most of the discussions in this thread.
Hopefully I will understand the most importand stuff at some point so I can test it on my end.
My goal for now would be to prioritize game traffic (port based) on my PS4 and keep everything else as simple as possible.

1 Like

Its typo, sorry for that, i mean c5 and c6

just tag everything CS0, and tag ps4 CS6 except port 80,443 leave them CS0

1 Like

Hi, for me SQM and QOS is very new, then network is not my best :frowning:

I tried to set up this but I got this error:

Thu Nov 28 19:12:46 2019 user.notice vpn-policy-routing [7488]: service started with gateways: wan/eth0.2/192.168.1.1 [0;32m[✓][0m PIA_VPN/tun1/10.25.10.6 VPN_SERVER/tun0/10.8.0.1
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.470100] IPv6: ADDRCONF(NETDEV_UP): veth0: link is not ready
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.474424] IPv6: ADDRCONF(NETDEV_UP): veth1: link is not ready
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.474874] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.481100] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.492996] device veth1 entered promiscuous mode
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.500319] br-lan: port 4(veth1) entered blocking state
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.500349] br-lan: port 4(veth1) entered disabled state
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.505337] br-lan: port 4(veth1) entered blocking state
Thu Nov 28 19:12:52 2019 kern.info kernel: [  104.510003] br-lan: port 4(veth1) entered forwarding state
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done: RTNETLINK answers: No such file or directory
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done: iptables v1.6.2: Couldn't load match `hashlimit':No such file or directory
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done:
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done: Try `iptables -h' or 'iptables --help' for more information.
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done: iptables v1.6.2: Couldn't load match `hashlimit':No such file or directory
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done:
Thu Nov 28 19:12:52 2019 daemon.notice procd: /etc/rc.d/S95done: Try `iptables -h' or 'iptables --help' for more information.
Thu Nov 28 19:12:57 2019 user.notice SQM: Stopping SQM on eth0
Thu Nov 28 19:12:57 2019 daemon.notice procd: /etc/rc.d/S95done: SQM: Stopping SQM on eth0
Thu Nov 28 19:12:58 2019 user.notice SQM: Starting SQM script: layer_cake.qos on eth0, in: 0 Kbps, out: 16000 Kbps
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S95done: SQM: Starting SQM script: layer_cake.qos on eth0, in: 0 Kbps, out: 16000 Kbps
Thu Nov 28 19:12:58 2019 user.notice SQM: layer_cake.qos was started on eth0 successfully
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S95done: SQM: layer_cake.qos was started on eth0 successfully
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S96led: setting up led USB 1
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S96led: setting up led USB 2
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S96led: setting up led WAN
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S96led: setting up led eSATA
Thu Nov 28 19:12:58 2019 daemon.notice procd: /etc/rc.d/S96led: Skipping trigger 'ide-disk' for led 'eSATA' due to missing kernel module
Thu Nov 28 19:12:58 2019 daemon.info procd: - init complete -

For the setup I followed the 1st post and I installed all packages, then sysctl.conf and /etc/config/sqm always from 1st post, after I created dscp.sh and modified dnsmasq.conf from here:

Then I set /usr/lib/sqm/defaults.sh to diffserv8, and the last passage is to put

sleep 5
./root/dscp.sh
sleep 5
/etc/init.d/sqm restart

Inside the startup form

Actually my line is one FFTH 1000/100, do you think I've to modify some values?

Sorry if I did some wrong but for me networking is very hard to understand

J

https://en.wikipedia.org/wiki/KISS_principle ?:smile:

1 Like

for a connection this fast you probably need an x86. you can do sqm on the upstream 100Mbps much easier than the 1000 down.

the scripts here are not suitable for someone without some moderate experience in networking. to get started I recommend standard SQM maybe the diffserv4 variant if you want to set up a gaming machine with high priority DSCP

2 Likes

You right! My device is one R7800, I tried a speed test with no other devices and a speed test with one large torrent on nas, results is speed about at 80Mb/s, so pretty good for me.
Like you already understood I'm not so able on networking, I'm not a noob on pc but networking is my bad!
I don't play any game or if I do, is only smartphone games :slight_smile:
My only thought is about the streaming like prime video or netflix or other when torrent client is under heavy load.
So, I'll reset all script to default settings and I try with default sqm?

J

given this plus your line is fast and your requirements modest
i'd say you'd fare well with limiting the torrent client itself to about 80% of max speed and call it a day.
video streaming should work fine.

you dont mention what device you are using but as mentioned, for this speed you basically need a x86 (real computer) to run sqm.

I suggest to enable SQM on upstream direction, and then set your torrent client to limit download direction to 700Mbps, it seems like most torrent clients have some such limit settings. you should be fine for video streaming. your bigger problem would be VOIP or games but you say you don't do that stuff so you definitely don't need more than standard upstream sqm

Ok, thanks, I'll revert all and I'll apply sqm only on upstream.
To reply to some answers:
@fuller -> my device is Netgear R7800 (Qualcomm Atheros IPQ8065)
@dlakelan -> actually I use VOIP because my ISP modem/router have it for the phone. On the main modem/router there's already a QOS section to set VOIP on max priority and wan on lowest

Do you think I need some special note or some extra tips to do?

J

The problem here just is that ipq806x is running into CPU contention, as you overtax it with a 1 GBit/s WAN even before SQM - and SQM reduces that even further. 'Fortunately' your upstream is significantly slower, making it perhaps a workable compromise if you restrict SQM to merely cover the upstream.

Why cover the upstream?
I don't know exactly why but if I have to guess, I can only think that the reason is like on torrent, to prevent to use all upload bandwidth and prevent some extra lag because the upstream is full
Any special settings for the sqm upstream configuratio?

Because for example your typical downstream TCP flows need to send ACK packets in the reverse direction (the receiver tells the sender what data was received), if the upstream is saturated or badly bloated, these ACK packets will be delayed, which in turn slows down your downloads.

Well, a number of users is quite happy with per-internal IP fairness, see https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details, Making cake sing and dance, on a tight rope without a safety net (aka advanced features) for details.

1 Like

Ok, thanks, I'll try

Good idea. Feel free! Incidentally I've a new attempt to get the iptables flavour upstream - it's technically more correct now. https://lore.kernel.org/netfilter-devel/20191203160652.44396-1-ldir@darbyshire-bryant.me.uk/T/#t