Ultimate SQM settings: Layer_cake + DSCP marks (New Script!)

moeller0 · January 30, 2020, 7:38am

DSCP-awareness is not really the issue IMHO, all you need to do is select a diffserv-mode for sqm (other than besteeffort) that evaluates DSCPs in the first place, and set

so that sqm will honor the selected diffserv mode for ingress. (If squash_ingress equals 1, sqm will simply make sure the ingress cake instance uses betseffort, that way the DSCPs are not touched, but will also not result in any priority tiering, squash_dscp however will re-map the DSCP of all packets to 0, AFTER cake has used the field to steer packets into the appropriate priority tin).

The real issue, IMHO is how to make sure that ingress DDCPs are set in accordance with your policy. And there the options are in decreasing levels of painful:

A) use sqm' default ingress redirection via an ifb and tc to manually filter packets and set the desired DSCPs, should work for ports but will have issues with getting internal IP addresses as tc is run before NAT and also is not conntrack aware, as far as I know.

B.1) use the veth method to move the ingress shaper onto the inside of your NAT domain or

B.2) instantiate the sqm instance for internet ingress on the router's LAN interface (which will cause issues, if the router also has wifi interfaces you want to use)

and then use standard iptables to re-mark/re-map incoming DSCP to follow your own policy

C) use standard sqm with ifb-ingress and use nftables' ingress-hook/keyword to directly apply the power of nftables n incoming packets before sqm's shaper actually sees them (note that nftables should have access to the conntrack tables and hence will be able to see NAT-internal IP addresses).

Kherby · January 30, 2020, 9:04pm

@moeller0
Thanks for this valueable informations. I haven't had the time to dive more into the veth method as there is currently lot of traffic in my network. So currently I'm back to my old SQM config (this time with diffserv4 instead of 3) + some firewall rules to priotize some gaming ports for my PS4 but as far as I understand it correctly those rules with only work for egress and not for ingress....?

config queue 'eth1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option itarget 'auto'
        option etarget 'auto'
        option interface 'pppoe-wan'
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'cake'
        option script 'layer_cake.qos'
        option qdisc_advanced '1'
        option squash_dscp '0'
        option squash_ingress '0'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'nat dual-dsthost ingress'
        option eqdisc_opts 'nat dual-srchost'
        option linklayer 'ethernet'
        option overhead '34'
        option linklayer_advanced '1'
        option tcMTU '2047'
        option tcTSIZE '128'
        option tcMPU '64'
        option linklayer_adaptation_mechanism 'default'
        option enabled '1'
        option upload '35375'
        option download '102150'

##GAMING DSCP SQM
iptables -t mangle -A FORWARD -p udp --match multiport --sport 3074,2304,3479:3480 -j DSCP --set-dscp-class CS6
iptables -t mangle -A FORWARD -p udp --match multiport --dport 3074,3479:3480 -j DSCP --set-dscp-class CS6
iptables -t mangle -A POSTROUTING -p udp --match multiport --sport 3074,2304,3479:3480 -j DSCP --set-dscp-class CS6
iptables -t mangle -A POSTROUTING -p udp --match multiport --dport 3074,3479:3480 -j DSCP --set-dscp-class CS6

What I would like to do is prioritize all the traffic (mostly gaming traffic) as a generall rule for my PS4 and put torrent traffic (if possible generall downloads as well) into the bulk tin.
Hisham's script and the veth method seems to be really good but I had some trouble settings everything up correctly and ended two times with a inaccessible router.

I'm also gonne look into nftables method, thanks.

@hisham2630
Sorry for OT but I'm trying to understand all the diffrent methods and SQM in general. Let me know If I should move my questions into a new thread.

moeller0 · January 30, 2020, 9:10pm

In essence yes, the default IFB that SQM uses forces you to either use tc to configure all filtering DSCP marking (and do so on the WAN side of the NAT) or hopefully in the future use nftables ingress hook (which does not work on current OpenWrt as the firewall depends on iptables and iptables and nftables are not fully compatible on the library level).

Kherby · January 31, 2020, 2:00am

What about the following approach? Ultimate SQM settings: Layer_cake + DSCP marks - #698 by Barrakketh
I don't see nftables or veth involved...

Yea, hopefully as this will make it way more easy for users like me right?

dlakelan · January 31, 2020, 2:16am

Hisham's script are based on my veth method. You don't need veth if you have two real ethernet interfaces... which occurs when your access point is a separate device from your router....

aha I should have clicked your link before commenting... that script sets up some marking chains, but they fundamentally run into the same problem. iptables runs after the IFB has queued the packet... so it works for egress on WAN but not for ingress on WAN.

moeller0 · January 31, 2020, 6:40am

That approach will allow you to copy egress DSCPs from to ingress packets of the same connection. This will solve quite a number of issues regarding marking of packets but by far not all.
The easiest method currently is to disable wifi on the primary router and instantiate the ingress shaper on the LAN facing interface and supplement with an dedicated AP for wifi. Have alook at @dlakelan's recent thread demonstrating that the raspberry pi4 seems to make an excellent relative low cost non-wifi primary router.

hisham2630 · January 31, 2020, 4:25pm

Thank you, but did you enabled forwarding in the firewall for all zones? and even the default settings?

i think you can use nft for tag packets.

No problem, but your case is helpful here, cause maybe there some people's have a same setup.

hisham2630 · January 31, 2020, 4:33pm

way's to fix your problem:

1.veth method, routeing between veth0 and wan interface then bridge veth1 to br-lan
in your case you will need 2 veth pairs because you have lan and guest and using your internal router wifi
You have to enable forwarding in firewall

2.veth method, but using only veth0 and veth1, and shoving veth0 in second name space and bridge it with wan interface, then run pppoe or dhcp on veth1 in main name space, but the problem, you have
to compile your openwrt image with kernel flag that will enable second name space.

3.like @dlakelan said,

Kherby · February 1, 2020, 9:03am

I'll try to the first veth method this weekend and report back. I'm not able to compile my own image for the WRT3200ACM and I'm very happy with the builds of davidc, so the second method is out of scope for me.
I still haven't looked closely into nftables but I'll try to read about it this weekend.

Are there any known (and noticeable) differences in performance between the veth and nftable method? I think that I do have enough horsepower with the WRT3200ACM for my current 100/37mbps connection but I'm asking because I plan to upgrade my connection to 250/40mbps in the near future...

dlakelan · February 1, 2020, 1:57pm

wrt3200 should handle 250/250 without a sweat either way

Gremlin · February 25, 2020, 2:58am

Hi,

I'm also using a WRT3200ACM however connected to a cable modem.. However not using a AP but do have a switch connected..also using onboard WiFi and Guest Network. Also have four interfaces for VPN, would method 1 also work for me?

Also does this play nicely with VPN policy-based routing as this also uses DSCP tagging?

Many thanks and also great work , I spent all day yesterday reading all the 700+ contributions to this topic on the old thread...

warchief · February 29, 2020, 8:12pm

That's the spirit!

SlotTech · February 29, 2020, 8:28pm

That's a lot of reading. I just implemented it minus the veth stuff and hoped for the best. Only thing I messed up was the CAKE config, didn't know I had to move my WAN download to LAN upload to get DSCP tags to work. To be fair I messed around with the original a few months ago but never got it to work.

warchief · February 29, 2020, 8:38pm

I did the same. Default config basically, install openwrt>install cake>configure cake. Simple, easy. But I also read 700 pages and tried everything, it is still a work in progress but I think I have narrowed it down to optimizing tcp connections. Starting from the stone age with iron, a hammer, and command prompt seems like the best way but luci and openwrt have accelerated the process - the developers and the community deserve a ton of credit!

Bear in mind great progress comes without great loss. I just fried a $200 finalmouse UL2 and could not recover it.

warchief · February 29, 2020, 9:35pm

@moeller0 If a device is put into a DMZ or is given port forwards does this affect the DSCP marking or affect cakes sqm - diffserv3, layercake, piece of cake?

Also, what is the best approach for putting a device outside common firewall rules for direct communication from traffic outside the wan aside from specialized firewall configs?

1. Traffic Forwarding

iptables -A FORWARD -s 192.1.2.3 -j ACCEPT
iptables -A FORWARD -d 192.1.2.3 -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.1.2.3 -j DSCP --set-dscp-class CS6
iptables -t mangle -A PREROUTING -d 192.1.2.3 -j DSCP --set-dscp-class CS6

2. Traffic Forwarding + Port Foward

iptables -A FORWARD -s 192.1.2.3 -j ACCEPT
iptables -A FORWARD -d 192.1.2.3 -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.1.2.3 -j DSCP --set-dscp-class CS6
iptables -t mangle -A PREROUTING -d 192.1.2.3 -j DSCP --set-dscp-class CS6

Network>Firewall>Port Fowards
Rule #1 TCP+UDP>Source Zone wan>external port 1-65535>Destination Zone lan>Internal IP 192.1.2.3>Internal Port any
Rule #2 ICMP>Source Zone wan>external port 1-65535>Destination Zone lan>Internal IP 192.1.2.3>Internal Port any

3. Traffic Fowarding + Port Forward + DMZ

iptables -A FORWARD -s 192.1.2.3 -j ACCEPT
iptables -A FORWARD -d 192.1.2.3 -j ACCEPT
iptables -t mangle -A PREROUTING -s 192.1.2.3 -j DSCP --set-dscp-class CS6
iptables -t mangle -A PREROUTING -d 192.1.2.3 -j DSCP --set-dscp-class CS6

Network>Firewall>Port Forwards
Rule #1 TCP+UDP>Source Zone wan>external port any>Destination Zone lan>Internal IP 192.1.2.3>Internal Port any
Rule #2 ICMP>Source Zone wan>external port any>Destination Zone lan>Internal IP 192.1.2.3>Internal Port any

Network>Firewall>Traffic Rules

Rule #1: Protocol any>source zone wan>source address any>source port any>destination zone lan>destionation address 192.1.2.3>destination port any

I feel like I get better latency performance with Traffic Forwarding+Port Forward+DMZ but that comes with more packet loss than Traffic Forwarding without Port Forwarding+DMZ. Are the rules iptables -A FORWARD -s 192.1.2.3 -j ACCEPT iptables -A FORWARD -d 192.1.2.3 -j ACCEPT meant to have the same effect as Port Forwarding and DMZ on a network?

SlotTech · March 11, 2020, 10:32am

@hisham2630 I've been using your script for a little while now and it's been working great! I'm having an issue on a remote network I manage where thing went south last night, download speeds were bad. I wasn't able to login to the router this morning and the only thing I noticed that was odd is the "tc-s qdisc" output and memory used is 4196480b of 4Mb, which is just over 4Mb After a reboot memory used returned to normal. The screen shot was taken when there was next to no traffic. This runs without veth as it's running on an old PC with only two nics one for WAN and one for LAN.

Any ideas would be greatly appreciated.

Attached is a screenshot:

hisham2630 · March 11, 2020, 3:33pm

i think it's a cake problem, maybe a bug.

SlotTech · March 11, 2020, 3:43pm

Thanks, I'll keep an eye on it and see what happens.

Edit: I decided to set the memlimit to 10Mb as my other network has a faster connection and it was 4Mb also, apparently it supposed to be based on speed cake is set to and rtt which doesn't seem to be the case.

hisham2630 · March 11, 2020, 8:07pm

try diffserv4 and see what will happen.

SlotTech · March 12, 2020, 9:54am

Thanks, I just switched to diffserv4 this morning as it ate through 10Mb in under 24 hours. It's a highly utilized network so I should find out soon if it's worked. My next step will be to remove the ingress keyword from the LAN upload (WAN download).