Ultimate SQM settings: Layer_cake + DSCP marks

hisham2630 · May 17, 2019, 2:58pm

just keep it CS0, this will set incoming dscp marks from internet to CS0(we don't trust isp dscp marks)

hagenauwa · May 17, 2019, 3:03pm

Thank you!
Now i'm facing something weird..this is my current firewall.user and i only set up youtube in the dnsmask.conf..

IPT="iptables"

#Clear interface dscp marks, we don't trust ISP marks(also to use our own marks).
$IPT -t mangle -A PREROUTING -i pppoe-wan -j DSCP --set-dscp 0

#Ping
$IPT -t mangle -A PREROUTING -p icmp -j DSCP --set-dscp-class CS6 ##dscp tag for ping packets

#Gaming
$IPT -t mangle -A PREROUTING -p tcp -m multiport --port 9339 -j DSCP --set-dscp-class CS6                                                         #Brawl Stars
$IPT -t mangle -A PREROUTING -p udp -m multiport --port 9339 -j DSCP --set-dscp-class CS6                                                        #Brawl Stars

$IPT -t mangle -A PREROUTING -p udp -m multiport --ports 49003:49006 -j DSCP --set-dscp-class CS6                                         #GeForceNow
$IPT -t mangle -A PREROUTING -p tcp -m multiport --port 49006 -j DSCP --set-dscp-class CS6                                                       #GeForceNow

#Prova Youtube
$IPT -t mangle -A PREROUTING -m set --match-set vidstream src -j DSCP --set-dscp-class AF41
$IPT -t mangle -A PREROUTING -p tcp -m multiport --dport 1935 -j DSCP --set-dscp-class AF41

Why if i try to do a speedtest using fast.com, it tells me that it is AF41?
When i try to download an app wireshark correctly shows me CS0

hisham2630 · May 17, 2019, 3:09pm

make ping CS5, and replace all PREROUTING to POSTROUTING !

who is told you about AF41? (fast.com)???

just trust wireshark!, you can use tcpdump (run it on br-lan after that run it on pppoe0)on router then open the generated pcap with wire shark and check marks!

hagenauwa · May 17, 2019, 3:12pm

Sorry i mean in Wireshark when i do a download test from that site, i see the inbound packets marked as AF41, only for that site, dslreports is CS0 as it should be From what i set only youtube videos should be AF41 am i right?

Also why do i have to change to POSTROUTING?

hisham2630 · May 17, 2019, 3:18pm

first you have to flush the PREROUTING or POSTROUTING table before reapplying rules(so it won't duplicate).

Right, i think fast.com send packets marked as AF41 or it's your isp!

that's better cause it's have the packet processed and you will see right tags on inbound, so prerouting is not needed at all!

hagenauwa · May 17, 2019, 3:37pm

But it should be CS0 from this rule

$IPT -t mangle -A POSTROUTING -i pppoe-wan -j DSCP --set-dscp-class CS0

I've an headache

Btw again thank you so much for helping !

EDIT: The problem is in the youtube rule, if i change that to CS4 instead of AF41, fast.com traffic is marked CS4
$IPT -t mangle -A POSTROUTING -m set --match-set vidstream src -j DSCP --set-dscp-class CS4

Is it possible that fast.com uses the same domain googlevideo of youtube to deliver its speetest?

hisham2630 · May 17, 2019, 4:57pm

in my case, fast.com is using Netflix video domain?!, so don't worry about fast.com.

warchief · May 19, 2019, 9:29am

I did what dlakelan suggested and added 2 cake layers and installed the iptables script with the dscp markings on the ports. I just want to clean up firewall for gaming using iptables marking packets and controlling them.

example:

## game traffic
iptables -t mangle -A dscp_mark -p udp  --sport 1119:1120 -j DSCP --set-dscp-class CS5
ip6tables -t mangle -A dscp_mark -p udp --sport 1119:1120 -j DSCP --set-dscp-class CS5
iptables -t mangle -A dscp_mark -p udp --dport 1119:1120 -j DSCP --set-dscp-class CS5
ip6tables -t mangle -A dscp_mark -p udp --dport 1119:1120 -j DSCP --set-dscp-class CS5

I would like these rules to be MAC source marked for the egress and ip destination marked for the ingress.

and I would also like to prioritize a single machine over all other network traffic using iptables

example:

gaming machine class CS5
all other network traffic CS0

dlakelan · May 19, 2019, 4:58pm

I think you can just mark all packets from the given Mac or to the given IP. iptables has a -m mac match, and a dst match, see iptables manual

warchief · May 19, 2019, 7:20pm

## icmp for egress mac and ingress IP gets CS5
iptables -t mangle -A dscp_mark -p icmp -m mac --mac-source 00:00:00:00:00:00 -j DSCP --set-dscp-class CS5
iptables -t mangle -A dscp_mark -p icmpv6 -m mac --mac-source 00:00:00:00:00:00 -j DSCP --set-dscp-class CS5
iptables -t mangle -A dscp_mark -p icmp -d 192.0.0.0 -j DSCP --set-dscp-class CS5
iptables -t mangle -A dscp_mark -p icmpv6 -d 192.0.0.0 -j DSCP --set-dscp-class CS5


## game traffic egress mac and ingress IP
iptables -t mangle -A dscp_mark -p udp --sport 1119:1120 -m mac --mac-source 00:00:00:00:00:00 -j DSCP --set-dscp-class CS5
ip6tables -t mangle -A dscp_mark -p udp --sport 1119:1120 -m mac --mac-source 00:00:00:00:00:00 -j DSCP --set-dscp-class CS5
iptables -t mangle -A dscp_mark -p udp --dport 1119:1120 -d 192.0.0.0 -j DSCP --set-dscp-class CS5
ip6tables -t mangle -A dscp_mark -p udp --dport 1119:1120 -d 192.0.0.0 -j DSCP --set-dscp-class CS5

If this is correct how do I set all other traffic on the WAN and/or LAN to CS0 or does that even matter now that the packets are prioritized?

hisham2630 · May 19, 2019, 7:25pm

its already CS0, if not:

iptables -t mangle -A dscp_mark -i pppoe0 -j DSCP --set-dscp-class CS0

use the same rule but change pppoe0 to br-lan

warchief · May 19, 2019, 11:32pm

hisham2630 Thank you for the feedback

I'm sorry I asked the wrong question. Obviously the traffic is already tagged as normal traffic and shaped on the wan, lan through layer_cake and again through piece of cake on the lan on the ingress for wifi.

What I meant to ask was how would I assign 1 MAC address priority over all other WAN traffic through iptables? Maybe it does not matter as long as the UDP and ICMP is prioritized in iptables.

hisham2630 · May 21, 2019, 8:40pm

no problem, its all about what do you want to achieve, if you already prioritied ping and udp thats enough.

hagenauwa · May 22, 2019, 12:30pm

Why do i have "iptables v1.6.2: Couldn't load match `hashlimit':No such file or directory" while setting this rule in the script? Am i missing some packages?

$IPT -t mangle -A POSTROUTING -p udp -m hashlimit --hashlimit-name udp_high_prio --hashlimit-above 120/sec --hashlimit-burst 50 --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55 -m comment --comment "connmark for udp"

dlakelan · May 22, 2019, 1:56pm

not in front of a router at the moment but just search packages for kmod related to Hashlimit. probably kmod-ipt-hashlimit or something similar

warchief · May 26, 2019, 11:02am

dlakelan:

hisham2630:

instead of second name space is it possible to create a bridge between eth0.2 and veth0 without set any ip(keep them layer 2), then set veth1 as wan interface, then is it possible to tag packets ?

I set a test up on an old TP-link WDR-3600 I have lying around.
with ip-full and kmod-veth and all that installed, I used this script to create veth pair:
ip link add type veth
ip link set veth0 up
ip link set veth1 up
I then went to luci and created a new interface "prewan", I made it a bridge between eth0.2 and veth0, then I moved WAN to be physically connected to veth1.

I then put SQM instance on egress only on veth0 and veth1. The veth0 rate was "download" rate of actual wan, and the veth1 rate was "upload" rate of actual wan.

and IT WORKED just fine.

EDIT: obviously you want to fiddle around with the iptables rules for tagging with DSCP, and perhaps it requires you to do use bridge netfilter to ensure the iptables are called when the prewan bridge receives packets. But it's a very cool idea. @moeller0 might be interested.

SQM, Flow Offloading, VLAN Tagging and Gaming

lezz:

Could I just put another SQM on wifi?

Well you could but it won't necessarily work so well, the issue is that you need to keep the total download below your 22Mbps or whatever it is. To do that you can do the following:

create a veth0 - veth1 pair (requires you to install the veth packages on your router and maybe the full "ip" suite)

put veth1 into your br-lan bridge

alter your routing table so that all packets coming in on WAN get forwarded via veth0, which will send them down a virtual wire to veth1 and inject them into the bridge where they'll get distributed between wired and wireless

place your layer_cake SQM on the veth0 device uplink/egress

You can create the veth pair as follows in your firewall.user script (untested)
ip link add type veth
ip link set up veth0
ip link set up veth1
ip link set veth1 master br-lan
then you'll need to add a static route so that all packets coming in WAN destined for LAN are sent to veth0 rather than br-lan, and put an SQM on veth0

note that this is a little tricky, you could lose your connection to your router if you route to veth0 and it's not properly got veth1 in the bridge or something like that. Save your config backup before you try this stuff, and be prepared to revert via failsafe.

I will follow the second set of instructions setting up a veth pair. The first set I almost bricked my router lol.....

EDIT:
When layer_cake is enabled on pppoe-wan egress, eth0.1(lan) ingress my bufferbloat on both links stays under 10

When I:
add veth1 to br-lan
layer_cake egress veth0, layer_cake ingress veth1
apply these rules

iptables -A FORWARD -i eth1.2 -o veth0 -j ACCEPT
iptables -A FORWARD -s -i eth1.2 -d br-lan -o veth0 -j ACCEPT

my bufferbloat stays above 25 for download
my bufferbloat stays at 15 for upload

Have I done the setup correctly?

dlakelan · May 26, 2019, 10:47pm

those iptables rules bypass your firewall entirely, get rid of them. what you need is changes to the routing table, not iptables

warchief · May 26, 2019, 11:02pm

Ok to set up veth pair I do not use any iptables rules.

I will be putting the script to change the openwrt routingtable in network >firewall>custom scripts correct? Forgive me, but I do not know how to construct this script. Would you provide a step by step guide through this process for OpenWrt?

warchief · May 28, 2019, 9:07am

I managed to figure out how to follow these steps in OpenWrt.
I created veth-pair.
I created interface prewan and set protocol: unmanaged, cover the following interface: veth0
I bridged prewan(veth0) and Switch VLAN "eth0.1" (lan)
I set WAN(pppoe-wan) interface physical settings: veth1

With the first set of instructions this is not functional for me. My pppoe will not connect.

With the second set of instructions I am not successful setting this bit properly in OpenWrt Network>Firewall>Firewall - Zone Settings.

hisham2630:

WANIF="pppoe-wan" #wan interface

tc qdisc add dev wlan0 root mq #enable multi queue on wlan0 #setup multi queue for wifi device

## set up a pair of veth devices to handle inbound and outbound traffic
ip link show | grep veth0 || ip link add type veth

## get new veth interfaces up
ip link set veth0 up ip link set veth1 up

## trun on promisc mode,sometimes it's needed to make bridge work
ip link set veth1 promisc on

## add veth1 to bridge
brctl addif br-lan veth1

## just to make sure there's nothing inside those 2 tables
ip rule del priority 100 ip route flush table 100

##ipset for streming sites.they are bening filled by dnsmasq
ipset create vidstream hash:ip
ipset create usrcdn hash:ip
ipset create bulk hash:ip
ipset create latsens hash:ip

## flush mangle table
$IPT -t mangle -F PREROUTING

## add routing for veth0 this will handle all slow traffic
ip route add default dev veth0 table 100 ip rule add iif $WANIF table 100 priority 100

I also attempted to edit this script to work with my OpenWrt and I am not able successfully sqm.

dlakelan · May 28, 2019, 10:32am

the prewan idea requires namespaces and they are not included in openwrt kernels afaik. it works on Debian for example.