Ok thanks. Can I use the iptables I posted above? Just want these udp packets to travel fast as possible and ICMP priotized.
Well, just try it.
tc -s qdisc will give you statistics where cake reports how many packets traversed through the different priority tiers, so just run a test and see whether the correct counters increase.
Typically on a wan link's ingress side qdiscs run before iptables got hold of the packets, so on ingress the marking most likely is not going to have an effect on egress it might (unless the NAT system had to re-map the ports to different internal port numbers)...
It would be awesome if SQM (luci-app) had a option to prioritize packets of specific gaming ports when using layer_cake. I would love to prioritize a few gaming ports but reading thrue this thread I figured out that it does seem to be kinda complicated and not very userfriendly.
@lajkasir
If this worked for you please report back as I would like to set up something similar for some games.
2 Likes
Im using gargoyle rn but dont know if its possible to run this on openwrt thats why im asking...
On gargoyle i post these in firewall.user and run this command in putty
/etc/init.d/firewall restart
Then reboot
Edit: ok i just tried but get error. Do i need to have SQM and Layer cake enabled to get this working?
I've tested your rules with Modern Warfare on my PS4 and it does seem to work for the egress side (layer_cake@pppoe). On the ingress side it didn't had any effect and I guess the reason is the one that moeller0 stated above...
Yes but my test showed me that your solution will only work on the upload side (egress) and sadly I don't know any simple solution for the download side (ingress). To make things rly work it has to be Up+Dowload sided.
If you're using a snapshot of master you can use sqm-scripts 1.4.0 (or newer) with cake and also install kmod-sched-ctinfo, kmod-sched-connmark, and kmod-sched-ipset (the latter is optional but I make use of it). I use a simple script based off of what @ldir originally posted:
#!/bin/sh
# Cero3 Shaper
# A cake shaper and AQM solution that allows several diffserv marking schemes
# for ethernet gateways
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
#
# Copyright (C) 2012-5 Michael D. Taht, Toke Høiland-Jørgensen, Sebastian Moeller
#sm: TODO pass in the cake diffserv keyword
. ${SQM_LIB_DIR}/defaults.sh
QDISC=cake
# Default traffic classication is passed in INGRESS_CAKE_OPTS and EGRESS_CAKE_OPTS, defined in defaults.sh now
egress() {
SILENT=1 $TC qdisc del dev $IFACE root
$TC qdisc add dev $IFACE root handle cacf: $( get_stab_string ) cake \
bandwidth ${UPLINK}kbit $( get_cake_lla_string ) ${EGRESS_CAKE_OPTS} ${EQDISC_OPTS}
# put an action on the egress interface to set DSCP from the stored connmark.
# this seems counter intuitive but it ensures once the mark is set that all
# subsequent egress packets have the same stored DSCP avoiding iptables rules
# to mark every packet, ctinfo does it for us and then CAKE is happy using the
# DSCP
$TC filter add dev $IFACE protocol all prio 10 u32 match u32 0 0 action \
ctinfo dscp 0xfc000000 0x01000000
}
ingress() {
SILENT=1 $TC qdisc del dev $IFACE handle ffff: ingress
$TC qdisc add dev $IFACE handle ffff: ingress
SILENT=1 $TC qdisc del dev $DEV root
[ "$IGNORE_DSCP_INGRESS" -eq "1" ] && INGRESS_CAKE_OPTS="$INGRESS_CAKE_OPTS besteffort"
[ "$ZERO_DSCP_INGRESS" -eq "1" ] && INGRESS_CAKE_OPTS="$INGRESS_CAKE_OPTS wash"
$TC qdisc add dev $DEV root handle cace: $( get_stab_string ) cake \
bandwidth ${DOWNLINK}kbit $( get_cake_lla_string ) ${INGRESS_CAKE_OPTS} ${IQDISC_OPTS}
$IP link set dev $DEV up
# redirect all IP packets arriving in $IFACE to ifb0
# set DSCP from conntrack mark
$TC filter add dev $IFACE parent ffff: protocol all prio 10 u32 \
match u32 0 0 action \
ctinfo dscp 0xfc000000 0x01000000 \
mirred egress redirect dev $DEV
# Configure iptables chain to mark packets
ipt -t mangle -N QOS_MARK_${IFACE}
# Change DSCP of relevant hosts/packets
# and save the DSCP to the connmark using savedscp
#From sched_cake.c:
# /* Further pruned list of traffic classes for four-class system:
# *
# * Latency Sensitive (CS7, CS6, EF, VA, CS5, CS4)
# * Streaming Media (AF4x, AF3x, CS3, AF2x, TOS4, CS2, TOS1)
# * Best Effort (CS0, AF1x, TOS2, and those not specified)
# * Background Traffic (CS1)
# *
# * Total 4 traffic classes.
# */
#and for diffserv8:
# /* Pruned list of traffic classes for typical applications:
# *
# * Network Control (CS6, CS7)
# * Minimum Latency (EF, VA, CS5, CS4)
# * Interactive Shell (CS2, TOS1)
# * Low Latency Transactions (AF2x, TOS4)
# * Video Streaming (AF4x, AF3x, CS3)
# * Bog Standard (CS0 etc.)
# * High Throughput (AF1x, TOS2)
# * Background Traffic (CS1)
# *
# * Total 8 traffic classes.
# */
GAMING_CLASS=AF41
ipt -t mangle -A QOS_MARK_${IFACE} -m set --match-set gameset dst -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -m set --match-set bulkset dst -j DSCP --set-dscp-class CS1
ipt -t mangle -A QOS_MARK_${IFACE} -p tcp -s 192.168.1.3 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p udp -s 192.168.1.3 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p tcp -s 192.168.1.4 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p udp -s 192.168.1.4 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p tcp -s 192.168.1.166 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p udp -s 192.168.1.166 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p tcp -s 192.168.1.152 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -t mangle -A QOS_MARK_${IFACE} -p udp -s 192.168.1.152 -m multiport --dports 1024:65535 -j DSCP --set-dscp-class ${GAMING_CLASS}
ipt -A QOS_MARK_${IFACE} -t mangle -j CONNMARK --savedscp-mark 0xfc000000/0x01000000
# Send unmarked connections to the marking chain
# top 6 bits are DSCP, LSB is DSCP is valid flag
# ipt -t mangle -A PREROUTING -i $IFACE -m connmark --mark 0x00000000/0x01000000 -g QOS_MARK_${IFACE}
ipt -t mangle -A POSTROUTING -o $IFACE -m connmark --mark 0x00000000/0x01000000 -g QOS_MARK_${IFACE}
#you could just send every packet to the marking chain and update the stored DSCP for every packet
#which should work for dynamic type marking but at a cpu cost
}
sqm_prepare_script() {
do_modules
verify_qdisc $QDISC "cake" || return 1
}
I have limited upload bandwidth so I use cakes video tin rather than voice. The '--savedscp-mark' saves the DSCP marks we set to the connection and the 'ctinfo' action restores the DSCP value to the packets before cake takes them.
The patches for savedscp support weren't merged into 19.07, but I've been building my own images with it since the beginning of August and everything has been working perfectly for me. If you want a fancier script ldir has a good example here.
I use dnsmasq to populate two ipsets, but you need to use dnsmasq-full for that. One is for a couple of games that use a HTTPS connection for game traffic, the other is mainly to confine Windows/Xbox updates to the bulk tin.
6 Likes
Thanks for your input!
I'm using a Snapshot build of davidc502 (OpenWrt SNAPSHOT r11086-4fc0a61ed3) and it has sqm-scripts 1.4.0-1 installed and I'm also able to install the packages which you mentioned above.
I'll give your solution/script a try later on. My Upload speed is rather on the larger side (~40mbps) so I guess that the default voice tin for gaming would make sense for me. I'll just need to figure out how to adjust the script for my needs because all I want for now is to priotize my PS4 Gaming traffic and I think some source/destination port rules for the gaming/server ports would be a good start.
All of the clients I specified are game consoles so I let them have the full range of ports I expect them to possibly use for game traffic (not worried about bittorrent clients on the consoles). Since the marks are being set on egress and you have no way to predetermine what hosts they will connect to except in limited circumstances (DNS requests for hosted servers, which are handled by ipsets) I considered that the easiest way to handle them.
Did you feel any diffrence
Edit: I will try to flash my wrt1900acsv2 with davidc502 firmware and try this:
iptables -t mangle -A POSTROUTING -j DSCP --set-dscp-class CS0
iptables -t mangle -A POSTROUTING -p icmp -j DSCP --set-dscp-class CS4
iptables -t mangle -A POSTROUTING -p udp --sport 3074 -j DSCP --set-dscp-class CS4
iptables -t mangle -A POSTROUTING -p udp --dport 3074 -j DSCP --set-dscp-class CS4
No I couldn't tell a difference in case of the hit detection but as I said these rules only work for egress and I can only notice a noticeable (negative) impact on gaming when there is heavy torrenting or steam updates are going on. So a working priotizizing for ingress is needed to make things better for gaming.
Hopefully I'll find time at the weekend to try out the script which Barrakketh provided and see if that fixes my problems.
I will try to respond to all of you tomorrow!
Needs veth. Pages 1-14 was a huge discussion with dlakelan and hisham2630 directing newcomers with the veth settings and firewall interaction within openwrt. This long thread has so many resources if yall just dig...
Looking at devmail at cake, there has been some attempts at trying to get cake to carry connmarks but doubt that will go mainline.
After ~9 months of soaking this SQM settings, I just like to say it works really well!
- way better than the mark everything best effort which really defeats the purpose of WMM
- it really made me appreciate WMM, DSCP nuts and bolts
- there is a certain snappiness, when done right, that you can feel vs "regular" wifi.
Thanks again @dlakelan for the original idea from your blog + contributions and @hisham2630 for creating this thread repository + contributions. Some user stated this thread should be marked "solved" -- that had to be a joke right??
3 Likes
@hisham2630
You turn on ack filtering in one of your later posts -- I have unacceptable tradeoff experience with it compared to ack filtering off.
Unacceptable in this case as defined: 95% of the time it works wonderfully but sometimes I have website timeouts whereas ack filter off--100% website is reached, no timeouts.
Comparing notes, do you have 100% positive experience?
Thanks for your feedback @QOS, hisham and I spent a lot of time trying out many different things. He has a fairly unique and unusual network connection, with different channels to caches that offer him higher speed vs the general internet. In any case. I agree with you that when you mark with DSCP and you get end-to-end QOS including WMM you do feel the snap of it all for realtime communications, such as VOIP or games or whatever.
kmod-sched-ctinfo
What does it do? And when should I use it?
ctinfo action in tc(8) Linux ctinfo action in tc(8)
NAME
ctinfo - tc connmark processing action
SYNOPSIS
tc ... action ctinfo [ dscp MASK [STATEMASK] ] [ cpmark [MASK] ] [ zone ZONE ] [ CONTROL ] [ index ]
DESCRIPTION
CTINFO (Conntrack Information) is a tc action for retrieving data from conntrack marks into various fields. At
present it has two independent processing modes which may be viewed as sub-functions.
DSCP mode copies a DSCP stored in conntrack's connmark into the IPv4/v6 diffserv field. The copying may condi-
tionally occur based on a flag also stored in the connmark. DSCP mode was designed to assist in restoring
packet classifications on ingress, classifications which may then be used by qdiscs such as CAKE. It may be
used in any circumstance where ingress classification needs to be maintained across links that otherwise bleach
or remap according to their own policies.
CPMARK (copymark) mode copies the conntrack connmark into the packet's mark field. Without additional parame-
ters it is functionally completely equivalent to the existing connmark action. An optional mask may be speci-
fied to mask which bits of the connmark are restored. This may be useful when DSCP and CPMARK modes are com-
bined.
Simple statistics (tc -s) on DSCP restores and CPMARK copies are maintained where values for set indicate a
count of packets altered for that mode. DSCP includes an error count where the destination packet's diffserv
field was unwriteable.
PARAMETERS
DSCP mode parameters:
mask A mask of 6 contiguous bits indicating where the DSCP value is located in the 32 bit conntrack mark
field. A mask must be provided for this mode. mask is a 32 bit unsigned value.
statemask
A mask of at least 1 bit indicating where a conditional restore flag is located in the 32 bit conntrack
mark field. The statemask bit/s must NOT overlap the mask bits. The DSCP will be restored if the con-
ntrack mark logically ANDed with the statemask yields a non-zero result. statemask is an optional
unsigned 32 bit value.
CPMARK mode parameters:
mask Store the logically ANDed result of conntrack mark and mask into the packet's mark field. Default is
0xffffffff i.e. the whole mark field. mask is an optional unsigned 32 bit value
Overall action parameters:
zone Specify the conntrack zone when doing conntrack lookups for packets. zone is a 16bit unsigned decimal
value. Default is 0.
CONTROL
The following keywords allow to control how the tree of qdisc, classes, filters and actions is further
traversed after this action.
reclassify
Restart with the first filter in the current list.
pipe Continue with the next action attached to the same filter.
drop Drop the packet.
shot synonym for drop
continue
Continue classification with the next filter in line.
pass Finish classification process and return to calling qdisc for further packet processing. This is
the default.
index Specify an index for this action in order to being able to identify it in later commands. index is a
32bit unsigned decimal value.
EXAMPLES
Example showing conditional restoration of DSCP on ingress via an IFB
#Set up the IFB interface
tc qdisc add dev ifb4eth0 handle ffff: ingress
#Put CAKE qdisc on it
tc qdisc add dev ifb4eth0 root cake bandwidth 40mbit
#Set interface UP
ip link set dev ifb4eth0 up
#Add 2 actions, ctinfo to restore dscp & mirred to redirect the packets to IFB
tc filter add dev eth0 parent ffff: protocol all prio 10 u32 \
match u32 0 0 flowid 1:1 action \
ctinfo dscp 0xfc000000 0x01000000 \
mirred egress redirect dev ifb4eth0
tc -s qdisc show dev eth0 ingress
filter parent ffff: protocol all pref 10 u32 chain 0
filter parent ffff: protocol all pref 10 u32 chain 0 fh 800: ht divisor 1
filter parent ffff: protocol all pref 10 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:1
not_in_hw
match 00000000/00000000 at 0
action order 1: ctinfo zone 0 pipe
index 2 ref 1 bind 1 dscp 0xfc000000 0x01000000 installed 72 sec used 0 sec DSCP set 1333 error 0
CPMARK set 0
Action statistics:
Sent 658484 bytes 1833 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
action order 2: mirred (Egress Redirect to device ifb4eth0) stolen
index 1 ref 1 bind 1 installed 72 sec used 0 sec
Action statistics:
Sent 658484 bytes 1833 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Example showing conditional restoration of DSCP on egress
This may appear nonsensical since iptables marking of egress packets is easy to achieve, however the iptables
flow classification rules may be extensive and so some sort of set once and forget may be useful especially on
cpu constrained devices.
# Send unmarked connections to a marking chain which needs to store a DSCP and set statemask bit in the
connmark
iptables -t mangle -A POSTROUTING -o eth0 -m connmark \
--mark 0x00000000/0x01000000 -g CLASS_MARKING_CHAIN
# Apply marked DSCP to the packets
tc filter add dev eth0 protocol all prio 10 u32 \
match u32 0 0 flowid 1:1 action \
ctinfo dscp 0xfc000000 0x01000000
tc -s filter show dev eth0
filter parent 800e: protocol all pref 10 u32 chain 0
filter parent 800e: protocol all pref 10 u32 chain 0 fh 800: ht divisor 1
filter parent 800e: protocol all pref 10 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:1
not_in_hw
match 00000000/00000000 at 0
action order 1: ctinfo zone 0 pipe
index 1 ref 1 bind 1 dscp 0xfc000000 0x01000000 installed 7414 sec used 0 sec DSCP set 53404 error 0
CPMARK set 0
Action statistics:
Sent 32890260 bytes 120441 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
SEE ALSO
tc(8), tc-cake(8) tc-connmark(8) tc-mirred(8)
AUTHORS
ctinfo was written by Kevin Darbyshire-Bryant.
4 Likes
Part of this was answered while typing, but:
I'll quote the author:
ctinfo is a new tc filter action module. It is designed to restore DSCPs
stored in conntrack marks into the ipv4/v6 diffserv field.The feature is intended for use and has been found useful for restoring
ingress classifications based on egress classifications across links
that bleach or otherwise change DSCP, typically home ISP Internet links.
Restoring DSCP on ingress on the WAN link allows qdiscs such as CAKE to
shape inbound packets according to policies that are easier to indicate
on egress.Ingress classification is traditionally a challenging task since
iptables rules haven't yet run and tc filter/eBPF programs are pre-NAT
lookups, hence are unable to see internal IPv4 addresses as used on the
typical home masquerading gateway.
Using sqm-scripts, it is added to a QOS script like the one I posted above. You'll notice it at the end of the egress function. And for ingress, the filter used that steals packets for cake in the original script is:
$TC filter add dev $IFACE parent ffff: protocol all prio 10 u32 \
match u32 0 0 flowid 1:1 action mirred egress redirect dev $DEV
We change that to:
$TC filter add dev $IFACE parent ffff: protocol all prio 10 u32 \
match u32 0 0 action \
ctinfo dscp 0xfc000000 0x01000000 \
mirred egress redirect dev $DEV
You'll notice ctinfo is added before mirred, which sets the DSCP flag back to the packets before cake gets them. Normally netfilter would never see those packets which is why we couldn't use iptables rules to mark them on ingress.
1 Like
With kmod-sched-ctinfo you don't need kmod-sched-connmark. ctinfo can do the same thing as sched-connmark, actually it's slightly more powerful in that you can mask which bits of the connmark to restore.
The elephant in the room is that kmod-sched-ctinfo does the copying of stuff stored in the conntrack connmark field to the packet's DSCP/connmark fields and is in the kernel, but the thing that puts the DSCP into the conntrack connmark field is at present an openwrt special and iptables only.
upstream kernel want an nftables implementation of CONNMARK --savedscp-mark as found in ,"iptables -A QOS_MARK_eth0 -t mangle -j CONNMARK --savedscp-mark 0xfc000000/0x01000000"
1 Like
This reminds me, I still want to include a version of @ldir's helpful qos script into sqm-scripts, but I never seem to be getting around to actually testing it first (and getting the required kmod installed in the first place)
Well, a quick check showed kmod-sched-ctinfo to be installed already, so all I need is time to test...
Does that require a specific package ot be installed or does this come with OpenWrt's default iptables packages?