Using DSCP for QoS

That's what I do, and is the purpose of the veth. The veth just provides a place to inject routed traffic into the lan bridge through a qdisc.

It gets complicated if you have vlans. It's possible to put your real wan iface and a veth both into a separate namespace, and treat the other end of the veth as your main router namespaces wan...

The latest cake now supports tc filter, so DSCP mark is no longer needed.

Nice.
But only in the upstream-4.18 branch or?

I just did a test, and currently cake in openwrt still cannot use tc filter. I think your solution is the best.

DSCP marking is still useful as it will affect the use of different priority queues in managed switches and WMM in the APs particularly for larger networks with multiple stand alone Enterprise APs and VoIP installs you would want to get the DSCP marks right.

After looking into WMM (ty for the hint) using CS3 for video seems bad.
AF4x seems better.
Don't use cake wash on ingress so wmm can make use of the dscp flags.
For egress i don't know if the common smart phone OSs use dscp.
Then the default override to CS0 needs to be changed?

So here is an updated version:

  • Switched to port and hostname based matching (to get rid of ndpi)
  • Changed the connmark/mark to use the actual decimal dscp value
    (maybe use hex values instead ?)

/etc/firewall.user

###################################################################
###  Create Custom Mangle Chains
###################################################################

### Flush Mangle Table
$IPT -t mangle -F

### Prerouting
$IPT -t mangle -N zone_lan_prerouting

$IPT -t mangle -A PREROUTING -i br-lan -j zone_lan_prerouting

### Forward
$IPT -t mangle -N zone_lan_forward

$IPT -t mangle -A FORWARD -i br-lan -j zone_lan_forward

###################################################################
### Make use of Cake's DiffServ4 Implementation
### by using dscp flags
###################################################################

### Setup some custom chains
$IPT -t mangle -N dscp_classify
$IPT -t mangle -N dscp_mark_cs6
$IPT -t mangle -N dscp_mark_af41
$IPT -t mangle -N dscp_mark_cs1
$IPT -t mangle -N dscp_set

### Since we dont know if lan users are messing around with dscp flags
### default all packets to cs0 (BE) and override later on
$IPT -t mangle -A zone_lan_prerouting -j DSCP --set-dscp-class CS0 -m comment --comment "Set DSCP CS0/BE as default"

### Setup dscp chain jumps
$IPT -t mangle -A zone_lan_forward -o eth1 -j dscp_classify
$IPT -t mangle -A zone_lan_forward -o eth1 -j dscp_set

### match packet mark and set appropriate dscp flag
$IPT -t mangle -A dscp_set -m connmark --mark 48 -j DSCP --set-dscp-class CS6 -m comment --comment "CS6 Voice Tin"
$IPT -t mangle -A dscp_set -m connmark --mark 48 -j RETURN
$IPT -t mangle -A dscp_set -m connmark --mark 34 -j DSCP --set-dscp-class AF41 -m comment --comment "AF41 Video Tin"
$IPT -t mangle -A dscp_set -m connmark --mark 34 -j RETURN
$IPT -t mangle -A dscp_set -m connmark --mark 8 -j DSCP --set-dscp-class CS1 -m comment --comment "CS1 Bulk Tin"
$IPT -t mangle -A dscp_set -m connmark --mark 8 -j RETURN

### set connection mark
### return back to main table/chain
$IPT -t mangle -A dscp_mark_cs6 -j CONNMARK --set-mark 48
$IPT -t mangle -A dscp_mark_cs6 -j RETURN
$IPT -t mangle -A dscp_mark_af41 -j CONNMARK --set-mark 34
$IPT -t mangle -A dscp_mark_af41 -j RETURN
$IPT -t mangle -A dscp_mark_cs1 -j CONNMARK --set-mark 8
$IPT -t mangle -A dscp_mark_cs1 -j RETURN

### check if packet has already been marked
### if true return to back to main table/chain
### if false set connmark based on iptables rules below
$IPT -t mangle -A dscp_classify -m connmark ! --mark 0 -j RETURN

###################################################################
# Latency Sensitive (Voice Tin)
###################################################################

### IPset (Hostnames)
$IPT -t mangle -A dscp_classify -m set --match-set dscp_cs6 dst -g dscp_mark_ef -m comment --comment "IPSet CS6 Hostname Match"

### Generic
$IPT -t mangle -A dscp_classify -p icmp -m limit --limit 1/sec --limit-burst 30 -g dscp_mark_cs6 -m comment --comment "ICMP"
### Gaming
# Steam Generic
$IPT -t mangle -A dscp_classify -p udp --sport 27000:27015 -g dscp_mark_cs6 -m comment --comment "Steam Generic"
# Steam CS:GO
$IPT -t mangle -A dscp_classify -p udp --dport 27020:27100 -g dscp_mark_cs6 -m comment --comment "CS:GO"
# PS4
$IPT -t mangle -A dscp_classify -p udp -s 10.0.1.60 -m multiport ! --dports 9000 -g dscp_mark_cs6 -m comment --comment "PS4 UDP"
$IPT -t mangle -A dscp_classify -p tcp -s 10.0.1.60 -m multiport ! --dports 80,443 -g dscp_mark_cs6 -m comment --comment "PS4 TCP"
# League of Legends
$IPT -t mangle -A dscp_classify -p udp --dport 5000:5500 -g dscp_mark_cs6 -m comment --comment "League of Legends"
# World Of Warcraft
$IPT -t mangle -A dscp_classify -p tcp -m multiport --dports 1119,3724 -g dscp_mark_cs6 -m comment --comment "World of Warcraft"

###################################################################
### Streaming Media (Video Tin)
###################################################################

# IPset (Hostnames)
$IPT -t mangle -A dscp_classify -m set --match-set dscp_af41 dst -g dscp_mark_af41 -m comment --comment "IPSet AF41 Hostname Match"

###################################################################
# Background Traffic (Bulk Tin)
###################################################################

### IPset (Hostnames)
$IPT -t mangle -A dscp_classify -m set --match-set dscp_cs1 dst -g dscp_mark_cs1 -m comment --comment "IPSet CS1 Hostname Match"

### Mail
# SMTP/S
$IPT -t mangle -A dscp_classify -p tcp -m multiport --dports 25,465,587 -g dscp_mark_cs1 -m comment --comment "SMTP/S"
# IMAP/S
$IPT -t mangle -A dscp_classify -p tcp -m multiport --dports 143,993 -g dscp_mark_cs1 -m comment --comment "IMAP/S"
# POP/S
$IPT -t mangle -A dscp_classify -p tcp -m multiport --dports 110,995 -g dscp_mark_cs1 -m comment --comment "POP3"

# Consider "large" HTTP/S traffic (out/in) as Bulk
# Need some better solution for this
# But should work for most use cases
$IPT -t mangle -A dscp_classify -p tcp -m connbytes --connbytes 10485760: --connbytes-dir reply --connbytes-mode bytes -m multiport --dports 80,443 -g dscp_mark_cs1
$IPT -t mangle -A dscp_classify -p tcp -m connbytes --connbytes 1048576: --connbytes-dir original --connbytes-mode bytes -m multiport --dports 80,443 -g dscp_mark_cs1

/etc/config/firewall

config ipset
	option name 'dscp_cs6'
	option family 'ipv4'
	option storage 'hash'
	option hashsize '64'
	option match 'ip'
	option timeout '86400'

config ipset
	option name 'dscp_af41'
	option family 'ipv4'
	option storage 'hash'
	option hashsize '64'
	option match 'ip'
	option timeout '86400'

config ipset
	option name 'dscp_cs1'
	option family 'ipv4'
	option storage 'hash'
	option hashsize '64'
	option match 'ip'
	option timeout '86400'

/usr/lib/sqm/layer_cake.qos

    # Set default class to best effort
    $TC filter add dev $IFACE parent ffff: protocol ip prio 10 u32 \
    match u32 0 0 flowid :1 \
    action pedit munge ip tos set 0 \
    pipe csum ip continue
	
    # restore connection's mark value into the packet's fwmark.
    $TC filter add dev $IFACE parent ffff: protocol ip prio 20 u32 \
    match u32 0 0 flowid :1 \
    action connmark continue
	
    # match fwmarks and set tos flag in ip header
    # fix checksum
    # immediately redirect to ifb4eth1 to avoid further unnecessary packet matching

    # CS6
    $TC filter add dev $IFACE parent ffff: protocol ip prio 50 u32 \
    match mark 48 0xff flowid :1 \
    action pedit munge ip tos set 192 \
    pipe csum ip \
    pipe mirred egress redirect dev $DEV

    # AF41
    $TC filter add dev $IFACE parent ffff: protocol ip prio 40 u32 \
    match mark 34 0xff flowid :1 \
    action pedit munge ip tos set 136 \
    pipe csum ip \
    pipe mirred egress redirect dev $DEV

    # CS1
    $TC filter add dev $IFACE parent ffff: protocol ip prio 30 u32 \
    match mark 8 0xff flowid :1 \
    action pedit munge ip tos set 32 \
    pipe csum ip \
    pipe mirred egress redirect dev $DEV
	
    # Redirect everything else to ifb4eth1
    $TC filter add dev $IFACE parent ffff: protocol ip prio 100 u32 \
    match u32 0 0 flowid :1 \
    action mirred egress redirect dev $DEV

some hostname matching examples:
dnsmasq needs ipset support
also add /etc/dnsmasq.conf to files that should be saved.

/etc/dnsmasq.conf

### AF41
# Youtube
ipset=/googlevideo.com/dscp_af41
# NetFlix
ipset=/nflxvideo.net/dscp_af41
# AmazonVideo
ipset=/s3.ll.dash.row.aiv-cdn.net/d25xi40x97liuc.cloudfront.net/aiv-delivery.net/dscp_af41
# Facebook
ipset=/fbcdn.net/dscp_af41
# Twitch
ipset=/ttvnw.net/dscp_af41
# VeVo
ipset=/vevo.com/dscp_af41
# Spotify
ipset=/audio-fa.scdn.cot/dscp_af41
# Deezer
ipset=/deezer.com/dscp_af41
# SoundCloud
ipset=/sndcdn.com/dscp_af41
# last.fm
ipset=/last.fm/dscp_af41

### CS1
# Steam Download
ipset=/steamcontent.com/dscp_cs1
# PSN Download
ipset=/gs2.ww.prod.dl.playstation.net/dscp_cs1
# DropBox
ipset=/dropbox.com/dropboxstatic.com/dropbox-dns.com/log.getdropbox.com/dscp_cs1
# Google Drive
ipset=/drive.google.com/drive-thirdparty.googleusercontent.com//dscp_cs1
# Google Docs
ipset=/docs.google.com/docs.googleusercontent.com/dscp_cs1
# PlayStore Download
ipset=/gvt1.com/dscp_cs1
# WhatsApp Files
ipset=/mmg-fna.whatsapp.net/dscp_cs1
# Youtube Upload
ipset=/upload.youtube.com/upload.video.google.com/dscp_cs1
# WindowsUpdate
ipset=/windowsupdate.com/update.microsoft.com/dscp_cs1

1 Like

EF is the standard for voice but unfortunately the Linux device driver writers didn't know this, they use the top 3 bits and so EF winds up in VID WMM queue, because of this I strongly recommend CS6 for voice

1 Like

Can give more detail please?

This entire DSCP to WMM mapping...
Not all manufacturers use 100% the same mapping?

But it seems most use this mapping:
Maps precedence 6 and 7 to WMM Voice.
That would be CS6 and CS7.
EF (precedence 5) would results in the WMM video tin then.

So you right cs6 seems to be a better choice.
Ty =)

//edit
Answering my own question x)
I know what you mean now.
WMM (apparently?) uses precedence 4 and 5 for the "video access category" .
So CS5, VA, EF would all end up in this category.
Thanks again for pointing this out.

For reference what the kernel does, see:
https://wireless.wiki.kernel.org/en/developers/documentation/mac80211/queues

For 802.1p see:
https://en.wikipedia.org/wiki/IEEE_P802.1p

For 802.11e (WMM) see:
https://en.wikipedia.org/wiki/IEEE_802.11e-2005

Note how 802.1p still calls the "CS5" bit pattern VO/Voice, ist is only WMM which gets it "wrong"... well not technically wrong, as there is no hard definition for the meaning of the bits, just different traditions.

The whole thing is a mess, IMHO mainly caused by the unwillingness/inability to re-define DSCP 0 to denote background instead of best-effort, thereby throwing away a simple magnitude to priority mapping that would be understandable.

Mmmh, "ip tos set 192" will that leave the two ECN bits alone? I guess I will need to do some research...

Ty for the links.

Hmm seems like it does?
The ToS Header Field carries an 8bit value.
DSCP uses 6bit + 2bit for ECN.
tc pedit assumes an 8bit value.
So using 192 will result in 11000000
6bit dscp = 110000 = 48 = CS6
and 2 ecn bits set to 0
If we want to set both ecn bits to 1
195 should work.
Which is 11000011 in binary.

Yes that is my point, we want to leave the ECN bits alone as their value changes depending on congestion along the link, so I believe the mask should be 0xfc to leave the two ECN bits alone. At least that is the mask we use in sqm-scripts to extract the 6 dscp-bits out of the 8 TOS/priority bits.

Also something I found:
https://patchwork.kernel.org/patch/3212651/
This indicates it should be possible to teach/configure the kernel to correctly map EF to AC_VO. Assuming that one actually wants to use AC_VO in the first place, as this class does no aggregation but gets access to air-time with a higher likelihood than other ACs, thereby reducing the total usable bandwidth considerably (each wifi transmission has a relative high overhead that can be amortized better if more user data is send behind it, which is what aggregation does).

Best Regards

A mask value of 0xfc (11111100 ?) seems to be fine i guess.

Most systems like cisco, sonicwall, allow to remap the wmm classes?

I just had a look at hostapd and find the following in hostapd.conf:

# QoS Map Set configuration
#
# Comma delimited QoS Map Set in decimal values
# (see IEEE Std 802.11-2012, 8.4.2.97)
# 
# format:
# [<DSCP Exceptions[DSCP,UP]>,]<UP 0 range[low,high]>,...<UP 7 range[low,high]>
# 
# There can be up to 21 optional DSCP Exceptions which are pairs of DSCP Value
# (0..63 or 255) and User Priority (0..7). This is followed by eight DSCP Range
# descriptions with DSCP Low Value and DSCP High Value pairs (0..63 or 255) for
# each UP starting from 0. If both low and high value are set to 255, the
# corresponding UP is not used.
# 
# default: not set
#qos_map_set=53,2,22,6,8,15,0,7,255,255,16,31,32,39,255,255,40,47,255,255

So mapping to EF might still be okay assuming one also defines a proper qos_map. Now I need to figure out how to configure hostapd under openwrt :wink:

Hmm...

default: not set

So by default hostapd doesn't make use of dscp to wmm class mappings?

No, I believe that simply means that the kernel uses whatever default mapping it has.

@nbd, do you have any insight in openwrt's default dscp to WMM user priorities mapping?

So how to create a proper dscp to wmm mapping?

DSCP classes start with a decimal value of 8.
But its possible to have ToS values below that.

qos_map_set=0,7,8,15,16,23,24,31,32,39,40,47,48,55,56,255 ??

If i want to add dscp ef to wmm voice class:

qos_map_set=46,6,0,7,8,15,16,23,24,31,32,39,40,47,48,55,56,255

is this correct?

I believe adding "46,6" should work, but I am not sure whether all the the values actually make sense, UP 1 and 2 map to BK, but only DSCP CS1 should be signifying background/BK (well there is an current RFC for using another bit pattern out of CS0 for BK, but I digress). So I probably would change to:

qos_map_set=46,6,8,1,0,7,255,255,255,255,8,31,32,39,40,47,48,55,56,255

but I have no clue whether that actually is sane... I guess I will need to figure out how to diagnose/measure the AC classes of over the air packets first, because this will require looking at real data...

Sidenote: personally I am not even sure whether playing games with WMM is actually worth it at all, but I agree if one does it it would be nice to do it right :wink:

I think WMM voice class does result in much better audio quality. I think that is "worth it" for my use case at least. Thanks for this discussion on hostapd, I may work on that as well.