DSCP Classify - a service for applying DSCP to connections

moeller0 · January 9, 2023, 9:14am

This will configure the ingress qdisc to use besteffort instead of the default diffserv3. However the i/eqdisc opts will still override this*. The goal here is to simply ignore DSCPs by treating all DSCPs identical and avoid busywork, by first re-setting DSCPs and then still sort packets into the different tins (where all will end up in the besteffort tin once the DSCP has been changed to 0 anyways, so the sorting is useless work).

The way tc parses cake's options means that for any "class" of parameters the latest "wins", so with a sequence of diffserv3 ... besteffort ... diffserv4, diffserv4 will be selected.
Without any over-ride options configured the outcome is going to be:
squash_ingress = TRUE: diffserv3 ... besteffort` -> besteffort squash_ingress = FALSE: diffserv3 ...` -> diffserv3

moeller0 · January 10, 2023, 1:39pm

I just checked, diffserv8 actually sorts AF11-3 into tin 1 while besteffort resides in tin 2, while diffserv3/4 treat AF11-3 exactly like CS0. So not sure how to concisely describe that, but "prioritised amongst Best Effort" seems not intuitive to say, priority <= besteffort, no?

yelreve · February 13, 2023, 3:11pm

Update 2023-02-13:

The service config file now supports specifying the number of connections required to classify a client or service as threaded (i.e. P2P/multi connection Steam downloads).
Refactored the code to support service reloads using the current snapshot build nft binaries (versions >1.0.2)

rspierz · February 14, 2023, 1:36pm

i just tried installing this from your intstuctions but unsure if its setup correctly?

service starts but displays the following:

active with no instances

Is there a way to check if its working properly?

rspierz · February 14, 2023, 2:15pm

 nft -f "/etc/dscpclassify.d/main.nft"
/etc/dscpclassify.d/main.nft:38:1-44: Error: File not found: /tmp/etc/dscpclassify-pre.include
include "/tmp/etc/dscpclassify-pre.include"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/etc/dscpclassify.d/main.nft:124:1-45: Error: File not found: /tmp/etc/dscpclassify-post.include
include "/tmp/etc/dscpclassify-post.include"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

could this be root cause?

yelreve · February 14, 2023, 6:51pm

I suspect it is in fact working for you That shell message would be generated by the status check I introduced for correctly handling service reload, line of code responsible below:
service dscpclassify status 2>/dev/null

I'll supress this in a quick commit to avoid confusion (I'd already supressed stderr).

You can manually check if the service has started successfully and created the dscpclassify firewall table by running:
nft list table inet dscpclassify

Update:
Commit Improve manual service config reload messaging. · jeverley/dscpclassify@d5d20b0 (github.com) pushed to address this, @rspierz let me know of any further observations on your end.

rspierz · February 14, 2023, 7:36pm

great looks like it is working.

I added one rule so far for webex, although i dont think its perfect yet

config rule
	option name 'Webex video/audio'
	option proto 'udp'
	option dest_port '9000'
	option dest_port '5004'
	list dest_ip '23.89.0.0/16'
        list dest_ip '62.109.192.0/18'
	list dest_ip '64.68.96.0/19'
	list dest_ip '66.114.160.0/20'
	list dest_ip '66.163.32.0/19'
	list dest_ip '69.26.160.0/19'
	list dest_ip '114.29.192.0/19'
	list dest_ip '150.253.128.0/17'
	list dest_ip '170.72.0.0/16'
	list dest_ip '170.133.128.0/18'
	list dest_ip '173.39.224.0/19'
	list dest_ip '173.243.0.0/20'
	list dest_ip '207.182.160.0/19'
	list dest_ip '209.197.192.0/19'
	list dest_ip '210.4.192.0/20'
	list dest_ip '216.151.128.0/19'
	option class 'af41'

rspierz · February 14, 2023, 7:43pm

although since my webex meetings are over vpn. I dont think its working as all my flows are going to best effort with a little going to voice probably for phone wifi calling.

qdisc cake 8008: dev ifb4eth1 root refcnt 2 bandwidth 925Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 46 mpu 84 
 Sent 13977533038 bytes 15296305 pkt (dropped 25, overlimits 14670169 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 1326784b of 15140Kb
 capacity estimate: 925Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       92 /    1546
 average network hdr offset:           14

                   Bulk  Best Effort        Video        Voice
  thresh      57812Kbit      925Mbit   462500Kbit   231250Kbit
  target            5ms          5ms          5ms          5ms
  interval        100ms        100ms        100ms        100ms
  pk_delay          0us         24us          0us         12us
  av_delay          0us          4us          0us          2us
  sp_delay          0us          0us          0us          2us
  backlog            0b           0b           0b           0b
  pkts                0     15284746            0        11584
  bytes               0  13976366196            0      1204692
  way_inds            0      2938299            0            0
  way_miss            0        79658            0            2
  way_cols            0            0            0            0
  drops               0           25            0            0
  marks               0            0            0            0
  ack_drop            0            0            0            0
  sp_flows            0            1            0            1
  bk_flows            0            1            0            0
  un_flows            0            0            0            0
  max_len             0         1514            0          104
  quantum          1514         1514         1514         1514

yelreve · February 16, 2023, 5:56pm

Is the VPN on your client device or router? OpenWrt won't be able to see packet destinations/ports within a VPN connection established from a client device.

rspierz · February 16, 2023, 6:11pm

Yeah correct client device:(. For now I'm stuck but maybe I'll try to get it split tunneled.

rspierz · February 17, 2023, 3:30pm

Should i be seeing traffic being passed into the different tins like bulk and video? I still do not see anything in these ones only a little in voice and most in best effort. Curious if your output is different then mine

Dopam-IT_1987 · February 17, 2023, 4:00pm

hello for the vpn is the overhead the same as the wan and can we use diffserv4? thanks

Hudra · February 17, 2023, 6:03pm

Depends on what your DSCPCLASSIFY config looks like.

What is the output of:

cat /etc/config/sqm
cat /etc/config/dscpclassify

Also keep in mind that with the latest version there seems to be a bug with the automatic reload of DSCPCLASSIFY after a reboot or if an interface is down, so first check if it is running properly:

nft list table inet dscpclassify

or you can also check via luci Status/Firewall and search for "dscpclassify" or just scroll down...

rspierz · February 17, 2023, 6:20pm

cat /etc/config/sqm

config queue 'eth1'
	option ingress_ecn 'ECN'
	option debug_logging '0'
	option verbosity '5'
	option qdisc_advanced '1'
	option linklayer 'ethernet'
	option qdisc_really_really_advanced '1'
	option interface 'eth1'
	option egress_ecn 'NOECN'
	option qdisc 'cake'
	option enabled '1'
	option download '925000'
	option upload '915000'
	option script 'layer_cake_ct.qos'
	option squash_dscp '0'
	option squash_ingress '0'
	option iqdisc_opts 'diffserv4 nat dual-dsthost ingress mpu 74 noatm'
	option eqdisc_opts 'diffserv4 nat dual-srchost mpu 74 noatm'
	option overhead '35'

cat /etc/config/dscpclassify

config global 'global'
	option class_bulk 'le'
	option class_high_throughput 'af13'
	option client_hints '1'
	option threaded_client_min_bytes '10000'
	option threaded_service_min_bytes '1000000'
	option wmm '0'

config rule
	option name 'DNS'
	list proto 'tcp'
	list proto 'udp'
	list dest_port '53'
	list dest_port '853'
	list dest_port '5353'
	option class 'cs5'

config rule
	option name 'DoH'
	list proto 'tcp'
	list proto 'udp'
	list dest_ip '8.8.8.8'				# Google
	list dest_ip '8.8.4.4'				# Google
	list dest_ip '1.1.1.1'				# Cloudflare
	list dest_ip '1.0.0.1'				# Cloudflare
	list dest_ip '9.9.9.9'				# Quad9 Secured
	list dest_ip '149.112.112.112'		# Quad9 Secured
	list dest_ip '9.9.9.11'				# Quad9 Secured w/ECS
	list dest_ip '149.112.112.11'		# Quad9 Secured w/ECS
	list dest_ip '94.140.14.0/24'		# AdGuard
	list dest_ip '2001:4860:4860::8888' # Google
	list dest_ip '2001:4860:4860::8844' # Google
	list dest_ip '2606:4700:4700::1111' # Cloudflare
	list dest_ip '2606:4700:4700::1001' # Cloudflare
	list dest_ip '2620:fe::fe'			# Quad9 Secured
	list dest_ip '2620:fe::9'			# Quad9 Secured
	list dest_ip '2620:fe::11'			# Quad9 Secured w/ECS
	list dest_ip '2620:fe::fe:11'		# Quad9 Secured w/ECS
	list dest_ip '2a10:50c0::ad1:ff'	# AdGuard
	list dest_ip '2a10:50c0::ad2:ff'	# AdGuard
	list dest_ip '2a10:50c0::ded:ff'	# AdGuard Dedicated
	list dest_port '443'
	option class 'cs5'

config rule
	option name 'BOOTP/DHCP'
	option proto 'udp'
	list dest_port '67'
	list dest_port '68'
	option class 'cs5'

config rule
	option name 'NTP'
	option proto 'udp'
	option dest_port '123'
	option class 'cs5'

config rule
	option name 'SSH'
	option proto 'tcp'
	option dest_port '22'
	option class 'cs2'


config rule
	option name 'Microsoft Teams voice'
	option proto 'udp'
	option src_port '50000-50019'
	list dest_ip '13.107.64.0/18'
	list dest_ip '52.112.0.0/14'
	list dest_ip '52.122.0.0/15'
	list dest_ip '2603:1063::/39'
	option class 'ef'

config rule
	option name 'Microsoft Teams video'
	option proto 'udp'
	option src_port '50020-50039'
	option dest_port '3478-3481'
	list dest_ip '13.107.64.0/18'
	list dest_ip '52.112.0.0/14'
	list dest_ip '52.122.0.0/15'
	list dest_ip '2603:1063::/39'
	option class 'af41'

config rule
	option name 'Microsoft Teams sharing'
	option proto 'udp'
	option src_port '50040-50059'
	option dest_port '3478-3481'
	list dest_ip '13.107.64.0/18'
	list dest_ip '52.112.0.0/14'
	list dest_ip '52.122.0.0/15'
	list dest_ip '2603:1063::/39'
	option class 'af21'

config rule
	option name 'Webex video/audio'
	option proto 'udp'
	option dest_port '9000'
	option dest_port '5004'
	option dest_port '4501'
	list dest_ip '23.89.0.0/16'
        list dest_ip '62.109.192.0/18'
	list dest_ip '64.68.96.0/19'
	list dest_ip '66.114.160.0/20'
	list dest_ip '66.163.32.0/19'
	list dest_ip '69.26.160.0/19'
	list dest_ip '114.29.192.0/19'
	list dest_ip '150.253.128.0/17'
	list dest_ip '170.72.0.0/16'
	list dest_ip '170.133.128.0/18'
	list dest_ip '173.39.224.0/19'
	list dest_ip '173.243.0.0/20'
	list dest_ip '207.182.160.0/19'
	list dest_ip '209.197.192.0/19'
	list dest_ip '210.4.192.0/20'
	list dest_ip '216.151.128.0/19'
	option class 'af41'

config rule # A rule which marks all non-HTTP UDP connections from a specific IP as cs4
	option name 'Game Console non-HTTP'
	option proto 'udp'
	list src_ip '192.168.107.100'
	list dest_port '!80'
	list dest_port '!443'
	option class 'cs4'

config rule
	option name 'Google Meet'
	option proto 'udp'
	option dest_port '19302-19309'
	list dest_ip '142.250.82.0/24'
	option class 'af41'

nft list tablet inet dscpclassify command spits out the full config looks like

ive also been doing a restart of the service instead of reloads when changing the config.

i've been testing webex and thought i'd see the metrics in the video tin reflect it.

Hudra · February 17, 2023, 7:42pm

As @yelreve told you before:

Within a VPN connection everything will be sent over one specific port to a fixed destination IP. Mostly port 1194 or 443 so you can’t really target one specific port or Ip.

You could try to establish the vpn connection directly from your router and use Policy Routing to just send traffic from your specific client over the vpn.

The config looks good to me, everything should work as expected… if you are unsure if it works you can just use a client which has no vpn connection and just ping 8.8.8.8 and make a rule to mark icmp or 8.8.8.8 to cakes video tin…

rspierz · February 17, 2023, 9:09pm

i was able to bump a webex session outside of the vpn to dst ip on port 4501 matching the webex rule in dscpclassfiy. However i would have expected to see the tin metrics increase for video no? Or is my af41 classification not mapping to this tin?

rspierz · February 17, 2023, 9:13pm

following the advice to map icmp to 1.1.1.1 and map to af41 also does not show traffic entering the video tin. Perhaps a bug?

Hudra · February 17, 2023, 9:21pm

af41 should go into the video tin…

Mmmh maybe… what is the whole output of:

tc -s qdisc

rspierz · February 17, 2023, 9:53pm

qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 166349174659 bytes 150344670 pkt (dropped 0, overlimits 0 requeues 170667) 
 backlog 0b 0p requeues 170667
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 23242580887 bytes 22961519 pkt (dropped 0, overlimits 0 requeues 63440) 
 backlog 0b 0p requeues 63440
  maxpacket 12112 drop_overlimit 0 new_flow_count 17503 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 61938279886 bytes 56174754 pkt (dropped 0, overlimits 0 requeues 74432) 
 backlog 0b 0p requeues 74432
  maxpacket 68130 drop_overlimit 0 new_flow_count 36801 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 29057736 bytes 71879 pkt (dropped 0, overlimits 0 requeues 3203) 
 backlog 0b 0p requeues 3203
  maxpacket 12112 drop_overlimit 0 new_flow_count 7 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 22178933581 bytes 18249558 pkt (dropped 0, overlimits 0 requeues 21876) 
 backlog 0b 0p requeues 21876
  maxpacket 68130 drop_overlimit 0 new_flow_count 9584 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 58960322569 bytes 52886960 pkt (dropped 0, overlimits 0 requeues 7716) 
 backlog 0b 0p requeues 7716
  maxpacket 13626 drop_overlimit 0 new_flow_count 5595 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 802f: dev eth1 root refcnt 2 bandwidth 915Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 35 mpu 74 
 Sent 8949676052 bytes 11950708 pkt (dropped 162, overlimits 6548560 requeues 1949) 
 backlog 0b 0p requeues 1949
 memory used: 1921600b of 15140Kb
 capacity estimate: 915Mbit
 min/max network layer size:           16 /    1500
 min/max overhead-adjusted size:       74 /    1535
 average network hdr offset:           14

                   Bulk  Best Effort        Video        Voice
  thresh      57187Kbit      915Mbit   457500Kbit   228750Kbit
  target            5ms          5ms          5ms          5ms
  interval        100ms        100ms        100ms        100ms
  pk_delay          0us          4us          0us          0us
  av_delay          0us          1us          0us          0us
  sp_delay          0us          0us          0us          0us
  backlog            0b           0b           0b           0b
  pkts                0     11950870            0            0
  bytes               0   8949912928            0            0
  way_inds            0       112905            0            0
  way_miss            0        65862            0            0
  way_cols            0            0            0            0
  drops               0          162            0            0
  marks               0            0            0            0
  ack_drop            0            0            0            0
  sp_flows            0            4            0            0
  bk_flows            0            1            0            0
  un_flows            0            0            0            0
  max_len             0         1514            0            0
  quantum          1514         1514         1514         1514

qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
 Sent 16712520782 bytes 20197945 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.107 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 18006736391 bytes 31021778 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 24480 drop_overlimit 0 new_flow_count 776871 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8030: dev ifb4eth1 root refcnt 2 bandwidth 925Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 35 mpu 74 
 Sent 16908378873 bytes 20140524 pkt (dropped 57421, overlimits 19420659 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 7999872b of 15140Kb
 capacity estimate: 925Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       81 /    1535
 average network hdr offset:           14

                   Bulk  Best Effort        Video        Voice
  thresh      57812Kbit      925Mbit   462500Kbit   231250Kbit
  target            5ms          5ms          5ms          5ms
  interval        100ms        100ms        100ms        100ms
  pk_delay          0us        109us          0us          8us
  av_delay          0us         44us          0us          2us
  sp_delay          0us          0us          0us          2us
  backlog            0b           0b           0b           0b
  pkts                0     20194399            0         3546
  bytes               0  16994923228            0       368784
  way_inds            0       444747            0            0
  way_miss            0        68649            0            1
  way_cols            0            0            0            0
  drops               0        57421            0            0
  marks               0            0            0            0
  ack_drop            0            0            0            0
  sp_flows            0            4            0            1
  bk_flows            0            1            0            0
  un_flows            0            0            0            0
  max_len             0         1514            0          104
  quantum          1514         1514         1514         1514

Hudra · February 17, 2023, 10:13pm

Another attempt:
Could you please try out this rule:

config rule
	option name 'ICMP'
	option proto 'icmp'
	option class 'cs3'
	option enabled '1'
	option counter '1'

And then ping 8.8.8.8 on one of your clients. If this also doesn’t work then you could try to disable client_hints and try again.