Qosify: new package for DSCP marking + cake

It makes sense to specify the entire range of ports if you have a stadia controller.
I analyze the traffic with the tcpdump command.

Otherwise agree with you, my chromecast ultra is using port 44700

1 Like

Sorry I misread that. If you use a managed switch it could be tagging packets. At least my cisco does.

Nonetheless I think there is an issue with tagging ports in the 00-defaults.conf. Could you do me a favor and test something for me?

Your video tin should be totally empty by now. So, if you would change:

To

Those packets should actually go into the video tin. Can you verify that?

I'm mapping everything except IPTV, ICMP, browsing and avg_pkt_length under 500 bytes to bulk queue now. Browsing with more than 100 packets per second also gets mapped to bulk.

So DNS is proritzed in the Voice queue, IPTV is prioritzed in the Video queue, ICMP and browsing gets mapped to best effort, and if there's a download with more than 100 packets per seconds on port 80 or port 443 the flow gets mapped to bulk. P2P gaming flows gets mapped to best effort, as the the avg_pkt_length is < 500 bytes.

This is the first setup which for me which works excellent without a stuttering IPTV.

config defaults
        list defaults /etc/qosify/*.conf
        option dscp_prio besteffort
        option dscp_icmp besteffort
        option dscp_default_udp bulk
        option dscp_default_tcp bulk
        option dscp_bulk bulk
        option prio_max_avg_pkt_len 500

config class besteffort
        option ingress CS0
        option egress CS0
        option bulk_trigger_pps 100
        option bulk_trigger_timeout 5
        option dscp_bulk bulk

config class bulk
        option ingress CS1
        option egress CS1

config class video
        option ingress CS3
        option egress CS3

config class voice
        option ingress CS6
        option egress CS6
        option bulk_trigger_pps 100
        option bulk_trigger_timeout 5
        option dscp_bulk bulk

So here is the problem in spite if there being registries for which application uses which port, that really is not enforced or even enforceable, so pure port-based classification is prone to false-positives, the fewer ports you use per rule the less likely are accidental collisions with other application using the same ports. So it seems sane to only specify the minimal number of ports that are actually used. In addition it can also help (if supported by the code) to restrict such rules to combinations of SRC-ports and internal DST-addresses (think only DSCP-mark all those UDP packets coming from port 44700 AND destined for 192.168.1.42 (as a stand-in for the internal IP address of the console/computer that expects these packets)).

2 Likes

If you replace this with
option dscp_icmp +besteffort
you can actually still use ping's command line options to steer ICMP packets into any desired tier to probe the latency under load there...

2 Likes

Hi @moeller0,

thanks for your advice.
I've edited my post again. Maybe you read over it again and tell me what do you think about the setup. :grin:

Thanks! :slight_smile:

btw.: I don't know exactly why but with irqbalance and a restart of the CPE the first CPU core isn't running out anymore. :+1:

I am not using qosify myself (so no first hand experience, due to my router still being on an OpenWrt19 variant), so can not say much about qosify configs. Personally I think being able to use ping to probe all priority tiers is simply a good idea (at least unix ping has easily accessible command line arguments to set the DSCP or TOS value, side note if you have a decimal (6bit) DSCP but ping wants a (8bit) decimal TOS value, simply multiply the DSCP by four, which is equivalent to shifting two bits). For anything else you need to decide whether your configuration meets your intended policy :wink:

2 Likes

Ok, yes, I thought you could a few objections :grin:

But thank you anyway! :slight_smile:

How could I have objections to your configuration :wink: after all this this is mostly a policy question. If the policy and implementation do not agree I am happy to have a look (constrained by my lack of expertise with qosify) but as I say "your network, your rules".

One thing I am quite happy about is that fq_codel/cake basically do the right thing by default and give new/sparse flows a gentle boost over queueing flows, eradicating the need for the complex priority hierarchies that people used in the past... being lazy, I mostly relay on that sparse boosting and have no explicit DSCP marking rule (my VoIP base station marks EF as does my ISP for VoIP/SIP packets, since I am using their services), but still I run diffserv4 so if I have to I can quickly whip up a marking rule to get priority to flows that need them (so far that has not happened though). Between the sparse boosting, the flow queueing scheduler and the per-internal-IP fairness modes I lost the need for further prioritization :wink:

3 Likes

@Hudra see ....

I read that. My post wasn’t related to your issue. I’m just curious if tagging (done in 00-defaults.conf) is working properly or if there is a bug. So, I asked you to do me a favor and verify/test that by switching udp:4470 from Premium to CS3. If qosify is working correct you should then see packets in your video tin when playing with stadia. If not, I’m guess I’m not the only one experiencing this issue.

I have tired this and confirm that Video and Bulk gets lots of hits but others also, Now Voice is empty for me where CS0 was.

Out of interest? What device do you run for qosify? @ilshatms

Edit:
Did some Google searches and im using tcpdump -v -n -i br-lan portrange 1024-65535 and then using ToS Hexadecimal here to find is CS vales.
https://linuxreviews.org/Type_of_Service_(ToS)_and_DSCP_Values

So, Could it be problem with Downloads in Qosify if set to Bulk = LE? as it shows 0x0 = CS0 = WRONG CLASS?
IP (tos 0x0, ttl 128, id 25810, offset 0, flags [none], proto UDP (17), length 67)
192.168.1.253.57997 > AMAZONIP.9009: UDP, length 39

Uploading in Qosify as this goes into 0x4 = LE - No problems here
(tos 0x4, ttl 44, id 11690, offset 0, flags [none], proto UDP (17), length 96)
AMAZONIP.9009 > 192.168.1.253.57997: UDP, length 68

I think is a issue with targets/ipq40xx devices unless someone else with a different target can confirm is just not the ipq40xx devices for this type of config - Seems like other ports are not going into Bulk, E.g SSH port 22 going into AF21 and iptables ToS name Minimize-Delay so there is no other DSCP?

01:53:28.674451 IP (tos 0x48, ttl 64, id 15478, offset 0, flags [DF], proto TCP (6), length 52)
192.168.1.11.38488 > 192.168.1.1.22: Flags [.], cksum 0x0a0e (correct), ack 1587088, win 12645, options [nop,nop,TS val 2623088975 ecr 3848474613], length 0

01:53:28.674541 IP (tos 0x10, ttl 64, id 18321, offset 0, flags [DF], proto TCP (6), length 480)
192.168.1.1.22 > 192.168.1.11.38488: Flags [P.], cksum 0x892f (incorrect -> 0xdb79), seq 1587088:1587516, ack 1045, win 2003, options [nop,nop,TS val 3848474613 ecr 2623088975], length 428

/etc/config/qosify

config defaults
	list defaults /etc/qosify/*.conf
	option dscp_icmp Premium
	option dscp_default_tcp Bulk
	option dscp_default_udp Bulk

config class Premium
	option ingress CS0
	option egress CS0

config class Bulk
	option ingress LE
	option egress LE

config interface wan
	option name wan
	option disabled 0
	option bandwidth_down 60mbit
	option bandwidth_up 17mbit
	option overhead_type pppoe-ptm
	# defaults:
	option ingress 1
	option egress 1
	option mode diffserv4
	option nat 1
	option host_isolate 1
	option autorate_ingress 0
	option ingress_options ""
	option egress_options "wash"
	option options "ether-vlan"

With a simple 00-defaults.conf

# Google Stadia
udp:44700	Premium

Router to LAN traffic doesn’t get modified by Qosify since the BPF filters are on the WAN interface only.

To monitor qosify results, tcpdump needs to run traffic in the proper direction. I’ve been experimenting with capturing all non-CS0 traffic.

Download (exclude ssh port 22 since output will be sent over port 22):

tcpdump -i br-lan -v -n 'dst net 192.168.1.0/24 and ( ip and ip[1] & 0xfc > 0 ) and ! port 22'

Upload (replace WANIP and maybe WAN interface name):

tcpdump -i eth1 -v -n 'src host WANIP and ( ip and ip[1] & 0xfc > 0 )'
1 Like

Learn something new, Thanks :slight_smile:

@dave14305

For me, This seems better for when things are "Downloading from the router" as is saying from MYPC port 57666 > GoogleIP port 993


tcpdump -i br-lan -v -n 'src net 192.168.1.0/24 and ( ip and ip[1] & 0xfc > 0 ) and ! port 22'

@dave14305
possible Multicast that's going into tos 0xc0 = CS6 ? Even though is not into the 0-defaults.conf

02:44:29.235719 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 40, options (RA))
HuaweiY6.lan > 224.0.0.22: igmp v3 report, 1 group record(s) [gaddr 224.0.0.251 to_in, 0 source(s)]
02:44:29.298432 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 40, options (RA))
HuaweiY6.lan > 224.0.0.22: igmp v3 report, 1 group record(s) [gaddr 224.0.0.251 to_in, 0 source(s)]

This is captured before it is processed by qosify. Since it’s igmp, I think it gets bypassed since it is not icmp, tcp or udp; but I’m not sure how to interpret the code where key=0 and what value gets returned.

https://lxr.openwrt.org/source/qosify/qosify-bpf.c#L273

belkin rt3200

1 Like

Found two way of getting around this (for now) BWARE if your ISP uses IPTV, This will affect you!

1 iptables
iptables -I FORWARD -o br-lan -s 224.0.0.0/4 -j DROP

2 Luci
Login into GUI 192.168.1.1 > Network > Devices > br-lan > configure > Advanced device options > Enable multicast support UNTICKED and save, save and apply

3 SSH /etc/config/network
Add below the config file.

# WAN
config device
	option name 'eth0.2'
	option ipv6 '0'
	option multicast '0'

# 2.4 GHz Wireless
config device
	option name 'wlan0'
	option multicast '0'

# 5 GHz Wireless
config device
	option name 'wlan1'
	option multicast '0'

and then dont forgot to Reboot the Router.

1 Like

I wanted to share these Qosify commands or script that I use to run on first boot when I build my custom OpenWrt firmware or to copy and paste at the same time in the SSH client after doing a factory reset because I upgraded the firmware, so that I already have Qosify configured as I recommend in my above post.

You only have to write your bandwidth and the type of connection you have and nothing else:

# Qosify
BANDWIDTH_UP="50mbit"
BANDWIDTH_DOWN="320mbit"
OVERHEAD_TYPE="docsis"
INGRESS_OPTIONS=""
EGRESS_OPTIONS="wash"
INGRESS_EGRESS_OPTIONS="ether-vlan"

######################################
#   JESUS CHRIST IS THE LORD! AMEN   #
######################################

# Qosify
# Clear default config
uci del qosify.@defaults[0].dscp_prio
uci del qosify.@defaults[0].prio_max_avg_pkt_len
uci del qosify.besteffort
uci del qosify.bulk
uci del qosify.video
uci del qosify.voice
# ICMP (ping) and default class for "unmarked traffic"
uci set qosify.@defaults[0].dscp_icmp="+besteffort"
uci set qosify.@defaults[0].dscp_default_tcp="unmarked_traffic"
uci set qosify.@defaults[0].dscp_default_udp="unmarked_traffic"
# unmarked_traffic class
uci set qosify.unmarked_traffic=class
uci set qosify.unmarked_traffic.ingress="CS1"
uci set qosify.unmarked_traffic.egress="CS1"
uci set qosify.unmarked_traffic.prio_max_avg_pkt_len="1256"
uci set qosify.unmarked_traffic.dscp_prio="CS4"
uci set qosify.unmarked_traffic.bulk_trigger_pps="250"
uci set qosify.unmarked_traffic.bulk_trigger_timeout="10"
uci set qosify.unmarked_traffic.dscp_bulk="CS1"
# browsing class
uci set qosify.browsing=class
uci set qosify.browsing.ingress="CS0"
uci set qosify.browsing.egress="CS0"
uci set qosify.browsing.prio_max_avg_pkt_len="575"
uci set qosify.browsing.dscp_prio="AF41"
uci set qosify.browsing.bulk_trigger_pps="1000"
uci set qosify.browsing.bulk_trigger_timeout="10"
uci set qosify.browsing.dscp_bulk="CS1"
# bulk class
uci set qosify.bulk=class
uci set qosify.bulk.ingress="CS1"
uci set qosify.bulk.egress="CS1"
# besteffort class
uci set qosify.besteffort=class
uci set qosify.besteffort.ingress="CS0"
uci set qosify.besteffort.egress="CS0"
# network_services class
uci set qosify.network_services=class
uci set qosify.network_services.ingress="CS2"
uci set qosify.network_services.egress="CS2"
# broadcast_video class
uci set qosify.broadcast_video=class
uci set qosify.broadcast_video.ingress="CS3"
uci set qosify.broadcast_video.egress="CS3"
# gaming class
uci set qosify.gaming=class
uci set qosify.gaming.ingress="CS4"
uci set qosify.gaming.egress="CS4"
# multimedia_conferencing class
uci set qosify.multimedia_conferencing=class
uci set qosify.multimedia_conferencing.ingress="AF42"
uci set qosify.multimedia_conferencing.egress="AF42"
uci set qosify.multimedia_conferencing.prio_max_avg_pkt_len="575"
uci set qosify.multimedia_conferencing.dscp_prio="AF41"
# telephony class
uci set qosify.telephony=class
uci set qosify.telephony.ingress="EF"
uci set qosify.telephony.egress="EF"
# Reorder classes
uci reorder qosify.unmarked_traffic=1
uci reorder qosify.browsing=2
uci reorder qosify.bulk=3
uci reorder qosify.besteffort=4
uci reorder qosify.network_services=5
uci reorder qosify.broadcast_video=6
uci reorder qosify.gaming=7
uci reorder qosify.multimedia_conferencing=8
uci reorder qosify.telephony=9
# CAKE
uci set qosify.wan.name="wan"
uci set qosify.wan.disabled="0"
uci set qosify.wan.bandwidth_up="$BANDWIDTH_UP"
uci set qosify.wan.bandwidth_down="$BANDWIDTH_DOWN"
uci set qosify.wan.overhead_type="$OVERHEAD_TYPE"
uci set qosify.wan.autorate_ingress="0"
uci set qosify.wan.ingress_options="$INGRESS_OPTIONS"
uci set qosify.wan.egress_options="$EGRESS_OPTIONS"
uci set qosify.wan.options="$INGRESS_EGRESS_OPTIONS"
# Rules
cat << "EOF" > /etc/qosify/00-defaults.conf
# SSH
tcp:22    network_services

# NTP
udp:123   network_services

# DNS
tcp:53    network_services
tcp:5353  network_services
udp:53    network_services
udp:5353  network_services

# DNS over TLS (DoT)
tcp:853   multimedia_conferencing
udp:853   multimedia_conferencing

# HTTP/HTTPS/QUIC
tcp:80    browsing
tcp:443   browsing
udp:80    browsing
udp:443   browsing

# Microsoft (Download)
dns:*1drv*                 bulk
dns:*backblaze*            bulk
dns:*backblazeb2*          bulk
dns:*ms-acdc.office*       bulk
dns:*onedrive*             bulk
dns:*sharepoint*           bulk
dns:*update.microsoft*     bulk
dns:*windowsupdate*        bulk

# MEGA (Download)
dns:*mega*                 bulk

# Dropbox (Download)
dns:*dropboxusercontent*   bulk

# Google (Download)
dns:*drive.google*         bulk
dns:*googleusercontent*    bulk

# Steam (Download)
dns:*steamcontent*         bulk

# Epic Games (Download)
dns:*download.epicgames*   bulk
dns:*download2.epicgames*  bulk
dns:*download3.epicgames*  bulk
dns:*download4.epicgames*  bulk
dns:*epicgames-download1*  bulk

# YouTube
dns:*googlevideo*   besteffort

# Facebook
dns:*fbcdn*         besteffort

# Twitch
dns:*ttvnw*         besteffort

# TikTok
dns:*tiktok*        besteffort

# Netflix
dns:*nflxvideo*     besteffort

# Amazon Prime Video
dns:*aiv-cdn*       besteffort
dns:*aiv-delivery*  besteffort
dns:*pv-cdn*        besteffort

# Disney Plus
dns:*disney*        besteffort
dns:*dssott*        besteffort

# HBO
dns:*hbo*           besteffort
dns:*hbomaxcdn*     besteffort

# BitTorrent
tcp:6881-7000    bulk
tcp:51413        bulk
udp:6771         bulk
udp:6881-7000    bulk
udp:51413        bulk

# Usenet
tcp:119          bulk
tcp:563          bulk

# Live Streaming to YouTube Live, Twitch, Vimeo and LinkedIn Live
tcp:1935-1936    broadcast_video
tcp:2396         broadcast_video
tcp:2935         broadcast_video

# Xbox
tcp:3074         gaming
udp:88           gaming
#udp:500         gaming # UDP port already used in "VoWiFi" rules
udp:3074         gaming
udp:3544         gaming
#udp:4500        gaming # UDP port already used in "VoWiFi" rules

# PlayStation
tcp:3478-3480    gaming
#udp:3478-3479   gaming # UDP ports already used in "Zoom" rules

# Call of Duty
#tcp:3074        gaming # TCP port already used in "Xbox" rules
tcp:3075-3076    gaming
#udp:3074        gaming # UDP port already used in "Xbox" rules
udp:3075-3079    gaming
udp:3658         gaming

# FIFA
tcp:3659         gaming
udp:3659         gaming

# Minecraft
tcp:25565        gaming
udp:19132-19133  gaming
udp:25565        gaming

# Supercell Games
tcp:9339         gaming
udp:9339         gaming

# Zoom, Microsoft Teams, Skype and FaceTime (they use these same ports)
udp:3478-3497    multimedia_conferencing

# Zoom
dns:*zoom*       multimedia_conferencing
tcp:8801-8802    multimedia_conferencing
udp:8801-8810    multimedia_conferencing

# Skype
dns:*skype*      multimedia_conferencing

# FaceTime
udp:16384-16387  multimedia_conferencing
udp:16393-16402  multimedia_conferencing

# GoToMeeting
udp:1853         multimedia_conferencing
udp:8200         multimedia_conferencing

# Webex Meeting
tcp:5004         multimedia_conferencing
udp:9000         multimedia_conferencing

# Jitsi Meet
tcp:5349         multimedia_conferencing
udp:10000        multimedia_conferencing

# Google Meet
udp:19302-19309  multimedia_conferencing

# TeamViewer
tcp:5938         multimedia_conferencing
udp:5938         multimedia_conferencing

# Voice over Internet Protocol (VoIP)
tcp:5060-5061    telephony
udp:5060-5061    telephony

# Voice over WiFi or WiFi Calling (VoWiFi)
udp:500          telephony
udp:4500         telephony
EOF

# Saving modified values
uci commit qosify
/etc/init.d/qosify restart

Information about my Qosify configuration:

7 Likes

These keywords in CAKE have been provided to represent a number of common link technologies:

Write the keyword below
***********************
"conservative"     Failsafe     (overhead 48 atm)
"ethernet"         Ethernet     (overhead 38 mpu 84 noatm)
"docsis"           Cable Modem  (overhead 18 mpu 64 noatm)
"pppoe-ptm"        VDSL2        (overhead 30 ptm)
"bridged-ptm"      VDSL2        (overhead 22 ptm)
"pppoa-vcmux"      ADSL         (overhead 10 atm)
"pppoa-llc"        ADSL         (overhead 14 atm)
"pppoe-vcmux"      ADSL         (overhead 32 atm)
"pppoe-llcsnap"    ADSL         (overhead 40 atm)
"bridged-vcmux"    ADSL         (overhead 24 atm)
"bridged-llcsnap"  ADSL         (overhead 32 atm)
"ipoa-vcmux"       ADSL         (overhead 8  atm)
"ipoa-llcsnap"     ADSL         (overhead 16 atm)

# If you are unsure, then write "conservative" as a general safe value.
# "raw" keyword turns off all overhead compensation in CAKE.

"ether-vlan" keyword information:

"ether-vlan" adds '4 bytes' to the overhead.

- This keyword is not for standalone use, but act as a modifier to some previous presets or keywords.
- In addition to those previous presets or keywords it is common to have VLAN tags (4 extra bytes) or PPPoE encapsulation (8 extra bytes).
# This keyword "ether-vlan" may be repeated as necessary.

For true ATM links (ADSL), one often can measure the real per-packet overhead empirically, see for further information how to do that:

4 Likes