This is captured before it is processed by qosify. Since it’s igmp, I think it gets bypassed since it is not icmp, tcp or udp; but I’m not sure how to interpret the code where key=0 and what value gets returned.
I wanted to share these Qosify commands or script that I use to run on first boot when I build my custom OpenWrt firmware or to copy and paste at the same time in the SSH client after doing a factory reset because I upgraded the firmware, so that I already have Qosify configured as I recommend in my above post.
You only have to write your bandwidth and the type of connection you have and nothing else:
# Qosify
BANDWIDTH_UP="50mbit"
BANDWIDTH_DOWN="320mbit"
OVERHEAD_TYPE="docsis"
INGRESS_OPTIONS=""
EGRESS_OPTIONS="wash"
INGRESS_EGRESS_OPTIONS="ether-vlan"
######################################
# JESUS CHRIST IS THE LORD! AMEN #
######################################
# Qosify
# Clear default config
uci del qosify.@defaults[0].dscp_prio
uci del qosify.@defaults[0].prio_max_avg_pkt_len
uci del qosify.besteffort
uci del qosify.bulk
uci del qosify.video
uci del qosify.voice
# ICMP (ping) and default class for "unmarked traffic"
uci set qosify.@defaults[0].dscp_icmp="+besteffort"
uci set qosify.@defaults[0].dscp_default_tcp="unmarked_traffic"
uci set qosify.@defaults[0].dscp_default_udp="unmarked_traffic"
# unmarked_traffic class
uci set qosify.unmarked_traffic=class
uci set qosify.unmarked_traffic.ingress="CS1"
uci set qosify.unmarked_traffic.egress="CS1"
uci set qosify.unmarked_traffic.prio_max_avg_pkt_len="1256"
uci set qosify.unmarked_traffic.dscp_prio="CS4"
uci set qosify.unmarked_traffic.bulk_trigger_pps="250"
uci set qosify.unmarked_traffic.bulk_trigger_timeout="10"
uci set qosify.unmarked_traffic.dscp_bulk="CS1"
# browsing class
uci set qosify.browsing=class
uci set qosify.browsing.ingress="CS0"
uci set qosify.browsing.egress="CS0"
uci set qosify.browsing.prio_max_avg_pkt_len="575"
uci set qosify.browsing.dscp_prio="AF41"
uci set qosify.browsing.bulk_trigger_pps="1000"
uci set qosify.browsing.bulk_trigger_timeout="10"
uci set qosify.browsing.dscp_bulk="CS1"
# bulk class
uci set qosify.bulk=class
uci set qosify.bulk.ingress="CS1"
uci set qosify.bulk.egress="CS1"
# besteffort class
uci set qosify.besteffort=class
uci set qosify.besteffort.ingress="CS0"
uci set qosify.besteffort.egress="CS0"
# network_services class
uci set qosify.network_services=class
uci set qosify.network_services.ingress="CS2"
uci set qosify.network_services.egress="CS2"
# broadcast_video class
uci set qosify.broadcast_video=class
uci set qosify.broadcast_video.ingress="CS3"
uci set qosify.broadcast_video.egress="CS3"
# gaming class
uci set qosify.gaming=class
uci set qosify.gaming.ingress="CS4"
uci set qosify.gaming.egress="CS4"
# multimedia_conferencing class
uci set qosify.multimedia_conferencing=class
uci set qosify.multimedia_conferencing.ingress="AF42"
uci set qosify.multimedia_conferencing.egress="AF42"
uci set qosify.multimedia_conferencing.prio_max_avg_pkt_len="575"
uci set qosify.multimedia_conferencing.dscp_prio="AF41"
# telephony class
uci set qosify.telephony=class
uci set qosify.telephony.ingress="EF"
uci set qosify.telephony.egress="EF"
# Reorder classes
uci reorder qosify.unmarked_traffic=1
uci reorder qosify.browsing=2
uci reorder qosify.bulk=3
uci reorder qosify.besteffort=4
uci reorder qosify.network_services=5
uci reorder qosify.broadcast_video=6
uci reorder qosify.gaming=7
uci reorder qosify.multimedia_conferencing=8
uci reorder qosify.telephony=9
# CAKE
uci set qosify.wan.name="wan"
uci set qosify.wan.disabled="0"
uci set qosify.wan.bandwidth_up="$BANDWIDTH_UP"
uci set qosify.wan.bandwidth_down="$BANDWIDTH_DOWN"
uci set qosify.wan.overhead_type="$OVERHEAD_TYPE"
uci set qosify.wan.autorate_ingress="0"
uci set qosify.wan.ingress_options="$INGRESS_OPTIONS"
uci set qosify.wan.egress_options="$EGRESS_OPTIONS"
uci set qosify.wan.options="$INGRESS_EGRESS_OPTIONS"
# Rules
cat << "EOF" > /etc/qosify/00-defaults.conf
# SSH
tcp:22 network_services
# NTP
udp:123 network_services
# DNS
tcp:53 network_services
tcp:5353 network_services
udp:53 network_services
udp:5353 network_services
# DNS over TLS (DoT)
tcp:853 multimedia_conferencing
udp:853 multimedia_conferencing
# HTTP/HTTPS/QUIC
tcp:80 browsing
tcp:443 browsing
udp:80 browsing
udp:443 browsing
# Microsoft (Download)
dns:*1drv* bulk
dns:*backblaze* bulk
dns:*backblazeb2* bulk
dns:*ms-acdc.office* bulk
dns:*onedrive* bulk
dns:*sharepoint* bulk
dns:*update.microsoft* bulk
dns:*windowsupdate* bulk
# MEGA (Download)
dns:*mega* bulk
# Dropbox (Download)
dns:*dropboxusercontent* bulk
# Google (Download)
dns:*drive.google* bulk
dns:*googleusercontent* bulk
# Steam (Download)
dns:*steamcontent* bulk
# Epic Games (Download)
dns:*download.epicgames* bulk
dns:*download2.epicgames* bulk
dns:*download3.epicgames* bulk
dns:*download4.epicgames* bulk
dns:*epicgames-download1* bulk
# YouTube
dns:*googlevideo* besteffort
# Facebook
dns:*fbcdn* besteffort
# Twitch
dns:*ttvnw* besteffort
# TikTok
dns:*tiktok* besteffort
# Netflix
dns:*nflxvideo* besteffort
# Amazon Prime Video
dns:*aiv-cdn* besteffort
dns:*aiv-delivery* besteffort
dns:*pv-cdn* besteffort
# Disney Plus
dns:*disney* besteffort
dns:*dssott* besteffort
# HBO
dns:*hbo* besteffort
dns:*hbomaxcdn* besteffort
# BitTorrent
tcp:6881-7000 bulk
tcp:51413 bulk
udp:6771 bulk
udp:6881-7000 bulk
udp:51413 bulk
# Usenet
tcp:119 bulk
tcp:563 bulk
# Live Streaming to YouTube Live, Twitch, Vimeo and LinkedIn Live
tcp:1935-1936 broadcast_video
tcp:2396 broadcast_video
tcp:2935 broadcast_video
# Xbox
tcp:3074 gaming
udp:88 gaming
#udp:500 gaming # UDP port already used in "VoWiFi" rules
udp:3074 gaming
udp:3544 gaming
#udp:4500 gaming # UDP port already used in "VoWiFi" rules
# PlayStation
tcp:3478-3480 gaming
#udp:3478-3479 gaming # UDP ports already used in "Zoom" rules
# Call of Duty
#tcp:3074 gaming # TCP port already used in "Xbox" rules
tcp:3075-3076 gaming
#udp:3074 gaming # UDP port already used in "Xbox" rules
udp:3075-3079 gaming
udp:3658 gaming
# FIFA
tcp:3659 gaming
udp:3659 gaming
# Minecraft
tcp:25565 gaming
udp:19132-19133 gaming
udp:25565 gaming
# Supercell Games
tcp:9339 gaming
udp:9339 gaming
# Zoom, Microsoft Teams, Skype and FaceTime (they use these same ports)
udp:3478-3497 multimedia_conferencing
# Zoom
dns:*zoom* multimedia_conferencing
tcp:8801-8802 multimedia_conferencing
udp:8801-8810 multimedia_conferencing
# Skype
dns:*skype* multimedia_conferencing
# FaceTime
udp:16384-16387 multimedia_conferencing
udp:16393-16402 multimedia_conferencing
# GoToMeeting
udp:1853 multimedia_conferencing
udp:8200 multimedia_conferencing
# Webex Meeting
tcp:5004 multimedia_conferencing
udp:9000 multimedia_conferencing
# Jitsi Meet
tcp:5349 multimedia_conferencing
udp:10000 multimedia_conferencing
# Google Meet
udp:19302-19309 multimedia_conferencing
# TeamViewer
tcp:5938 multimedia_conferencing
udp:5938 multimedia_conferencing
# Voice over Internet Protocol (VoIP)
tcp:5060-5061 telephony
udp:5060-5061 telephony
# Voice over WiFi or WiFi Calling (VoWiFi)
udp:500 telephony
udp:4500 telephony
EOF
# Saving modified values
uci commit qosify
/etc/init.d/qosify restart
These keywords in CAKE have been provided to represent a number of common link technologies:
Write the keyword below
***********************
"conservative" Failsafe (overhead 48 atm)
"ethernet" Ethernet (overhead 38 mpu 84 noatm)
"docsis" Cable Modem (overhead 18 mpu 64 noatm)
"pppoe-ptm" VDSL2 (overhead 30 ptm)
"bridged-ptm" VDSL2 (overhead 22 ptm)
"pppoa-vcmux" ADSL (overhead 10 atm)
"pppoa-llc" ADSL (overhead 14 atm)
"pppoe-vcmux" ADSL (overhead 32 atm)
"pppoe-llcsnap" ADSL (overhead 40 atm)
"bridged-vcmux" ADSL (overhead 24 atm)
"bridged-llcsnap" ADSL (overhead 32 atm)
"ipoa-vcmux" ADSL (overhead 8 atm)
"ipoa-llcsnap" ADSL (overhead 16 atm)
# If you are unsure, then write "conservative" as a general safe value.
# "raw" keyword turns off all overhead compensation in CAKE.
"ether-vlan" keyword information:
"ether-vlan" adds '4 bytes' to the overhead.
- This keyword is not for standalone use, but act as a modifier to some previous presets or keywords.
- In addition to those previous presets or keywords it is common to have VLAN tags (4 extra bytes) or PPPoE encapsulation (8 extra bytes).
# This keyword "ether-vlan" may be repeated as necessary.
For true ATM links (ADSL), one often can measure the real per-packet overhead empirically, see for further information how to do that:
no worries, i thought you know something which under the hood silently ignores configuration.
now, i have a question: if multiple interface configuration is supported it looks there is no way tp specify config class and <port, ip, dns> rules per interface. is this correct? e.g. /etc/qosify/*.conf files rules are applied to all traffic on all configured interfaces?
The only way this would work, if you have multiples of internet connections (WANs) like an Cable or DSL line and then you add the "docisiscable" info from the interface you added by adding an VLAN and Interface name like so . Just remember. i wont recommend multiples of interfaces on 3G/4G5G type of connections, as they can differ in speeds the more users are using the phone masts.
Note; Cable/DOCSIS users that are going to use any type of QoS!!
most of the issues that you may experience with DOCSIS/Cable are causes by the shared access infrastructure. If youre in an overloaded CMTS cluster there's not much you can do. Of course you can have a small cable cluster that let's you always hit your full bandwidth no problem. Then you're lucky, but that won't happen often. And That's the nature of it, DOCSIS/Cable was never was designed to transmit individual users internet data.
Neither was phone wire, but with VDSL you only have shared core network, which rarely is a problem. With your dedicated twisted pair you are better off when it comes to stability and reliability of the internet access
# 63/19 plan
config interface wan
option name wan
``` option bandwidth_down 60mbit
option bandwidth_up 17mbit
option overhead_type pppoe-ptm
# 50/15 plan
config interface docisiscable
option name docsiscable
` option bandwidth_down 48mbit
option bandwidth_up 13mbit
option overhead_type docsis
``
Funny, here I thought that DOCSIS (Data Over Cable Service Interface Specification) was specifically designed for that purpose... You can even configure a segment such that all users have a guaranteed access rate... But it turns out that this rate is rather low, and most users' access links are idling most of the time anyhow. That means that provisioning each user more than her/his guaranteed share will result in users being better of most of the time. It now becomes the challenge for the ISP to make sure that this oversubscription of the link does not result in noticeable effects for the users, if the ISP manages to do that, over subscription becomes a valid and efficient way to share a scarce resource.
In most cases the uplink of the DSLAM/MSAN is a similar bottleneck as the aggregate segment-rate is in DOCSIS. Many DSLAM with (low) hundreds of users with 100 Mbps plans are still uplinked with single or double 1Gbps connections, giving similar oversubscription as in DOCSIS networks. However it should be considerably simpler/cheaper to exchange a 1Gbps fiber uplink with e.g. a 10 Gbps simply by exchanging the "optics" at both sides of the link, compared to DOCSIS where either segment bandwidth increases or segment splits are required both of which are more involved than replacing two optic modules. (Sure replacing optics only works if the devices the optics attach to already support higher rates, but for halfway modern DSL gear that appears to be the case).
(note: it will be long and total misunderstanding from my end, if it is stupid please be a noble soul and just ignore or point out why it is not working like this, and not why am stupid. thanks.)
according to link qosify a) is not only supporting wan and b) not only supporting a single interface.
basically qosify does two things:
sitting on top of tc, configuring tc, but tc provides the queuing mechanism (not qosify directly)
there is a rule set system which translates rules into actual traffic marking (with hooks to dns etc.) and a specific logic how for example port rules are working.
let's use this notation, according to #1 qosify can be configured on any ( ! ) interface, it is not limited to wan, so let's use this generalized topology:
[ host0 ] - [ network0 ] --- if0 -[ router w/ qosify on if0 ]- if1 --- [ network1 ] - [ host1 ]
where ifX is interface of router; if0 is the interface qosify applied on
where networkX is a network
where hostX is member of corresponding networkX
where host0 & host1 talking to each other
according to link the specific logic how qosify marks traffic is the following:
* cosify is applied to if0
* rule: tcp:5555 video
case1: in case of outgoing traffic from host1 via if0 (where qosify is applied) to host0:
host1:<port> -----> host0:5555 such traffic is marked as video
host1:<port> -----> host0:6666 such traffic is not marked as video as there is no rule
host1:5555 -----> host0:6666 such traffic is not marked as video as traffic is matched against destination port
case2: in case of incoming traffic from host0 via if0 (where qosify is applied) to host1:
host1:<port> <----- host0:5555 such traffic is marked as video as in case of ingress traffic source port match is used
host1:<port> <----- host0:6666 such traffic is not marked as video due to no source port match
host1:5555 <----- host0:6666 such traffic is not marked as video due to no source port match
so basically qosify always matching against the remote end.
the problem I see that there are traffic patterns where I only know/control the near end, and know nothing about the remote end prior connection so no rule can be defined. Typically VoIP, torrent is such kind of traffic. For example contrary to the VoIP configs flying around here, VoIP does not work the way some people thinks: defining the standard SIP signalling ports 5060/5061 will not help to have good call quality as the media is traveling in RTP using different ports and logic. And even if they know this and add rule for RTP, it will fail. Because usually RTP using dynamically picked port from a port range and depending on your vendor/setup it can vary by each end point, so you don't know what the other end will use. A typical voip call is a duplex steam meaning there are two streams: one from A to B, and another one from B to A. During call setup you advertise your end, i.e the port you expect the other end to connect to, which will be the destination port in case2 above and not the source port hence qosify will not mark your traffic with voice! Symmetric RTP port usage is very optional, and even so, if you really want direct media with anybody you'll have to capture any port 0:65535 ... which obviously makes no sense/and breaks qosify fine grained rule sets as will overlap with something else for sure and it is not supported. Similar problem if you use random listening port for torrent, you can only define your end and not the peer ports, so the configs flying around are neither correct because they assume all the other peers you meet will only use some handful of predefined ports.
obviously i can mark my traffic outside of qosify and just let qosify configure tc. or ... i might able to use qosify! after all it is not hardwired to only usable with single interface (by default the wan), so let's add a second interface configuration, apply qosify both on if0 and if1 as well ( ! ) on if1 just set for egress direction though. in this way I'd have this topology:
[ host0 ] - [ network0 ] --- if0 -[ router w/ qosify on if0 ]- if1 --- [ network1 ] - [ host1 ]
[ qosify on if1.egress ]
meaning two sets of tc diffserv queues, each assigned to if0 and if1 respectively
host1:<port> -----> host0:5555 such traffic is marked as video by if0 queue
host1:<port> -----> host0:6666 such traffic is not marked
host1:<port> <----- host0:5555 such traffic is marked as video as in case of ingress traffic source port match is used
host1:<port> <----- host0:6666 such traffic is not marked as video due to no source port match
host1:5555 <----- host0:<port> such traffic is marked as video by if1 queuing !
this way i could mimic near end based matching!
the problem though (apart it may not work at all, double queuing, cpu impact etc) that due to common rule set even if i mean a large port range only for if1 ingress, it will capture traffic on if0 too. hence it would be nice to extend qosify's syntax to override direction and or specify ip:port not just port imho.
I'm testing openwrt inside an LXC container in proxmox. Everything seems to work fine but Qosify and it seems that the problem is that it can't write into the /sys/fs/bpf folder. I know it's not qosify fault but does anybody know a way to make it work? Using privileged containers with all permissions seems to not work.
libbpf: failed to mkdir /sys/fs/bpf/qosify_data/config: Permission denied
libbpf: map 'config': failed to auto-pin at '/sys/fs/bpf/qosify_data/config': -13
libbpf: map 'config': failed to create: Permission denied(-13)
libbpf: failed to load object '/lib/bpf/qosify-bpf.o'
bpf_object__load: Permission denied
Thanks but the performance isn't even comparable. I tried SQM and cake and it didn't even use 3% of CPU, while in a KVM it uses near 30%.
For now i will use it in a VM but if i could make it to work in LXC i will use it, or at least try. As i said the only thing that does not work is qosify.
Are you sure that in one condition SQM really took 27%-age points more CPU cycles? Or could it be that in the container case the CPU cycles might simply not have been accounted to the container? Mind you I know little about containers so could be wrong, but the cycles SQM needs to do its work need to show up somewhere (possible that 3% is a realistic number).
Didn't make so many test but the global CPU usage from the host machine didn't increase that much. I don't know if the CPU usage was made by SQM or not. That CPU usage could be because the VM emulation it self, the drivers from the network (i used virtio)... If i'm not wrong LXC has direct acces to the network you atach so there's less overhead.
(possible that 3% is a realistic number).