Qosify: new package for DSCP marking + cake

I note that aside from giving people fun knobs to twiddle, I remain dubious as to qosify's actual value. Does anyone have a repeatable test demonstrating it to be:

A) doing something measurably useful remarking packets at the ISP link then transiting over wifi?

B) improving VOIP MOS

Etc?

Value:

  • Sets up CAKE qdiscs as requested
  • Allows easy marking of inbound and outbound traffic based on remote IP or port
  • Allows DSCP marks based on DNS names, filling the void left since dnsmasq 2.87 has not been released with nftables support yet
  • Hopefully works faster as a eBPF module than traditional iptables rules for marking traffic

What tests are needed beyond those tests that prove CAKE works as advertised?

1 Like

Some application that actually shows a benefit from the dscp marks generated vs a vs just plain cake fq.

So more of a question of the value of diffserv{n} versus the value of qosify? Interesting. You might crush the souls of a whole generation of tweakers if besteffort is the answer! (or is 42 the answer?)

2 Likes

Reading it implicitly -- you're saying WMM is useless?

lol -- this is a big claim. We're getting into "x"-truther territory here. Like someone in the above post, it could be aptly named "best effort truther" or "rtt truther".

there are no "claims" made here, but I am certainly in search of the truth. My goal is to find a test (or tests) multiple parties can run that reliably demonstrates what qosify can do to improve traffic behaviors. Over here is a really excellent test and plotting script that could also be applied to the isp link or to wifi.

irtt client --dscp=0xfe -i3ms -d5m an_irtt_server

In my cloud I presently have irtt and flent servers in toronto, sydney, singapore, atlanta, dallas, fremont, de (germany), and london on the .starlink.taht.net subdomain.

You can rewrite the dscp via qosify (irtt listens and transmits on udp and udp6 port 2112), and run other traffic against it.

I had stated somewhere on this thread that I think the biggest benefit to be had is setting the dscp on the wifi client to land in the VI queue (CS5 on linux). I certainly theorize that the "sparse station optimization" will largely take care of getting out packets rapidly to the client from the AP, but I am willing to be proven wrong with data. I could assert another point, re wifi, in that it is so fundamentally unreliable that diffserv markings or no, unless we make it more fundamentally reliable, most of the "missed shot" problem should be pointed at the wifi stack and no amount of twiddling dscps will help.

Given the hell we have been going through to just get the BE queue right in wifi:

it would actually be helpful if more folk were beating up the other wifi queues in whatever manner they like. Notably, stadia or an equivalent over the wifi VI queue would be good.

Another interesting test is to see if various dscp markings survive transit across the internet. (usually only ecn does)

I'm also on record as thinking that "background" is the most useful (in theory) diffserv class.

3 Likes

Be careful what you ask ;). There was a time when my macbook when using AC_VI or AC_VO in a bidirectional throughput test was essentially starving the AP for tx-ops*. Which makes me think one should first test how well up/down sharing works if a client is set to use AC_VI or AC_VO for a significant portion of its traffic.

This was within a RRUL test, so all 4 ACs likely where involved, so things might be different if just AC_VI is used, but I still would recommend to first confirm that that works...
Side-note: I still think the AP should monitor air time usage and adaptively change which AC it uses per default not to be starved by stations.

I strongly agree that actually testing what happens if people go crazy with the VI queue would be good!

1 Like

hi,

i am far from being expert on this subject, but my feeling is that there are different use cases vs expectation what qosify can or cannot do. so maybe first we should agree what qosify (or other similar tools) are aiming for, e.g.

  • is it a plugable "algorithm" for network devices to act as general improvement to reduce latency, bufferbloat, do qos/aqm,
  • is it meant to improve wifi traffic,
  • is it meant to help with custom prioritization and improve fairness in regards of internet traffic,
  • etc etc

there are many use cases and what each tool can cover should be set first then can compare against other similar tools in my opinion.

if you ask what qosify value is than maybe it is not the tool you looking for according to your test scenarios, maybe it is, i don't know.

But "typically*" based on the many question in this site people looking for solution for three kind of problems:

  1. reduce internet latency - by people who are using online games. they playing with few games with well defined traffic patterns (kind of) but suffering from latency.
  2. fair internet access sharing with priority - people/households working from home and need to share a (limited) internet access among work traffic, video conferences, online school, movie/music streaming service, which services running in the same time parallel. they probably want to prioritize work traffic daytime and then movie streaming at night.
  3. crowded wifi - many wifi clients connecting to very few to single AP, which causing wifi slowness in general.

(*) am sure there are many other uses case, it is not a comprehensive but a very subjective list based on what I feel as the questions recurring all the time.

Also, there is a huge expectation difference between very technical people (like you, you are talking about VI queue) and average joe (like me) in regards how easy the tool should be to use. people asking when qosify will get a luci menu for example.

for me the value in qosify is for example the very easy (**) setup and the way how it can connect/mark egress and ingress traffic for the same flow.

(**) it is relatively easy, the kind of reverse logic (remote ip + remote port) tends to confuse people, as they create very sophisticated but wrong configs for their local listening ports ...

so for me qosify is a configuration tool for custom packet marking to control internet facing traffic (***). maybe it is not the primary purpose, not sure.

(***) wish it would add features like local-ness config schema (local ip/ports) to better control traffic when only my near end is known but remote end is not (e.g. VoIP).

my 2 cents. if reached so far thanks for reading :slight_smile:

1 Like

Very simple thing:

Watching netflix with highest Qualifying during download a game without Client Limitation. Daily Business here

2 Likes

I like your deeply philosophical approach. I'm going to use an example of where too many knobs doesn't help. Recently I visited a ubnt site which has both "smart queues" and per device or IP rate limiting (which also uses htb + fq_codel) They had about 30Mbit down, shared among 70+ people and devices, and what they did was rate limit each to 8 mbits down, 300kbits up, and not set an overall limit at all.

They crippled everyone's performance all the time. They'd missed the smart queues setting entirely.

By setting the overall "smart queues" limit to the the correct value, the whole network improved by a lot, even without removing the individual limiters.

Another example like that is a very popular demo on making netduma's anti-bufferbloat and gaming "work" on youtube. The author experienced no benefit from the individual fancy gaming knobs, and a great deal of benefit from getting the overall knob, right (and didn't realize that was what he'd done - it's like, right there, in the video)

So from a UI perspective I prefer users be led to the knob that works, and not distracted by all the other stuff that may or may not accomplish anything.

There are other broad mis-perceptions. For example, DASH traffic is always starting up two flows to measure if it can go up or down, and allocating 20Mbit to netflix is possibly not enough if you want 4k video, because of this probing behavior.

3 Likes

However, qosify, by virtue of using cake's diffserv modes, can not actually allocate fixed bandwidth, the best you can do, is if one of the higher diffserv modes rate suffices you can mark only the netflix packets into that tin*, but that will be partly counter-acted by cake's overflow behavior for priority tins, instead of hard capping the tins, they will overflow into lower priority tins (all the way down to Best Effort, but not Bulk) somewhat counteracting the approach of moving sufficiently greedy flows into the higher priority tins.
For all its lack of bells and whistles, the HTB+fq_codel based simple.qos can be modified to change the guaranteed rates of the different priority tiers (but it painfully lacks cake's excellent per-IP-fairness mode).

*) I am really not a fan of deep marking/prioritization hierarchies that try to handle every eventuality but are based on heuristics; heck I do not even like the behavioral "prioritize small packet flows" de-prioritize bulk flows above a certain volume/throughput very much, they are better than just using external IP addresses and ports, but I see no reason why say a big security update (for a 0-day exploit) for my home-server would be less important than some video streaming happening at the same time. But I digress.

2 Likes

hello dave for having compared netduma and OpenWrt during long years, I can assure you that netduma except its geo filter and well below OpenWrt in term of qos and hierarchization, it is only my own opinion, my youtube gameplays with OpenWrt are quite simply fluid whereas with netduma not!

the bufferbloat results are more or less the same but for nothing in the world I would truncate my router against a netduma :wink:

Translated with www.DeepL.com/Translator (free version)

yes, there are many different use cases scale from single user to (commercial scale of) 70+ users, (promised) features vs real life experience, generally working use cases vs under performing specific corner cases.

as qosify is

if the expectation was that qosify will do something totally new in traffic shaping, probably i could say it is not the case. on the other hand as a configuration and marking layer on top of cake it is performing very well in my opinion. basically this was my point only.

but your point about too many knobs is very valid, though the other way around is neither good. depending on developers aspiration, i.e. to cover all/many different use cases generally good, or a few case but best way, will drive the number of knobs i think.

e.g. hard to satisfy both below in the same way

and

so i'd love to see a tool which has generic "best practice" profiles and support of advanced mode configuration if for whatever reason need tailored setup. and this can be also done with qosify.

ok, am not the author of qosify, i don't have to / want to defend it on behalf. this tool does it what it does. :slight_smile:

so, to answer your question if qosify measurably improve voice MOS the answer: the same way as cake does.

thanks.

Oh, I agree that qosify is a nifty tool for doing something that was/is traditionally hard (DSCP (re-)mapping before a qdisc on an IFB gets hold of packets). That it also learned to configure cake by itself is very convenient (and a sign that the idea of including "lessons learned from the SQM shell scriptery" into cake proper worked, ignoring for a second the feature cake acquired on top of what was easily possible in shell). Qosify is great, my "rant" was more about not going overboard in remarking rules just because qosify makes it easy. Or more precise even though qosify makes it effortless, still go and test whether a given rule a) works as intended, b) test whether the rule is required for "good enough" performance.

I think the developer should, in grand unix tradition, "offer enough rope for the use to shoot themself into the foot", or put different it is great of a tool like qosify makes even the "unreasonable" possible, users however might consider "spending" that with some self restraint.

+1; qosify with a restraint default policy might be a decent starting point (not having tested qosify, I will assume that the default police is already good).

Intersting. I would expect that qosify will make it easier/possible at all for mere mortals to exploit cake's different priority tiers if an environment has large enough issues that MOS scores are bad with cake defaults (e.g. on a very narrow link, where flow-fairness is not enough, as VoIP might actually need an unfair advantage to be usable, and if not VoIP maybe VC).

this is what qosify can do according to author: https://git.openwrt.org/?p=project/qosify.git;a=blob;f=README

probably it can improve situation if you can identify your voice traffic in distinct way using the features qosify offers.

(though i think VoIP is a slippery beast as it is a duplex communication, dynamic in nature, you usually know about your local end but not the remote end and signalling and bearer traffic has totally different characteristics ... hard to grab. i've seen so many problems related how networking was setup underneath causing voice service issues. it is hell hard to identify and fix rare, exceptional cases if otherwise voice quality is acceptible but these rare cases are too visible.)

anyhow, you must try the pudding to tell if tastes you or not :slight_smile:

I would naively guess that qosify is well prepared for VoIP detection heuristics, either by average packet size or by the IP address of the VoIP server (assuming VoIP centralized by an ISP, for true peer to peer calls I agree, somewhat hard to pin down).

i guess so there is prio_max_avg_pkt_len option, if you know your voip codec this can be calculated (or use some reasonable value).

I would recommend to just capture a VoIP conversation on the router and simply look at the packet size for the VoIP packets... However such a rule, will boost all small enough packets, which IMHO is not ideal. Is it possible in qosify to tie two rules together like "size <= X bytes AND remote-port = XXX" ? Because IMHO for classification to do the right thing it needs to have high hit and low miss rate. In the size only case such a rule likely would expedite all pure ACK packets, which IMHO is not the right thing to do, ACKs should ride in the same priority tier as their data packets (cake/fq_codel will by default already give them a small boost if they are sparse enough).

oh, the beauty of voip: UDP, signalling vs bearer traffic, NAT'ed networks ... (*)

my understanding is the following, which can be totally wrong though:
there are two layers qosify works with.

  1. can define classes based on traffic characteristics (like the mentioned average packet length metric), and rules to control overflows to other class (either denote traffic into "bulk" or promote to "priority" queue). so this layer defines the class management logic.
  2. can define marking rules, so based on remote IP, port, DNS match qosify will mark traffic accordingly and assign the flow to corresponding class.

as far as i understand cannot tie rules as you say, only to the extent if based on a marking rule a traffic flow is assigned to a class, in which based on the queue logic it may overflow. but as i understand cannot create a rule like "if port=X and packet size=Y then classify(voice) else classify(besteffort)" for example.

for example this is the default setup:

Summary
# /etc/config/qosify
config defaults
        list defaults /etc/qosify/*.conf
        option dscp_prio video
        option dscp_icmp +besteffort
        option dscp_default_udp besteffort
        option prio_max_avg_pkt_len 500

config class besteffort
        option ingress CS0
        option egress CS0

config class bulk
        option ingress LE
        option egress LE

config class video
        option ingress AF41
        option egress AF41

config class voice
        option ingress CS6
        option egress CS6
        option bulk_trigger_pps 100
        option bulk_trigger_timeout 5
        option dscp_bulk CS0
[..]

# /etc/qosify/00-defaults.conf
# DNS
tcp:53          voice
tcp:5353        voice
udp:53          voice
udp:5353        voice

# NTP
udp:123         voice

# SSH
tcp:22          +video

# HTTP/QUIC
tcp:80          +besteffort
tcp:443         +besteffort
udp:80          +besteffort
udp:443         +besteffort

first config file defines the classes, the second the marking rules. and as you can see in voice class has a protection if a flow is abusing it with high (maybe burst) pps traffic. but i am not aware of any granularity you're asking for.

(*) why voip is slippery:

Summary
  • identifying voip traffic is not that easy just to say remote port X. voip protocols by design specify "where am listing" and there are no per se "standard" voip ports. there are vendor specific "typical" port ranges but that's not stone written either. plus the NATing (unless you want to open udp port 16384-32768 which is usually used by Cisco for example, but other vendors using totally different port ranges). so if you really want to talk with anybody peer to peer you may end up open all 65k ports at the end - but that's a bad idea so rather use STUN/TURN/ICE. if you have a telco service you are in better position as you'll will connect to telco only so you know - hopefully - all needed information.
  • and yes you are right saying that all small packets (e.g. size < 500 byte) are voice not enough either.
  • voip is two (three) things actually: signalling and bearer traffic (and control traffic RTCP), the bearer traffic is the most important one, that's the real time bit, but i keep seeing shared configs for the default SIP 5060 which is only the signalling piece and not that sensitive. also while RTP is small (e.g. G.711 64kbps codec, so a 20ms sample is just 160Byte payload), the signalling piece, in case of SIP which is very chatty, can be well over 1500Byte.
2 Likes