Qosify: new package for DSCP marking + cake

[moeller0]

@Lynx

I mean who wouldn't want Teams or Zoom to be prioritized in this way?

Honestly, I do not even bother to guess. I would first test whether the 'normal' fairness modes, per-flow, and cake's speciality per-internal-IP-per-flow, are not already doing the right thing, beyond that we are talking about policy decisions each network admin needs to take individually.
The problem is, everybody intuitively knows which of the concurrently running applications they consider most important, but a router does not. So the challenge is to find heuristics that result in few false positives....

@Lynx

Is there a good standard defaults template for Qosify that handles such routine prioritisations? Something that will just work for most users?

Only if you drank the QoS koolaid.... prioritization is a zero-sum game and hence not free of potentially negative side effects, so IMHO prioritization should reluctantly and judiciously. This approach really does not rhyme with 'default prioritization rules' very well.

Looking at the specific rules it clearly contains carry over from before fq_codel cake, like the whole DNS in Voice stuff. I would always try to first check whether these rules are necessary and only use rules I have confirmed as actually helping interactivity. It is far too easy to built an elaborate QoS hierarchy, forget the details and then puzzle over unexpected side effects....

@Lynx

If my understanding is correct, these fun and games with classifications means that if packets are going to be dropped, then prioritize dropping stuff like Netflix or Prime rather than latency sensitive stuff?

No, in cake packets get dropped if they stayed too long in a queue, priority affects in which order the different flows with queued-up packets are served, so higher priority tins have a higher probability of being served (but they are rate limited so Voice will never fully starve the other tins). Also in cake traffic overflowing Voice will not be dropped but rather sent inside one of the other tins.

[Lynx]

As ever, very insightful. Thank you very much indeed.

Another question springs to mind: for the many of us that use WireGuard and apply CAKE on 'veth-lan' for download and on 'wan' for upload (using the flows keyword to give CAKE some visibility despite encryption) - is Qosify compatible with this setup?

For anyone wondering, VPN and SQM is not always totally insane. Vodafone UK selectively throttle 4G based on port number. VPN circumvents that. But bufferbloat in LTE remains a challenge because the ping can spike significantly and the capacity varies with time. Despite this I have found VPN plus CAKE (with autorate script) to work well together on my LTE connection.

[dMopp]

Set option autorate_ingress to 0 and use bandwidth_down 280mbit. And not sure which port will be used for speedtest, but i guess 443. At least it should NOT land in bulk while testing, thats important.

[francisuk1989]

@Lynx

for the many of us that use WireGuard and apply CAKE on 'veth-lan'

You could give the interface like so

config interface wg0
	option name wg0

But using a Mobile / Cell phone 4G/5G connection would be a bad idea due to the mast been used by others and you dont know hoe fast the other users are using, Also keep in mind that EE will give prioritization to ESN.

Also keep in mind that, ISPs will move over to VoIP/Digital Voice so ports 5060/5061 needed (BT and Zen) are doing Digital Voice at the moment.

Check @elan config here

Qosify: new package for DSCP marking + cake

I found this information about packet length size to prioritize non-bulk unmarked traffic like games, VoIP, etc. and added it to my config and wanted to share it with you. My QoS rules for gaming, browsing/streaming, torrents and bufferbloat - Gargoyle Forum Clarification If you have a bad or unstable connection even using CAKE (because your ISP is bad), you will still have a bad connection even if you use DSCP marking. The DSCP marking doesn't help you FIX the bufferbloat, it's CAKE that …

@elan

Use Lag Report to know which ISP offers the best connection and the lowest ping where you live:

Keep in mind, Business ISPs will always have higher prioritization, For example

[nbd]

I pushed a fix for the glibc build failure and also updated qosify in master to replace the dnsmasq hook with new DNS snooping code that should work with all DNS servers. I also changed the code to consider the order in which DNS rules are declared when matching. The dscp value of the first matching rule is used now. It is now also possible to define rules that only match on CNAME

[Lynx]

So to elaborate, for download I have a mixture of VPN (in an unencrypted state) and non-VPN traffic hit 'veth-lan' on which I have CAKE applied. Upload traffic is managed on 'wan' and again this is a mixture of VPN (in an encrypted state) and non-VPN traffic, and amazingly despite the encryption CAKE can work with this with the 'flows' keyword because it has a special WireGuard compatibility built-in (working with skb->hash) - see here:

https://lists.bufferbloat.net/pipermail/cake/2020-May/005257.html

Finally, an autorate script handles upload/download bandwidth fluctuation owing to varying capacity on the LTE link.

Here are the CAKE parameters:

qdisc cake 8065: dev wan root refcnt 2 bandwidth 30Mbit besteffort flows 
qdisc cake 804d: dev veth-lan root refcnt 2 bandwidth 25Mbit besteffort triple-isolate nonat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 92

Seems to work well for me with my Vodafone UK 4G connection (albeit getting the autorate right is difficult). VPN circumvents their port-selective throttling, and VPN bypass deals with Netflix/Prime. I see download bandwidth varying between 20-80 Mbit/s and upload bandwidth varying between 25-35 Mbit/s and latency (without bufferbloat is pretty tightly spread around 50ms).

But if possible it would certainly be nice to encourage CAKE to drop Netflix/Prime or Windows update traffic rather than Teams/Zoom traffic during periods of saturation.

So in this scenario (which is adopted by many who want CAKE to handle mixture of encrypted and unencrypted traffic), how would I set Qosify to work with these interfaces?

@nbd can Qosify work in this situation?

[dave14305]

@nbd

It is now also possible to define rules that only match on CNAME

Will a regular dns: rule match against either CNAME or A/AAAA records? Or does a user still need to figure out if their desired domain name is a CNAME or not? Or is dns_c: the only way for qosify to consider the CNAME response? I’m slightly confused by the “only match” aspect.

Thanks for another creative enhancement!

[marshmallow]

Will this work for those of us using solutions such as stubby(DoT)/https-dns-proxy(DoH)? I use the latter, so my DNS server is technically the local 192.168 IP of the router/gateway.

Behind the scenes it would send DNS requests via DNS over HTTPS port 443 to DNS servers such as Google/Cloudflare/Quad9 etc, then it would send the result to whichever device made the query via port 59.

So I wonder if this would interfere with qosify's DNS rules in this scenario or if adjustments are to be made somewhere in order to make them get along.

[Dopam-IT_1987]

i use dns https proxy too normally is work i think but i can say wrong

[echelon]

I am using elan example config :

config defaults
	list defaults /etc/qosify/*.conf
	option dscp_icmp +besteffort
	option dscp_default_tcp unmarked_traffic
	option dscp_default_udp unmarked_traffic

config class unmarked_traffic
	option ingress CS1
	option egress CS1
	option prio_max_avg_pkt_len 1270
	option dscp_prio CS4
	option bulk_trigger_pps 600
	option bulk_trigger_timeout 10
	option dscp_bulk CS1

config class browsing
	option ingress CS0
	option egress CS0
	option prio_max_avg_pkt_len 575
	option dscp_prio AF41
	option bulk_trigger_pps 1000
	option bulk_trigger_timeout 10
	option dscp_bulk CS1

config class bulk
	option ingress CS1
	option egress CS1

config class besteffort
	option ingress CS0
	option egress CS0

config class network_services
	option ingress CS2
	option egress CS2

config class broadcast_video
	option ingress CS3
	option egress CS3

config class gaming
	option ingress CS4
	option egress CS4

config class multimedia_conferencing
	option ingress AF42
	option egress AF42
	option prio_max_avg_pkt_len 575
	option dscp_prio AF41

config class telephony
	option ingress EF
	option egress EF

config interface pppoe-wan
	option name pppoe-wan
	option disabled 0
	option bandwidth_up 180mbit
	option bandwidth_down 180mbit
	option overhead_type docsis
	# defaults:
	option ingress 1
	option egress 1
	option mode diffserv4
	option nat 1
	option host_isolate 1
	option autorate_ingress 0
	option ingress_options ""
	option egress_options "wash"
	option options "ether-vlan"

config device wandev
	option disabled 1
	option name wan
	option bandwidth 100mbit

My LAN is connecting to the Internet through a PPPOE client named "pppoe-wan"
Here is my ip -a :

root@host:/etc/init.d# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:5b brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:5c brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:5d brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:5e brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:5f brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:60 brd ff:ff:ff:ff:ff:ff
8: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default qlen 1000
    link/tunnel6 :: brd :: permaddr 7eb6:f125:7960::
9: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
10: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN group default qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
11: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
12: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
13: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN group default qlen 1000
    link/gre6 :: brd :: permaddr 368b:66b6:96f0::
14: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
    link/void
16: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 40:62:31:0b:fc:5b brd ff:ff:ff:ff:ff:ff
    inet 10.10.8.1/21 brd 10.10.15.255 scope global br-lan
       valid_lft forever preferred_lft forever
17: pppoe-wan: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN group default qlen 3
    link/ppp
    inet 123.123.123.123 peer 123.123.123.1/32 scope global pppoe-wan
       valid_lft forever preferred_lft forever
    inet6 123::123:123:123:123 peer 123::123:123:123:123/128 scope link
       valid_lft forever preferred_lft forever
18: ifb-dns: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 32
    link/ether ba:f4:d5:83:d5:7b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::b8f4:d5ff:fe83:d57b/64 scope link
       valid_lft forever preferred_lft forever

When trying to querying qosify status, I have got a empty reply.

How to properly configuring the interface?

[EDIT]

Changing back interface and option name into "wan" seems working now :

config interface wan
	option name wan

[grrr2]

according to Qosify: new package for DSCP marking + cake - #39 by dave14305 solid IP rules (ie. without +) are the priority, so

dns:cloudflare.com voice
tcp:443 video

this should result voice marking and not video in spite you may connect to cloudflare.com:443 for DNS service (DNS over HTTPS). Also it means any other request to cloudflare.com will be in voice.

[dave14305]

Right now, it’s only designed to listen for unencrypted DNS traffic on port 53. Haven’t tried the new build myself yet, however.

If alternate listening on loopback ports is possible in future releases, this might help solutions that forward on the router using loopback.

[francisuk1989]

Same quesion here.

[kde@arch ~]$ dig stream-mz.planetradio.co.uk

; <<>> DiG 9.18.0 <<>> stream-mz.planetradio.co.uk
;; ANSWER SECTION:
stream-mz.planetradio.co.uk. 232 IN	CNAME	live-bauermz.sharp-stream.com.
live-bauermz.sharp-stream.com. 30 IN	CNAME	edge-bauermz-01-gos2.sharp-stream.com.
edge-bauermz-01-gos2.sharp-stream.com. 30 IN A	91.193.245.152

'
So my guess, is if i put

dns:stream-mz.planetradio.co.uk

resolves automatically to live-bauermz.sharp-stream.com / edge-bauermz-01-gos2.sharp-stream.com

[shm0]

When the WAN speed is equal the LAN Speed e.g.
WAN = 1Gbit
LAN = 1Gbit

It should be possible to the "wan" shaping on the lan interface?
How to do this with qosify?

What settings should be used for flow isolation? nat setting on or off?

Thanks.

[moeller0]

You can always instantiate the download-shaper either on an veth or ifb on the WAN interface or on the egress side of a LAN interface (but that only works if you do not have wifi or significant traffic amounts sourced by the router itself, as neither of these will see the shaper for its internet download traffic).

@shm0

What settings should be used for flow isolation? nat setting on or off?

On the LAN interface the NAT keyword should not be required (but it should still work), for flow isolation you still need to pick what you want, for per-internal-IP-fairness the download shaper will need dual-dsthost independent in which interface it is instantiated.

[shm0]

Well there is no significant traffic sourced by the router itself.
I didn't think about the wireless. Well time to get a separate access point x)

When runninf the ifb method.
What cake settings should be used on the lan interface?
By default openwrt is running fq_codel but i want to replace it with cake.

[moeller0]

@shm0

By default openwrt is running fq_codel but i want to replace it with cake.

Why? Unless you want to move the whole ingress/download shaper there fq_codel appears to be the better all around qdisc. If however you want to save the ~5% performance the IFB or VETH redirection would cost, or want to be able to use simple traffic marking rules based on internal IPs and ports, and instantiate cake as traffic shaper on lan, you probably still want to set 'ingress dual-dsthost nat' and the appropriate per-packet overhead and shaper rate.

[shm0]

5% perfomance, I don't know....
ifb method with fq_codel on lan: ~30%-40% CPU usage
wan egress + lan egress shaping: ~20% CPU usage

[moeller0]

The 5% are admittedly old, but I only saw a 5% higher throughput when replacing ifb-on-wan with native-HTB-fq_codel on LAN under conditions when the throughput was limited by the CPU in both cases. (Using flent's RRUL test to simultaneously saturate bot up and download to really stress the CPU). Well, possible that this cost has changed of time....

How do you asses CPU% here? It turns out the best way is to look at 100-idle% as loads of SQM's cost is accounted as SIRQ.

[shm0]

I just looked at htop CPU Usage :x