This is a really slow link.... 1.75 Mbps is not a lot and hence the delay targets for Voice and Video get pretty large... on such a link you might need to filter everything into Bulk and only let your sensitive traffic stay in Best Effort.... I guess it depends on what you use the link for.
1 Like
Does qosify supports AMD64 x86-64 platform?, when I was trying to compile it error out on cannot found "stdarg.h", when looking at bpf OpenWRT include files, the arch mentioned only to MIPS?.
What I have done is using OpenWRT latest snapshot, configuring the bpf's llvm toolchain to using host (LLVM 13), using glibc.
If it does supports AMD64, how to properly compile it?.
[EDIT]
Hmm it's successfully built on asu.aparcar.org while on my build VM getting "stdarg.h" not found.
My stdarg.h is located in :
/usr/lib/gcc/x86_64-linux-gnu/10/include
/home/echelon/works/openwrt/staging_dir/toolchain-x86_64_gcc-11.2.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/11.2.0/include
1 Like
This looks like you've configured glibc, instead of OpenWrt's default libc (musl), for your x86_64, I wouldn't be surprised at all, if nobody has ever tested compiling qosify with glibc.
2 Likes
I have a question. Without any classification whatsoever will CAKE prioritise Teams or Zoom traffic over Netflix or Prime traffic?
If not, what would be the minimum change needed to ensure this?
1 Like
Unsure, cake/fq_codel will give a small priority boost to flows that do not queue up more than quantum bytes between being serviced IIRC, DASH-style streamers will probably get this boost only for a few initial packets of each burst, while more continuously sending application might get it more often. But the question is will VC software pace out its packet or will it essentially burst out each new "frame" (whether full keyframe or incremental frame) as fast as possible? In the former case it will likely get the sparseness boost more often in the later case less so.
That said, if the dual-xxxhost isolation modes are used and VC and streaming terminates at different internal computers, the host isolation should help making VC robust in the light of dash streaming... (but ingress streaming is always a bit approximate so that depends on how reactive the dash stream is to marks/drops).
In that case you need "targeted" unfairness (if you do not because per-flow fairness or per-host-per-flow is sufficient you are already set), there is nothing the kernel can do without your help, since it will not be able to tell whether you prioritise the streaming over the VC or vice versa. qosify or similar would be your best bet to move VC into a higher (and/or streaming into a lower priority class).
2 Likes
I have noticed today with qosify enabled speed test reports aprox. 170/19 while without it around 280/24. Am I doing something wrong or is there good explanation to it?
My config:
/etc/config/qosify
config defaults
list defaults /etc/qosify/*.conf
option dscp_prio video
option dscp_icmp +besteffort
option dscp_default_udp besteffort
option prio_max_avg_pkt_len 500
config class besteffort
option ingress CS0
option egress CS0
config class bulk
option ingress LE
option egress LE
config class video
option ingress AF41
option egress AF41
config class voice
option ingress CS6
option egress CS6
option bulk_trigger_pps 100
option bulk_trigger_timeout 5
option dscp_bulk CS0
config class work
option ingress AF41
option egress AF41
config interface wan
option name wan
option disabled 0
option bandwidth_up 23mbit
# option bandwidth_down 295mbit
option overhead_type docsis
# defaults:
option ingress 1
option egress 1
option mode diffserv4
option nat 1
option host_isolate 1
option autorate_ingress 1
option ingress_options ""
option egress_options ""
option options ""
config device wandev
option disabled 1
option name wan
option bandwidth 300mbit
/etc/qosify/00-defaults.conf
DNS
tcp:53 voice
tcp:5353 voice
udp:53 voice
udp:5353 voice
NTP
udp:123 voice
SSH
tcp:22 +video
HTTP/QUIC
tcp:80 +besteffort
tcp:443 +besteffort
udp:80 +besteffort
udp:443 +besteffort
VoIP
tcp:5060 voice
tcp:5061 voice
TeamSpeak 3 VoiP
udp:9987 voice
Zoom
udp:8801-8810 +CS6
Bulk
dns:.backblaze.com CS1
dns:.backblazeb2.com CS1
dns:.ms-acdc.office.com CS1
dns:.windowsupdate.com CS1
dns:.update.microsoft.com CS1
dns:.onedrive.com CS1
dns:.1drv.ms CS1
dns:.1drv.com CS1
dns:*.sharepoint.com CS1
Video
dns:.googlevideo.com AF31
dns:.nflxvideo.net AF31
dns:.aiv-cdn.net AF31
dns:.r.cloudfront.net AF31
dns:.aiv-delivery.net AF31
dns:.mediaplatform.com AF31
dns:*.uplynk.com AF31
Voice
dns:.zoom.us CS6
dns:.skype.com CS6
Work
dns:.work1.com AF41
dns:.work2.com AF41
It seems like @perceival's defaults are close to what I think are my requirements (Teams/Zoom to be prioritized over OneDrive or Windows update or Netflix or Prime). But actually isn't this the case for many?
I mean who wouldn't want Teams or Zoom to be prioritized in this way?
Is there a good standard defaults template for Qosify that handles such routine prioritisations? Something that will just work for most users?
If my understanding is correct, these fun and games with classifications means that if packets are going to be dropped, then prioritize dropping stuff like Netflix or Prime rather than latency sensitive stuff? Or is there more to this? Does bandwidth allocation come into the equation? Like allocate more bandwidth to stuff that matters? Or is bandwidth allocation completely unrelated? Will this work on upload and download?
Would the 'tc qdisc change' commands needed for autorate scripts including my own still work?
1 Like
Honestly, I do not even bother to guess. I would first test whether the 'normal' fairness modes, per-flow, and cake's speciality per-internal-IP-per-flow, are not already doing the right thing, beyond that we are talking about policy decisions each network admin needs to take individually.
The problem is, everybody intuitively knows which of the concurrently running applications they consider most important, but a router does not. So the challenge is to find heuristics that result in few false positives....
Only if you drank the QoS koolaid.... prioritization is a zero-sum game and hence not free of potentially negative side effects, so IMHO prioritization should reluctantly and judiciously. This approach really does not rhyme with 'default prioritization rules' very well.
Looking at the specific rules it clearly contains carry over from before fq_codel cake, like the whole DNS in Voice stuff. I would always try to first check whether these rules are necessary and only use rules I have confirmed as actually helping interactivity. It is far too easy to built an elaborate QoS hierarchy, forget the details and then puzzle over unexpected side effects....
No, in cake packets get dropped if they stayed too long in a queue, priority affects in which order the different flows with queued-up packets are served, so higher priority tins have a higher probability of being served (but they are rate limited so Voice will never fully starve the other tins). Also in cake traffic overflowing Voice will not be dropped but rather sent inside one of the other tins.
2 Likes
As ever, very insightful. Thank you very much indeed.
Another question springs to mind: for the many of us that use WireGuard and apply CAKE on 'veth-lan' for download and on 'wan' for upload (using the flows keyword to give CAKE some visibility despite encryption) - is Qosify compatible with this setup?
For anyone wondering, VPN and SQM is not always totally insane. Vodafone UK selectively throttle 4G based on port number. VPN circumvents that. But bufferbloat in LTE remains a challenge because the ping can spike significantly and the capacity varies with time. Despite this I have found VPN plus CAKE (with autorate script) to work well together on my LTE connection.
2 Likes
Set option autorate_ingress to 0 and use bandwidth_down 280mbit. And not sure which port will be used for speedtest, but i guess 443. At least it should NOT land in bulk while testing, thats important.
1 Like
You could give the interface like so
config interface wg0
option name wg0
But using a Mobile / Cell phone 4G/5G connection would be a bad idea due to the mast been used by others and you dont know hoe fast the other users are using, Also keep in mind that EE will give prioritization to ESN.
Also keep in mind that, ISPs will move over to VoIP/Digital Voice so ports 5060/5061 needed (BT and Zen) are doing Digital Voice at the moment.
Check @elan config here
Keep in mind, Business ISPs will always have higher prioritization, For example
2 Likes
I pushed a fix for the glibc build failure and also updated qosify in master to replace the dnsmasq hook with new DNS snooping code that should work with all DNS servers. I also changed the code to consider the order in which DNS rules are declared when matching. The dscp value of the first matching rule is used now. It is now also possible to define rules that only match on CNAME
5 Likes
So to elaborate, for download I have a mixture of VPN (in an unencrypted state) and non-VPN traffic hit 'veth-lan' on which I have CAKE applied. Upload traffic is managed on 'wan' and again this is a mixture of VPN (in an encrypted state) and non-VPN traffic, and amazingly despite the encryption CAKE can work with this with the 'flows' keyword because it has a special WireGuard compatibility built-in (working with skb->hash) - see here:
https://lists.bufferbloat.net/pipermail/cake/2020-May/005257.html
Finally, an autorate script handles upload/download bandwidth fluctuation owing to varying capacity on the LTE link.
Here are the CAKE parameters:
qdisc cake 8065: dev wan root refcnt 2 bandwidth 30Mbit besteffort flows
qdisc cake 804d: dev veth-lan root refcnt 2 bandwidth 25Mbit besteffort triple-isolate nonat wash ingress no-ack-filter split-gso rtt 100ms noatm overhead 92
Seems to work well for me with my Vodafone UK 4G connection (albeit getting the autorate right is difficult). VPN circumvents their port-selective throttling, and VPN bypass deals with Netflix/Prime. I see download bandwidth varying between 20-80 Mbit/s and upload bandwidth varying between 25-35 Mbit/s and latency (without bufferbloat is pretty tightly spread around 50ms).
But if possible it would certainly be nice to encourage CAKE to drop Netflix/Prime or Windows update traffic rather than Teams/Zoom traffic during periods of saturation.
So in this scenario (which is adopted by many who want CAKE to handle mixture of encrypted and unencrypted traffic), how would I set Qosify to work with these interfaces?
@nbd can Qosify work in this situation?
1 Like
Will a regular dns: rule match against either CNAME or A/AAAA records? Or does a user still need to figure out if their desired domain name is a CNAME or not? Or is dns_c: the only way for qosify to consider the CNAME response? I’m slightly confused by the “only match” aspect.
Thanks for another creative enhancement!
2 Likes
Will this work for those of us using solutions such as stubby(DoT)/https-dns-proxy(DoH)? I use the latter, so my DNS server is technically the local 192.168 IP of the router/gateway.
Behind the scenes it would send DNS requests via DNS over HTTPS port 443 to DNS servers such as Google/Cloudflare/Quad9 etc, then it would send the result to whichever device made the query via port 59.
So I wonder if this would interfere with qosify's DNS rules in this scenario or if adjustments are to be made somewhere in order to make them get along.
1 Like
i use dns https proxy too normally is work i think but i can say wrong 
1 Like
I am using elan example config :
config defaults
list defaults /etc/qosify/*.conf
option dscp_icmp +besteffort
option dscp_default_tcp unmarked_traffic
option dscp_default_udp unmarked_traffic
config class unmarked_traffic
option ingress CS1
option egress CS1
option prio_max_avg_pkt_len 1270
option dscp_prio CS4
option bulk_trigger_pps 600
option bulk_trigger_timeout 10
option dscp_bulk CS1
config class browsing
option ingress CS0
option egress CS0
option prio_max_avg_pkt_len 575
option dscp_prio AF41
option bulk_trigger_pps 1000
option bulk_trigger_timeout 10
option dscp_bulk CS1
config class bulk
option ingress CS1
option egress CS1
config class besteffort
option ingress CS0
option egress CS0
config class network_services
option ingress CS2
option egress CS2
config class broadcast_video
option ingress CS3
option egress CS3
config class gaming
option ingress CS4
option egress CS4
config class multimedia_conferencing
option ingress AF42
option egress AF42
option prio_max_avg_pkt_len 575
option dscp_prio AF41
config class telephony
option ingress EF
option egress EF
config interface pppoe-wan
option name pppoe-wan
option disabled 0
option bandwidth_up 180mbit
option bandwidth_down 180mbit
option overhead_type docsis
# defaults:
option ingress 1
option egress 1
option mode diffserv4
option nat 1
option host_isolate 1
option autorate_ingress 0
option ingress_options ""
option egress_options "wash"
option options "ether-vlan"
config device wandev
option disabled 1
option name wan
option bandwidth 100mbit
My LAN is connecting to the Internet through a PPPOE client named "pppoe-wan"
Here is my ip -a :
root@host:/etc/init.d# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
link/ether 40:62:31:0b:fc:5b brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
link/ether 40:62:31:0b:fc:5c brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
link/ether 40:62:31:0b:fc:5d brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
link/ether 40:62:31:0b:fc:5e brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
link/ether 40:62:31:0b:fc:5f brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 40:62:31:0b:fc:60 brd ff:ff:ff:ff:ff:ff
8: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default qlen 1000
link/tunnel6 :: brd :: permaddr 7eb6:f125:7960::
9: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
10: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN group default qlen 1000
link/gre 0.0.0.0 brd 0.0.0.0
11: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN group default qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
12: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
13: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN group default qlen 1000
link/gre6 :: brd :: permaddr 368b:66b6:96f0::
14: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
link/void
16: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 40:62:31:0b:fc:5b brd ff:ff:ff:ff:ff:ff
inet 10.10.8.1/21 brd 10.10.15.255 scope global br-lan
valid_lft forever preferred_lft forever
17: pppoe-wan: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN group default qlen 3
link/ppp
inet 123.123.123.123 peer 123.123.123.1/32 scope global pppoe-wan
valid_lft forever preferred_lft forever
inet6 123::123:123:123:123 peer 123::123:123:123:123/128 scope link
valid_lft forever preferred_lft forever
18: ifb-dns: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 32
link/ether ba:f4:d5:83:d5:7b brd ff:ff:ff:ff:ff:ff
inet6 fe80::b8f4:d5ff:fe83:d57b/64 scope link
valid_lft forever preferred_lft forever
When trying to querying qosify status, I have got a empty reply.
How to properly configuring the interface?
[EDIT]
Changing back interface and option name into "wan" seems working now :
config interface wan
option name wan
according to Qosify: new package for DSCP marking + cake - #39 by dave14305 solid IP rules (ie. without +) are the priority, so
dns:cloudflare.com voice
tcp:443 video
this should result voice marking and not video in spite you may connect to cloudflare.com:443 for DNS service (DNS over HTTPS). Also it means any other request to cloudflare.com will be in voice.
1 Like