This is how i installed qosify on my wrt32x about a month ago not sure if there is an updated version or not though...
wget https://downloads.openwrt.org/snapshots/targets/mvebu/cortexa9/packages/qosify_2022-04-08-ef82defa-1_arm_cortex-a9_vfpv3-d16.ipk
opkg install qosify_2022-04-08-ef82defa-1_arm_cortex-a9_vfpv3-d16.ipk
just change the path to your firmware version...
Is it easy to set up qosify to ONLY mark packets and not do anything else?
I am a total noob, but following the examples in this forum I have been able to configure qosify successfully and without knowing how to test it thoroughly or how it works at all, my impression is that it is very simple to configure and that it works quite well.
Is it possible that there is something to make adaptive bandwidth work with qosify?
I would like to think so.
The below may reflect a lack of understanding on my part. But based on my limited understanding, I share the following thoughts.
Qosify describes itself in the following way:
QoSify is simple daemon for setting up and managing CAKE along with a custom eBPF based classifier that sets DSCP fields of packets.
I'm hopeful qosify doesn't force you to set up CAKE such that CAKE can be set up manually and rates adjusted manually outside qosify.
Perhaps qosify would be better off just being markify to mark packets rather than this potentially awkward mix of trying to combine application of CAKE and marking which will presumably not work for cases like mine where I have VPN pbr and need to manage my own ifb (via hotplug for when VPN goes down and back up) and settings to make upload and download get handled correctly despite mix of encrypted and unencrypted flows over the last mile connection I need to control.
Since it's not applying CAKE that's hard. That can be done easily using scripts or the sqm tools in OpenWrt.
The value add is facilitating marking without horrible netifd scripting which to me looks like a total mess.
So that's why I'm wondering if qosify ought to be split up into separate components or just handle the marking aspect.
Itâs a perfect total solution for 99% of users looking for QoS. You edge case is not reason enough to make it more complicated for the 99%, in my opinion.
The application of CAKE has been solved with existing OpenWrt solutions. That's never been a complicating issue for users. What has been a complicating issue is DSCP marking. The latter was the missing part that needed attention.
I think it's just the design choice that I struggle to wrap my head around because it seems like DSCP marking apart from applying CAKE makes sense. Aren't there times you'd want DSCP marking without necessarily even wanting CAKE at all?
It's like having one tool to offer a wine glass and better bottle opener in one when wine glasses were already working well. There seems to be something nice in my mind about having separate discrete tools for different jobs.
In any case, at the very least is it possible to just use qosify to DSCP mark without it managing CAKE for those who just want the DSCP marking aspect of qosify and not the CAKE aspect of qosify?
Tagging without sqm can make sense, think of a campus network with QoS on the switches (even cheap smart-managed L2 switches do offer that).
It should not be hard to change this to simply not emit the tc calls that instantiate the qdiscs.
Say either by supplying -1 as rate (0 is a legitimate rate for cake indicating "unlimited", which can make sense in specific conditions) or by adding a new control variable to enable/disable cake.
The qosify tagging (adding the bpf filter) is still relative to the interface where CAKE will be instantiated. So they canât be too decoupled otherwise you may not be tagging the traffic before it enters the qdisc.
Thereâs nothing preventing the autorate script from changing CAKE bandwidth even if qosify is in use.
root@router:~# tc filter show dev eth1 egress
filter protocol all pref 272 bpf chain 0
filter protocol all pref 272 bpf chain 0 handle 0x1 qosify_egress_eth:[*fsobj] direct-action not_in_hw id 4 tag 9638cd99f888e843 jited
root@router:~# tc filter show dev eth1 ingress
filter protocol all pref 272 bpf chain 0
filter protocol all pref 272 bpf chain 0 handle 0x1 qosify_ingress_eth:[*fsobj] direct-action not_in_hw id 9 tag 9638cd99f888e843 jited
filter protocol ip pref 273 u32 chain 0
filter protocol ip pref 273 u32 chain 0 fh 800: ht divisor 1
filter protocol ip pref 273 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:1 not_in_hw
match 00350000/ffff0000 at 20
action order 1: mirred (Egress Redirect to device ifb-dns) stolen
index 1 ref 1 bind 1
filter protocol 802.1Q pref 274 u32 chain 0
filter protocol 802.1Q pref 274 u32 chain 0 fh 801: ht divisor 1
filter protocol 802.1Q pref 274 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1:1 not_in_hw
match 00350000/ffff0000 at 20
offset plus 4
action order 1: mirred (Egress Redirect to device ifb-dns) stolen
index 2 ref 1 bind 1
filter protocol ipv6 pref 275 u32 chain 0
filter protocol ipv6 pref 275 u32 chain 0 fh 802: ht divisor 1
filter protocol ipv6 pref 275 u32 chain 0 fh 802::800 order 2048 key ht 802 bkt 0 flowid 1:1 not_in_hw
match 00350000/ffff0000 at 40
action order 1: mirred (Egress Redirect to device ifb-dns) stolen
index 3 ref 1 bind 1
filter protocol ipv6 pref 276 u32 chain 0
filter protocol ipv6 pref 276 u32 chain 0 fh 803: ht divisor 1
filter protocol ipv6 pref 276 u32 chain 0 fh 803::800 order 2048 key ht 803 bkt 0 flowid 1:1 not_in_hw
match 00350000/ffff0000 at 40
offset plus 4
action order 1: mirred (Egress Redirect to device ifb-dns) stolen
index 4 ref 1 bind 1
filter protocol all pref 277 u32 chain 0
filter protocol all pref 277 u32 chain 0 fh 804: ht divisor 1
filter protocol all pref 277 u32 chain 0 fh 804::800 order 2048 key ht 804 bkt 0 flowid 1:1 not_in_hw
match 00000000/00000000 at 0
action order 1: mirred (Egress Redirect to device ifb-eth1) stolen
index 5 ref 1 bind 1
By this you mean that you want to tag before you apply CAKE right? But tagging and applying CAKE are different things and you may not want to do both on the same device right?
You want to tag the traffic before it enters the qdisc for the benefit of CAKE tins, or HTB classes, etc. Much like your original concern long ago, if youâre using a VPN on the router, tagging on the WAN interface doesnât make sense if the outgoing traffic is already encrypted. So trying to separate tagging and tinning can get complicated.
My own network is not complicated, so thatâs why I appreciate Qosify for what it is.
Just one question - If using DSCP markings, is there any need to enable NAT and host isolation?
These are orthogonal IMHO. Cake in its diffserv modes can use different DSCPs to steer a packet into its 1, 3, 4 or 8 priority tiers. In side each of these tiers it will then apply the isolation mode requested...
If you are fine with pure per-flow-fairness (individually in each tier) you need neither the nat nor the isolation keywords, if however you still want per-internal-host fairness you will need the isolation keywords. The nat keyword technically is only required if, like in the default configuration with cake on the wan interface, the packets do not yet/anymore carry the post/pre-NAT internal IP addresses.
Good point. For ingress traffic just using the true wan interface should work however, egress is tricker as the marking need to change before the egress qdisc handles a packet.
ifstatus wan | grep -e device or just ifstatus wan if you want more detail.
or scripted way use /lib/functions/network.sh functions like network_find_wan, network_get_device etc
@amteza I reread the helpful posts you wrote about concerning me trying qosify. As a reminder, I have this setup:
I understand that using 'flows nonat' CAKE can still differentiate encrypted flows but that this only works for upload (@moeller0?) and hence for download I set up my IFB using this script in which unencrypted packets from both WAN and VPN are combined (taking care to skip over the encrypted packets on WAN to avoid duplicates).
In any case, I think you demonstrated above how with my setup I could get qosify to effectively DSCP mark in respect of download. That is, CAKE on IFB will work with the DSCP markings applied by qosify.
But how would I get this to work in respect of upload then, mindful that the WAN interface has a mixture of encrypted and unencrypted flows?
I looked into this solution here about preserving DSCP marking across WireGuard:
This example shows how to use BPF to preserve DSCP values across an encapsulating interface such as Wireguard. It relies on the encapsulation layer preserving the skb->hash value across the encapsulation, which is commonly the case on kernel encapsulation protocols (including Wireguard).
I'm hazy on what qosify would do here. I mean if qosify DSCP marks flows for upload and I apply 'diffserv4' will CAKE correctly see DSCP markings notwithstanding mixture of encapsulated and unencrypted flows? @moeller0 any clue about this?
So already I was surprised that qosify will allow DSCP marking to work in respect of the IFB - that is good. Now I am wondering if it will also allow DSCP marking to work in respect of WAN whilst relying on skb->hash preservation for CAKE to differentiate encrypted WireGuard encapsulated flows.
Also the preserve-dscp tool above gives the following warning:
- As mentioned above, doing this in the first place LEAKS DATA! I.e., it makes it possible for an outside observer to distinguish between different types of traffic inside the tunnel. This is generally a bad idea, especially if the traffic goes across the public internet.
Would this issue also apply if I use qosify in the way I am asking about (that is assuming it is even possible)?
I hope this all makes sense. There may be an easier way to tackle mixture of encrypted and unencrypted flows and make use of DSCP markings at the same time. If you have any suggestions please let me know.
@dlakelan suggested at one point putting WAN and VPN in a different namespace and then having a sort of 'fake wan' in the global namespace (with veth interconnecting the namespaces), but whilst I really like this idea I think setting that up looks really tricky 
.
Yes, wireguard can be helpful and set firewall marks, but only after wireguard has touched a packet, so for egress that works, but not for ingress. However with your ingress setup you do not actually need that, because cake-on-ifb sees only decrypted packets and can calculate its own hash (so on ingress all the diffserv modes cake offers are fair game for you).
That sounds like a decent but complicated way forward, using cake'e wash keyword would also solve the issue with DSCP leakage into the internet. At that point your egress cake would see usable DSCP marks, the question is more will the the diffserv modes do the right thing in regards to the skb->hash. You might want to ask @tohojo about this, he is the expert on this...
As I said just use cake's wash keyword or tc to reset the DSCP field to CS0 after using the original value to select the priority tin, et voila no leakage anymore...
Thanks. I guess I need an understanding of at what point qosify applies the DSCP markings. The preserve-dscp script that @tohojo wrote seems to allow restoration of DSCP markings across a WireGuard interface.
As I understand it, this works by creating an association between skb->hash and DSCP marking pre-encapsulation, then, reliant on the preservation of skb->hash post-encapsulation, looking up the table for matches and applying the DSCP marks to the encapsulated packets.
I don't know if this would be necessary or not using qosify - does anyone have any clue about this?
It would be nice if we could circumvent this - couldn't we have WireGuard retain DSCP markings during encapsulation and then have CAKE strip them using 'wash' after having prioritised?
Or has this already been done in this patch and seems to make a lot of sense to me:
https://lore.kernel.org/wireguard/20190318210309.5dcb8375@strong.id.au/
But looks like this was not added to the main code base .
Or perhaps there's another more elegant way forward here?
