CAKE w/ DSCPs - cake-qos-simple

I see that I have only traffic in best effort and voice due to the rules you put

do you think we can add like this this is my personal use case for games

tell me if you think it's a good idea

brownsing is not necessary but I would like to prioritize cod 3074

udp . 3074 : goto dscp_set_voice, # COD1
			tcp . 3074 : goto dscp_set_voice, # COD2
			tcp . 1935 : goto dscp_set_video, # TWITCH
			tcp . 80 : goto dscp_set_besteffort, # BROWSING
			tcp . 443 : goto dscp_set_besteffort, # BROWSING 
			tcp . 51413 : goto dscp_set_bulk ## BULK

Yes so it seems this script is perfect for you because you understand the templated nft syntax and are already adding your own rules. Nice!

But as for my recommendation about what to set for effective CAKE operation, that is more @moeller0's area of expertise than mine.

By the way, with this approach you can set DSCPs in the router or in LAN clients (for upload), and then these will get automatically applied on download too.

The way this works is that any DSCP in an upload packet is stored to conntrack, and then that stored DSCP is also set on the download packet for the same connection (restored from conntrack).

For example this can be done in Windows like this:

1 Like

i will test now thanks

seems good he work perfectly thanks again for your work

i will be implemented my rule and keep informed if i has an error

thanks for compliment :wink:

Mon Jan  2 18:30:51 2023 user.notice cake-qos-simple: Stopped cake-qos-simple.
Mon Jan  2 18:30:51 2023 user.notice cake-qos-simple: Started cake-qos-simple.

i'm play on ps5 not pc my gaming

he work well thanks i will verified my dscp now :wink:

can you thinked add in option my dscp is not cs4

a little like this

config rule
	option name 'Cod1'
	option proto 'udp'
	option dest_port '3074'
	option class 'cs4'
	option dest_ip '192.168.2.220'

config rule
	option name 'Cod2'
	option proto 'udp'
	option src_port '3074'
	option src_ip '192.168.2.220'	
	option class 'cs4'

and possibility to block port

like !80 (block for my ps5)

your script has very nice potential i'm sure like dscpclassify :wink:

92.204.186.5	192.168.2.220 (ps5)	UDP	CS0	42200	3074	42200 → 3074 Len=970

my wireshakr show cs0 in cod

1 Like

Thank you for implementing all my suggestions so quickly.

I was able to resolve most of the issues from time, such as Error: Exclusivity flag on, cannot modify..

I think I had an instance of QoSify or the default cake SQM running. After setting them all to disabled under the luci startup section and rebooting, the error disappeared.

But one command is still failing for me:

root@FriendlyWrt:~# tc filter add dev eth0 parent ffff: protocol ip matchall action ctinfo dscp 63 128 action mirred egress redirect dev ifb-eth0
Error: TC classifier not found.
We have an error talking to the kernel

Since it seems to be working for everyone else, I think it may have something to do with my system. I am not using the offical OpenWRT, but a fork of it for the NanoPi R4S router, which is called FriendlyWRT. Perhaps this build doesn't include some kernel modules.

But I have a few more suggestions:

Print command descriptions to make error output easier to debug

Currently the script outputs errors in this way:

root@FriendlyWrt:~/cake-qos-simple# /etc/init.d/cake-qos-simple start
Error: Exclusivity flag on, cannot modify.
RTNETLINK answers: File exists
Error: TC classifier not found.
We have an error talking to the kernel
Error: Exclusivity flag on, cannot modify.
Error: Exclusivity flag on, cannot modify.

It's very hard to understand which command these errors come from and what they mean.

Instead I would suggest output like this:

root@FriendlyWrt:~/cake-qos-simple# /etc/init.d/cake-qos-simple start
Adding qdisc for device eth0...
Error: Exclusivity flag on, cannot modify.

Creating ifb download interface ifb-eth0...
RTNETLINK answers: File exists

Bringing ifb download interface ifb-eth0 up...

Adding filter to restore upload DSCP onto download flows...
Error: TC classifier not found.
We have an error talking to the kernel

Setting up cake for the upload interface eth0 with rate 10Mbit...
Error: Exclusivity flag on, cannot modify.

Setting up cake for the download interface ifb-eth0 with rate 10Mbit...
Error: Exclusivity flag on, cannot modify.

Here is a patch for the echos I added that I found helpful:

diff --git a/cake-qos-simple b/cake-qos-simple
index f66332d..6fa59a2 100644
--- a/cake-qos-simple
+++ b/cake-qos-simple
@@ -45,25 +45,36 @@ EXTRA_HELP="

 start()
 {
+	echo -e "Adding qdisc for device ${ul_if}..."
 	tc qdisc add dev ${ul_if} handle ffff: ingress

 	# If no $dl_if specified, then create appropriate IFB and redirect ingress on $ul_if thereto
 	if [[ -z ${dl_if} ]]; then

 		dl_if=ifb-${ul_if}
+
+		echo -e "\nCreating ifb download interface ${dl_if}..."
 		ip link add name ${dl_if} type ifb
+
+		echo -e "\nBringing ifb download interface ${dl_if} up..."
 		ip link set ${dl_if} up
+
 		# For each upload interface ingress packet conditionally restore DSCP from conntrack if available and redirect to IFB
+		echo -e "\nAdding filter to restore upload DSCP onto download flows..."
 		tc filter add dev ${ul_if} parent ffff: protocol ip matchall action ctinfo dscp 63 128 action mirred egress redirect dev $dl_if

 	# If $dl_if specified, then skip IFB creation
 	else
 		# For each upload interface ingress packet conditionally restore DSCP from conntrack if available
+		echo -e "\nAdding filter to restore upload DSCP onto download flows..."
 		tc filter add dev ${ul_if} parent ffff: protocol ip matchall action ctinfo dscp 63 128
 	fi

 	# These lines set up the cake instances and must be modified for desired operation including cake bandwidths
+	echo -e "\nSetting up cake for the upload interface ${ul_if} with rate ${cake_ul_rate_Mbps}Mbit..."
 	tc qdisc add dev ${ul_if} root cake bandwidth ${cake_ul_rate_Mbps}Mbit ${cake_ul_options}
+
+	echo -e "\nSetting up cake for the download interface ${dl_if} with rate ${cake_dl_rate_Mbps}Mbit..."
 	tc qdisc add dev ${dl_if} root cake bandwidth ${cake_dl_rate_Mbps}Mbit ${cake_dl_options}

 	logger -t cake-qos-simple "Started cake-qos-simple."
@@ -72,12 +83,22 @@ start()
 stop()
 {
 	[[ -z ${dl_if} ]] && dl_if=ifb-${ul_if}
+
+	echo -e "\nDeleting ingress qdisc for ${ul_if}..."
 	tc qdisc del dev ${ul_if} ingress
+
+	echo -e "\nDeleting root qdisc for ${ul_if}..."
 	tc qdisc del dev ${ul_if} root
-        tc qdisc del dev ${dl_if} root
-        ip link set ${dl_if} down
-        ip link del ${dl_if}
-
+
+	echo -e "\nDeleting root qdisc for ${dl_if}..."
+	tc qdisc del dev ${dl_if} root
+
+	echo -e "\nPutting down interface ${dl_if}..."
+	ip link set ${dl_if} down
+
+	echo -e "\nDeleting interface ${dl_if}..."
+	ip link del ${dl_if}
+
 	logger -t cake-qos-simple "Stopped cake-qos-simple."
 }

Disabling the cake-qos-simple service doesn't seem to work

I set the service to Disabled in Luci -> Startup, but upon reboot it seemed like the script was started.

I am not sure how the enabled/disabled system is working in OpenWRT and whether the issue is with OpenWRT itself or the init.d script.

But I just wanted to make you aware of the possibility of this.

1 Like

So, one more datapoint, dscpclassify (or more specifically layer_cake_ct.qos) are not working on my my FriendlyElec NanoPi R4S with FriendlyWRT either.

I see this in the logs:

Tue Jan  3 18:55:30 2023 user.notice SQM: Starting SQM script: layer_cake_ct.qos on eth0, in: 25000 Kbps, out: 50000 Kbps
Tue Jan  3 18:55:30 2023 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc filter add dev eth0 parent ffff: matchall action ctinfo dscp 0x0000003f mirred egress redirect dev ifb4eth0
Tue Jan  3 18:55:30 2023 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: Error: TC classifier not found. We have an error talking to the kernel
Tue Jan  3 18:55:30 2023 user.notice SQM: WARNING: sqm_start_default: layer_cake_ct.qos lacks an ingress() function
Tue Jan  3 18:55:30 2023 daemon.info netdata[4643]: RRDSET: chart name 'net.ifb4eth0' on host 'FriendlyWrt' already exists.
Tue Jan  3 18:55:30 2023 daemon.info netdata[4643]: RRDSET: chart name 'net_operstate.ifb4eth0' on host 'FriendlyWrt' already exists.
Tue Jan  3 18:55:30 2023 daemon.info netdata[4643]: RRDSET: chart name 'net_carrier.ifb4eth0' on host 'FriendlyWrt' already exists.
Tue Jan  3 18:55:30 2023 daemon.info netdata[4643]: RRDSET: chart name 'net_mtu.ifb4eth0' on host 'FriendlyWrt' already exists.
Tue Jan  3 18:55:31 2023 user.notice SQM: layer_cake_ct.qos was started on eth0 successfully

So, the problem is not within your scripts.

The problem is that these patches don't seem to be included in the FriendlyWRT build:
https://github.com/openwrt/openwrt/search?q=connmark_tg

This makes ctinfo / kmod-sched-ctinfo not working on NanoPi R4S or other FriendlyWRT devices.

If someone searches for this issue and finds this post, here are some further links to read about how it all works:

I think the only solution is to use an actual OpenWRT build, but I am not sure if I am ready for that can of worms at the moment.

@kimcha do you has a windows or mac ?

I am on a mac :slight_smile:

1 Like

use textwrangler for edit the script
then copie direclty to text wrangler in winscp
you should see the script well colored it depends on the script
if you use notepad or other the script can not work sometimes

he would be work with software textwrangler :wink:

is equivalent to notepad++

keep me informed

Thank you, I really appreciate your help.

I identified that the problem is not caused by cake-qos-simple. The problem is that I am not using the official OpenWRT. Instead I am using FriendlyWRT, which is a fork of OpenWRT for FriendlyElec NanoPi devices and they didn't include some required patches.

1 Like

Nice work in figuring this out and I like your suggestions.

Are you minded to switch to a different build? Is your device officially supported?

Thanks for the answer, I was wondering what to do, your link helped a lot.

1 Like

Which patch you are missing in official openwrt image for r4s? Im using it without any problems. @Lynx Hope to test the cake-qos-simple soon and also provide a feedback. Thanks!

2 Likes

@Kimcha many thanks again for your excellent suggestions. I have implemented them with this commit:

Example output:

root@OpenWrt:/etc/init.d# service cake-qos-simple start
Setting up ingress handle for dev wan.

No dl_if specified, so setting up appropriate IFB.

Creating IFB device: ifb-wan.

Setting interface ifb-wan up.

Setting up tc filter to restore DSCPs from conntrack on ingress packets on interface wan and redirecting to IFB interface ifb-wan.

Setting up CAKE on interface wan with bandwidth 20Mbit/s and options: diffserv4 dual-srchost nonat wash no-ack-filter noatm overhead 0.

Setting up CAKE on interface ifb-wan with bandwidth 20Mbit/s and options: diffserv4 dual-dsthost nonat nowash ingress no-ack-filter noatm overhead 0.

Started cake-qos-simple.
root@OpenWrt:/etc/init.d# service cake-qos-simple stop
Removing ingress handle on wan.

Removing CAKE on wan.

Removing CAKE on ifb-wan.

Setting IFB interface ifb-wan down.

Removing IFB interface ifb-wan.

Stopped cake-qos-simple.

Has anyone other than @Dopam-IT_1987 successfully set up cake-qos-simple? I like it because it's super lightweight and supports easy handling of DSCPs.

While I agree, I do want to add, that sqm-scripts itself is not terribly resource intensive either.

3 Likes

Not sure why the TTL bump is in a QOS script, since most users won’t need it and should probably delete that chain locally once downloaded.

1 Like

Good point. I have it there only because I use it and I agree that for the general case it presumably provides no benefit (or harm?). The intention is just to provide a template .nft to get the user started.

I presume in practice that assigning DNS and NTP requests to the voice tin will have little effect even in low bandwidth situations given the sparse flow boost?

In my particular case really the most important function of the .nft rules is to store the DSCPs to conntrack on upload, i.e.:

table inet cake-qos-simple {

        chain hook-output {

                type filter hook output priority filter

                # OpenWrt->wan
                oifname wan goto classify-and-store-dscp

        }

        chain hook-forward {

                type filter hook forward priority filter

                # lan->wan
                iifname $IFACE_NAMES goto classify-and-store-dscp

        }

    	chain classify-and-store-dscp {

		# jump classify-dscp
		jump store-dscp-in-conntrack
	    }

     	chain store-dscp-in-conntrack {

                ip version 4 ct mark set (@nh,8,8 & 252) >> 2
                ip6 version 6 ct mark set (@nh,0,16 & 4032) >> 6
                ct mark set ct mark or 128
        }
}

because I like setting DSCPs in Windows-based LAN clients for Teams/Zoom sessions, which is my primary use case for DSCPs.

Hey @poxystudd just curious if you've tried the script yourself? Right now my solution has at least one user. That is myself.

root@OpenWrt:~# /etc/init.d/cake-qos-simple start
Setting up ingress handle for dev eth0.2.

No dl_if specified, so setting up appropriate IFB.

Creating IFB device: ifb-eth0.2.

Setting interface ifb-eth0.2 up.

Setting up tc filter to restore DSCPs from conntrack on ingress packets on interface eth0.2 and redirecting to IFB interface ifb-eth0.2.

Setting up CAKE on interface eth0.2 with bandwidth 0.7Mbit/s and options: diffserv8 dual-srchost nat wash ack-filter atm overhead 32 ether-vlan.

Setting up CAKE on interface ifb-eth0.2 with bandwidth 12Mbit/s and options: diffserv8 dual-dsthost nat nowash ingress no-ack-filter atm overhead 32 ether-vlan.

Started cake-qos-simple.

looks like the script is working as it should
now i should start testing the markings

1 Like

You can test with 'tcpdump -i ifb-eth0.2 -v'. And also 'service cake-qos-simple download'.

1 Like