Easiest way to get DSCP working on Ingress?

toasty · July 11, 2024, 11:30am

I have sqm set up, nft firewall dscp rules set up, as well as the layer_cake_ct.qos script selected (and kmod-sched-ctinfo is installed). What am I missing to get DSCP markings working on ingress?

brada4 · July 11, 2024, 11:33am

You are missing 3rd party sending dscp markings over internet and your provider not clearing them.

moeller0 · July 11, 2024, 11:45am

layer_cake_ct.qos aims to store the DSCPs from outgoing packets into the conntrack database and restore the same DSCPs on ingressing packets*... so the nauve expectation would be to get some return packets in all priority tins that are used by egressing packets, so tins 1,2, 6, amd 7, but they only see return traffic in tins 2 and 7.

Maybe time to whip out tcpdump and conform the egressing DSCPs first and then see what happens further...

*) That way the ISPs DSCP markings are completely ignored/over-ruled.

brada4 · July 11, 2024, 1:37pm

Very little effect, input is consumed in gro bulk packets.

moeller0 · July 11, 2024, 1:58pm

This is a bit terse and somewhat cryptic... That said re-assigning DSCPs in a controlled fashion for ingressing packets has the following rationale. It steers packets into one's priority tiers and allows for selective back pressure, e.g. not dropping gaming traffic at all, while reigning in bulk transfers somewhat more aggressively. It can also affect the priority access classes used by WiFi links.
Regrding GRO/GSO cake as a qdisc can and will by default dissect larger meta packets into "normal" roughly MTU sized packets and hence will not be affected by these techniques, for other shaper/schedulker/AQM set ups one might need to first go and disable GRO/GSO on all interfaces that might collect meats and send them to the qdisc...

toasty · July 12, 2024, 5:28am

looks like tcpdump is showing them correctly, and wireshark does as well.

Im a bit new to this stuff and I like learning, but I can't seem to find how to get layer_cake_ct.qos working like how it describes. What am I missing? I can get DSCPClassify and cake-qos-simple to work with no issues, but I would like how to do it myself. Ive looked at the files for all of those and I mostly understand how it all works minus the restoring the dscp on ingress part for each one.

moeller0 · July 12, 2024, 5:36am

The contrack trick requires 2 components:
a) taking the dscp stored in conntrack and assigning it to incoming packets (IIRC this is done with a simple tc command)
b) iptables/nftables commands/rules to take the dscp from outgoing packets and storing them into conntrack

Are you sure you are doing both things?

toasty · July 12, 2024, 8:49am

I think this would be the part I'm missing. I couldn't find anything about it before. Is there a page / article / post anywhere about how to set it up?

moeller0 · July 12, 2024, 8:53am

Your beste reference likely is looking at one of the projeczs that have this already implemented like DSCPclassify or cake-qos-simple. You might want to consider writring a wiki article/howto if you figure this out.

Lynx · July 12, 2024, 10:03am

cake-qos-simple is designed and written as a template as a sort of springboard for your own customisation using appropriate nftables rules. These can be added in the nftables file or even just using firewall rules in LuCi.

DSCPs are stored to conntrack for outgoing packets here:

github.com

lynxthecat/cake-qos-simple/blob/3e22ba6f8c7fd2da94f66bef37af3438f0b37116/cake-qos-simple#L211


      
          	                ip dscp set cs2
          	                ip6 dscp set cs2
          	        }
          	
          	        # designate packet for cake tin: voice
          	        chain dscp_set_voice {
          	                ip dscp set cs4
          	                ip6 dscp set cs4
          	        }
          	
          	        chain store-dscp-in-conntrack {
          	                ct mark set ip dscp or 128
          	                ct mark set ip6 dscp or 128
          	                # For older versions of nftables (OpenWrt versions prior to 23.05), replace the two lines above with:
          	                # meta nfproto ipv4 ct mark set (@nh,8,8 & 252) >> 2
          	                # meta nfproto ipv6 ct mark set (@nh,0,16 & 4032) >> 6
          			# ct mark set ct mark or 128
          	        }
          	}
          	EOT

DSCPs are restored from conntrack for incoming packets here:

github.com

lynxthecat/cake-qos-simple/blob/3e22ba6f8c7fd2da94f66bef37af3438f0b37116/cake-qos-simple#L258


      
          	then
          
          		printf "\nNo dl_if specified, so setting up appropriate IFB.\n"
          		dl_if="ifb-${ul_if}"
          		printf "\nCreating IFB device: '${dl_if}'.\n"
          		ip link add name ${dl_if} type ifb
          		printf "\nSetting interface: '${dl_if}' up.\n"
          		ip link set "${dl_if}" up
          		# For each upload interface ingress packet conditionally restore DSCP from conntrack if available and redirect to IFB
          		printf "\nSetting up tc filter to restore DSCPs from conntrack on ingress packets on interface: '${ul_if}' and redirecting to IFB interface: '${dl_if}'.\n"
          		tc filter add dev "${ul_if}" parent ffff: protocol all matchall action ctinfo dscp 63 128 mirred egress redirect dev "${dl_if}"
          
          	# if $dl_if specified, then skip IFB creation
          	else
          		# for each upload interface ingress packet conditionally restore DSCP from conntrack if available
          		printf "\nSetting up tc filter to restore DSCPs from conntrack on ingress packets on interface '${ul_if}'.\n"
          		tc filter add dev "${ul_if}" parent ffff: protocol all matchall action ctinfo dscp 63 128
          	fi
          
          	printf "\nSetting up CAKE on interface: '${ul_if}' with bandwidth: '${cake_ul_rate_Mbps}Mbit/s' and options: '${cake_ul_options}'.\n"
          	tc qdisc add dev "${ul_if}" root handle 1: cake bandwidth "${cake_ul_rate_Mbps}Mbit" ${cake_ul_options}

moeller0 · July 12, 2024, 10:43am

Quick question, what does the or 128 stanza do?

dave14305 · July 12, 2024, 11:10am

It sets the tc-ctinfo statemask for use in the ingress tc filter.

Hudra · July 12, 2024, 11:46am

Here is also a short explanation:

I was experimenting with SQM scripts a while ago and created an SQM script with ctinfo and just a few necessary nftables rules, allowing one to use LuCI for marking forwarded traffic. I believe it was working when I tested it.

My original idea was to include this, or at least something similar, as a SQM script in OpenWrt. However, I'm not sure if this is a good approach...

If you want to try it:

wget -O /usr/lib/sqm/Layer_cake_ctinfoHudra.qos https://raw.githubusercontent.com/hudra0/sqm-scripts/main/Layer_cake_ctinfoHudra.qos

wget -O /usr/lib/sqm/Layer_cake_ctinfoHudra.help https://raw.githubusercontent.com/hudra0/sqm-scripts/main/Layer_cake_ctinfoHudra.help

then just activate the sqm instance and mark via luci firewall rules...

moeller0 · July 12, 2024, 1:34pm

So this is still on my todo list, to make this optional for sqm scripts, but ATM my own promary router is still on iptables so I can not test this. But I think adding the DSCP saving and restoration capability to coere sqm-scripts seems like a good way forward. This still leaves all the actual clever rules and heuristics out of sqm-scripts and inside those projects that want that.

That said, I am buggy enough with code I actually can test, so I will not create any PR until I can test this myself... (I understand that it will need some checking for recent enough nftables and tc binaries, and it needs some thought on how to undo this cleanly from the sqm-stop routine, and these details require being able to actually test things). I also note that creating a PR does not guarantee that it will end up in sqm-scripts...

Hudra · July 12, 2024, 1:52pm

This is exactly what I was thinking as well.

The version I provided should do some clean up when stopped but like I said this was just an experiment and is not fully tested.... I guess it would be a starting point...

Is tohojo still maintaining sqm-sritps?

toasty · July 12, 2024, 1:55pm

Thank you so much for sharing this. I was able to get what I wanted working by looking through your script and putting "action ctinfo dscp 63 128 mirred egress redirect dev $DEV" in the layer_cake script like you had and by adding + adjusting "add rule inet sqm_dscp sqm_dscp ct mark set ip dscp or 128 counter" to my current nft rules.

I do also agree that your script would be a great addition to sqm scripts (Although, I have not tested it fully).

moeller0 · July 12, 2024, 2:38pm

Sure, and he is open to PRs

moeller0 · July 12, 2024, 2:40pm

I would say the capability would be great to add, but preferably not as a new script but as something that can be enabled for the existing scripts...

Lynx · July 12, 2024, 3:06pm

Also with simple restoration LuCi firewall can be used to set the outgoing DSCPs. Hence everything could be done using a GUI, which I think would be helpful for many.

brada4 · July 12, 2024, 11:12pm

It is bitmask
1000000b
mark =to become= 32bit value OR bitmask.

you can see that in -c -d netlink