Help using nft marks and tc filtering

Installing and Using OpenWrt
1

Hi.

I need some help with tc filters in regards to packets marked by nft.

I use 3 rules inside the mangle_postrouting chain to mark udp packets based on their payload length destined for a local ip (192.168.4.10) connected to the lan side of my openwrt vm.

nft add rule inet fw4 mangle_postrouting ip daddr 192.168.4.10 udp dport 55585 udp length \{ 1-100 \} counter meta mark set 1
nft add rule inet fw4 mangle_postrouting ip daddr 192.168.4.10 udp dport 55585 udp length \{ 101-200 \} counter meta mark set 2
nft add rule inet fw4 mangle_postrouting ip daddr 192.168.4.10 udp dport 55585 udp length \{ 201-1000 \} counter meta mark set 3

and then using 3 filters on tc, I feed those marked packets into 3 different flowids each one running the netem qdisc where I can monitor the stats with tc -s and redirect them into their own IFBs where I can monitor the streams with tcpdump

tc qdisc add dev br0 root handle 1: prio
tc filter add dev br0 protocol ip parent 1: prio 1 handle 1 fw flowid 1:1 action mirred egress redirect dev smallD
tc filter add dev br0 protocol ip parent 1: prio 2 handle 2 fw flowid 1:2 action mirred egress redirect dev mediumD
tc filter add dev br0 protocol ip parent 1: prio 3 handle 3 fw flowid 1:3 action mirred egress redirect dev largeD

and this works well enough, I can verify the packets on each IFB are the correct length. While watching the numbers on

watch -tn 0.2 'tc -s qdisc show dev br0'

The problem I am facing is when trying to do this in the opposite direction. I want to do the same with packets coming out of 192.168.4.10 destined for my application server upstream. So tweaking the nft rules by swapping daddr and dport with saddr and sport and then using the ingress qdisc for br0 with no success. I also tried creating a veth pair and first passing the traffic through it and tried different qdiscs on either side, including ingress and prio also with no success.

My thinking at this point was that nft is marking the packets too late for the tc filtering so I tried using different nft rules such as

table netdev filter {
	chain ingress {
		type filter hook ingress devices = { eth1 } priority -149; policy accept;
		ip saddr 192.168.4.10 udp sport 55585 udp length { 1-100 } counter meta mark set 1
		ip saddr 192.168.4.10 udp sport 55585 udp length { 101-200 } counter meta mark set 2
		ip saddr 192.168.4.10 udp sport 55585 udp length { 201-1000 } counter meta mark set 3
	}
}

also with no success. I tried most of the chains that OpenWRT creates by default in the inet fw4 table, also with no success.

I did verify everytime for sanity by tcpdumping eth1 and indeed the traffic flow in both directions show up with all the different lengths.

So the outcome I am hoping to achieve is to split the stream of traffic with the saddr of 192.168.4.10 and sport of 55585 into its own prio flowid and/or IFB. Any help would be really appreciated.

2

Are you using OpenWrt
Please post output of

ubus call system board

netdev table contains fragents, the length will not be representative, use inet ... prerouting prio=raw
in

table netdev filter {
	chain ingress {
		type filter hook ingress devices = { eth1 } priority -149; policy accept;
		ip saddr 192.168.4.10 udp sport 55585 udp length { 1-100 } counter meta mark set 1
		ip saddr 192.168.4.10 udp sport 55585 udp length { 101-200 } counter meta mark set 2
		ip saddr 192.168.4.10 udp sport 55585 udp length { 201-1000 } counter meta mark set 3
	}
}

out

ip saddr 192.168.4.10 udp sport 55585 meta mark set udp length map { 1-100 : 1 , 101-200 : 2 ...
1 Like
3

Hi @anon63541380 thanks for your reply mate. Yes I am indeed using OpenWRT
{
"kernel": "6.6.93",
"hostname": "owrt",
"system": "AMD Ryzen 7 5700G with Radeon Graphics",
"model": "QEMU Standard PC (i440FX + PIIX, 1996)",
"board_name": "qemu-standard-pc-i440fx-piix-1996",
"rootfs_type": "ext4",
"release": {
"distribution": "OpenWrt",
"version": "24.10.2",
"revision": "r28739-d9340319c6",
"target": "x86/64",
"description": "OpenWrt 24.10.2 r28739-d9340319c6",
"builddate": "1750711236"
}
}

Thank you for your suggestion, will test it out and update.

4

Ok, so I created the rules for the raw_prerouting chain that exists by default inside the inet fw4 table using these commands

nft add rule inet fw4 raw_prerouting ip saddr 192.168.5.10 udp sport 55585 udp length \{ 1-125 \} counter meta mark set 4
nft add rule inet fw4 raw_prerouting ip saddr 192.168.5.10 udp sport 55585 udp length \{ 126-176 \} counter meta mark set 5
nft add rule inet fw4 raw_prerouting ip saddr 192.168.5.10 udp sport 55585 udp length \{ 201-1200 \} counter meta mark set 6

then using the ingress qdisc on br0 I redirected the traffic from ip src 192.168.5.10 and ip sport 55585 to a newly created IFB “po”

then setup tc filter rules to split the traffic into the 3 prio bands based on fw mark

$ tc filter show dev po
filter parent 1: protocol ip pref 1 fw chain 0
filter parent 1: protocol ip pref 1 fw chain 0 handle 0x4 classid 1:1
filter parent 1: protocol ip pref 1 fw chain 0 handle 0x5 classid 1:2
filter parent 1: protocol ip pref 1 fw chain 0 handle 0x6 classid 1:3

at this point, monitoring the nft rule counter, it did appear to be correctly seeing and marking traffic

$ watch -ctn 0.1 'nft -a list chain inet fw4 raw_prerouting'
table inet fw4 {
        chain raw_prerouting { # handle 40
                type filter hook prerouting priority raw; policy accept;
                ip saddr 192.168.5.10 udp sport 55585 udp length 1-125 counter packets 7729 bytes 797467 meta mark set 0x00000004 # handle 577
                ip saddr 192.168.5.10 udp sport 55585 udp length 126-176 counter packets 1920 bytes 319200 meta mark set 0x00000005 # handle 578
                ip saddr 192.168.5.10 udp sport 55585 udp length 176-1200 counter packets 119 bytes 27063 meta mark set 0x00000006 # handle 579
        }
}

however when monitoring the qdisc stats using watch, all packets appears to be feeding into only 1 of the prio bands

$ watch -ctn 0.1 'tc -s qdisc show dev po'
qdisc prio 1: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 3703650 bytes 28484 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc netem 8001: parent 1:1 limit 1000 seed 9494236801328697090
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc netem 8003: parent 1:3 limit 1000 seed 11074632769230987876
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc netem 8002: parent 1:2 limit 1000 seed 457611588474986345
 Sent 2982178 bytes 22859 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

Any further suggestions is appreciated. Also thank you for all the work you do on the OpenWRT forums @anon63541380, many of your answers on other threads have helped me in better understanding how to use this powerful and sophisticated software.

5

Why not qosmate....

1 Like
6

I did hear about that project, never looked into it, I guess now is the time. Thanks mate.