Nft or pbr doesn't create set in Owrt 24.10

Installing and Using OpenWrt Network and Wireless Configuration
1

I made a custom script file for PBR and it worked with OpenWrt 23.05. It's more complex but if simplify (I use another interface and generate IPs in the file first) this command works in 23.05:

nft "add element inet fw4 pbr_wan_4_dst_ip_user { 139.59.210.197, }"

In 23.05 after command executed I can see in nft list ruleset that new set named pbr_wan_4_dst_ip_user created and some other rules, so it [was] not required to create it manualy.

Now I have router with OpenWrt 24.10 release and this command returns error:

Error: No such file or directory
add element inet fw4 pbr_wan_4_dst_ip_user { 139.59.210.197, }
                     ^^^^^^^^^^^^^^^^^^^^^

If I add single IP to pbr interface it works in both 23 and 24 owrt (without creating new set).

23.05 (official build for ax6s): pbr 1.1.6-20 (nft mode), dnsmasq 2.90-2
24.10 (custom with small patches for ax3000t): pbr 1.1.8-r10 (fw4 nft file mode), dnsmasq-full 2.90-r4

Is it an error in PBR or nft or anything else and how to make this command work?

2

Can you list the set back?

3

In 23.05 yes, nft list ruleset returns this (I cut most):

table inet fw4 {
        set pbr_wan_4_dst_ip_user {
                type ipv4_addr
                flags interval
                auto-merge
                comment ""
                elements = { 139.59.210.197 }
        }

        chain pbr_prerouting {
                ip daddr @pbr_wan_4_dst_ip_user goto pbr_mark_0x010000
        }
4

that comma does not look ok

6

I tried with and without comma and even brackets (copilot suggested), it doesn't work. Without comma:

root@OpenWrt:~# nft "add element inet fw4 pbr_wan_4_dst_ip_user { 139.59.210.197 }"
Error: No such file or directory
add element inet fw4 pbr_wan_4_dst_ip_user { 139.59.210.197 }
                     ^^^^^^^^^^^^^^^^^^^^^

(edited)

7

thanks, bye

8

Yep, it's not good, I tried deepseek r1 and it looks much better.

9

PBR changed there no longer is a set pbr_wan_4_dst_ip_user

So you have to create it first

nft 'add set inet fw4 pbr_wan_4_dst_ip_user { type ipv4_addr;   auto-merge;  counter;  flags interval; }'

Then you will have to make sure that set is also targeted from the pbr prerouting chain (or an other chain you want) e.g.:

nft 'add rule inet fw4 pbr_prerouting ip daddr @pbr_wan_4_dst_ip_user counter goto pbr_mark_0x010000 comment "domain_via_wan"'

If you get a working example I am interested in the end result

10

Thanks, I'm trying to go the same way. I already have added the set the same way. But in my real case I need pbr_mark_0x020000 instead of pbr_mark_0x010000. I added it like this (the same way it looks in 23.05):

nft 'add set inet fw4 my_wg_4_dst_ip_user {type ipv4_addr; flags interval; auto-merge; }'
nft add rule inet fw4 pbr_prerouting ip daddr @my_wg_4_dst_ip_user goto pbr_mark_0x020000

and added IP to the set and it even works.
But the problem is I don't want to hardcode pbr_mark_0x020000 because it may change and I don't mind what is this at all. Now I'm trying to find how this:

        chain pbr_mark_0x020000 {
                meta mark set meta mark & 0xff02ffff | 0x00020000
                return
        }

relates to the wg interface so I can parse something and extract it. I checked all these numbers in nft runtime config and in /etc/config/pbr but couldn't find any of these numbers.

11

ip rule show
or
service pbr status

1 Like
12

Well it's done. This is all the script pbr.user.tun2vless.my.

#!/bin/sh
# This file is heavily based on code from https://github.com/Xentrk/netflix-vpn-bypass/blob/master/IPSET_Netflix.sh

# Modified for OpenWrt 24.10 because its NFT and PBR don't create set automatically

TARGET_INTERFACE='tun2vless'
TARGET_SET="my_${TARGET_INTERFACE}_4_dst_ip_user"
TARGET_TABLE='inet fw4'
TARGET_DL_FILE="/etc/pbr.list.my"
TARGET_NFT_FILE="/etc/pbr.list.my.nft"
PBR_TABLE="pbr_${TARGET_INTERFACE}"

# Get pbr magic number 0x20000 from output string:
# 29998:  from all fwmark 0x20000/0xff0000 lookup pbr_tun2vless
# from command:
# ip rule show
# If doesn't work may try `ip route show table 100` if it will be like `100` instead of `pbr_tun2vless`
PBR_MARK_RAW=$(ip rule show | grep -m1 "${PBR_TABLE}" | awk 'match($0, /fwmark 0x[^\/]*/) {print substr($0, RSTART+9, RLENGTH-9)}')
# now PBR_MARK_RAW=20000

if [ -z "$PBR_MARK_RAW" ]; then
	echo "Error: PBR_MARK_RAW is empty"
	return 1
fi

# Find nft chain with this pbr mark, string looks like:
# chain pbr_mark_0x020000 {
NFT_CHAIN_RAW=$(nft list table inet fw4 | grep -m1 -E "\s*chain pbr_mark_0x0*${PBR_MARK_RAW}\s*\{\$")

if [ -z "$NFT_CHAIN_RAW" ]; then
	echo "Error: NFT_CHAIN_RAW is empty"
	return 1
fi

# Parse `pbr_mark_0x020000` chain name
NFT_CHAIN=$(echo "${NFT_CHAIN_RAW}" | awk 'match($0, /chain pbr_mark_0x[0-9a-f]*/) {print substr($0, RSTART+6, RLENGTH-6)}')
# now NFT_CHAIN=pbr_mark_0x020000

if [ -z "$NFT_CHAIN" ]; then
	echo "Error: NFT_CHAIN is empty"
	return 1
fi

# Create set
nft "add set inet fw4 ${TARGET_SET} {type ipv4_addr; flags interval; auto-merge; }"
# Create rule in chain
nft "add rule inet fw4 pbr_prerouting ip daddr @${TARGET_SET} goto ${NFT_CHAIN}"

[ -z "$nft" ] && nft="$(command -v nft)"
_ret=1

printf "add element %s %s { " "$TARGET_TABLE" "$TARGET_SET" > "$TARGET_NFT_FILE"
awk '/^\d/{printf $1 ", "}' "$TARGET_DL_FILE" >> "$TARGET_NFT_FILE"
printf " } " >> "$TARGET_NFT_FILE"
"$nft" -f "$TARGET_NFT_FILE"
_ret=0

return $_ret

It looks pbr.list.my file, skips empty and starting from # lines, takes other lines as IPv4, and adds them to go through specified interface.
Might be it may not work in some cases and then it's required to go deeper and extract interface name from like ip route show table 100 command (I put comment in case to remember), but for my it works without this.

1 Like
13

Great that you could solve it

If your problem is solved, please consider marking this topic as [Solved]. See How to mark a topic as [Solved] for a short how-to.
Thanks! :slight_smile:

1 Like
14

I got a similar issue with pbr in 24.10.0 in connectíon with DSCPclassify but in my case the problem seems to be that pbr overrides the nft list table inet dscpclassify with every interface change (e.g. Wireguard or OpenVPN/tunX). I could be wrong but is my issue somewhat related to yours?

15

Might be related. I found that nft seems not run at all when pbr executes custom user files. I created another question PBR in 24.10 creates set for custom file after its execution

16

Uhm, unless I misunderstood the issue, the statement above might not be 100% accurate.

The pbr has changed so that when the user files are ran, the set doesn't exist yet, however it should not be created in the user file, as the sets are added to the fw4 nft file which pbr creates: https://github.com/stangri/pbr/blob/51332a93f2899ee630a99522e1ed46cbf884050d/files/etc/init.d/pbr#L2094-L2108

So I'd say instead of making a user file which contradicts the logic of the fw4 nft file use, rewrite the user file so that it adds to the fw4 nft file that pbr creates rather than call nft commands directly.

17

You're right, the set doesn't exist. And nft file should be used.
My first attempt was to make it work by putting my nft files near /usr/share/nftables.d/ruleset-post/30-pbr.nft file and it worked, almost okay except when I uncheck pbr user file then nft file still existed and applied. I partially solved this to make this file symlink to nft file in /tmp (it disappears after reboot).
But later I found how it works and how to make it without this workaround: PBR in 24.10 creates set for custom file after its execution - #12 by kanani

1 Like
18

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.