Policy-Based-Routing (pbr) package discussion

That is exactly right!

I have a vps running a vpn that, ideally, will tunnel the traffic to a specific host on lan, and that host will similarly be able to respond to the incoming connection traffic back through the vpn.

I have it working with the vpn client tunning directly on the host, the lan machine, however I'd like the vpn client to be on my openwrt router and route the traffic coming in on the vpn interface on a specific port to the specific lan machine based on a specific port. Also any outgoing traffic on that same port from thag host should be routed throguh the vpn.

Please let me know if this makes sense, otherwise I can draw a diagram when I get home.

I understand this diagram isn't entirely accurate as the VPN interface is of course a tunnel that routes through the ISP's gateway and internet to reach my VPN server. However I figured I'd throw together something quickly so I can refer to a visual while explaining my goal.

As of now I have made some iptable rules on my vpn server to route incoming traffic on specific ports to the VPN client IP 10.7.0.2. When running the VPN client on a local machine directly, say 192.168.2.10, I am able to establish a connection from the Internet into my the local machine on any of the specified open ports.

My goal is to have OpenWRT run the vpn client, and selectively route incoming traffic using PBR from the VPN to specific clients, suppose 192.168.2.10 is forwarded incoming traffic on port 53 and 192.168.2.12 is forwarded traffic incoming on port 21. Additionally I'd need OpenWRT to selectively route outgoing traffic from 192.168.2.10 on port 53 back through the VPN.

I'm new to policy based routing and have limitations in my understanding of iptables, nf_tables, etc. So I'm really just looking for any similar examples to work off, or tips on how to debug. Of course I also wouldn't complain if someone capable came up with the rules for me :slight_smile:

First of all, thank you for this!

I'm trying to use pbr on my unbound-running router - it seems to work fine, as long as I disable resolver set support (obvs), despite claiming that it has failed to set up any gateway:

image

I've checked the rules and routes and nft sets it sets up and it's all good - my wireguard tunnel is being used for the individual policies I specified.

I was wondering if there was any way of setting up nft set like functionality with unbound?

Thanks again,
Tim

I upgraded from 11 to 13 both pbr and luci and the luci app doesn't come up.

TypeError
pbr.getInterfaces is not a function

is all I get. Tried rebooting same thing.

Hey Tim,

We may have different use cases but I'd appreciate it if you'd please provide a bit more info on the wireguard use case.

If you read my post above im struggling to use pbr to route traffic through my wireguard client running on openwrt.

Thanks

Can you test the firewall config first accessing the server via wan and then just switch the firewall zone to that of VPN so you could test the pbr policies? Then the example for the Plex server in the README should work with the exception of setting its interface to that of VPN.

Sorry, I'm unable to provide any more guidance than that, but I believe there are other users on the forum who might. I wonder if @trendy or @mk24 or @psherman would be willing to step in.

Thank you for posting this. The service errors are troubling, I'll try to figure out what might be causing them and get back to you with a new build to try.

With popularization of tokens, it's hard enough to look up anything nft-related, but combining unbound with nft as search terms I'm struggling to find anything relevant. I've tried search unbound docs for nft (https://unbound.docs.nlnetlabs.nl/en/latest/search.html?q=nft&check_keywords=yes&area=default) but it just hangs there searching. So if you can find a documentation or provide examples of the working unbound config/setup so that unbound would populate specific nft sets, I'd be happy to have a look. Since AdGuardHome now also supports separate ipset files, I'll be looking into revamping resolver_set support anyways.

Have you cleared browser cache?

2 Likes

I don't see any challenges there. For the ingress traffic a DNAT would forward the packets to the selected lan hosts. As for the egress traffic, a pbr rule would send the packets via the tunnel instead of the ISP router. I would advise to start a new topic where configs can be shared.

1 Like

Great work, I did not expect my transition to be this smooth). I did however notice that when a domain that has a DNS record of 127.0.0.1 is at the list, it causes the PBR to throw out an error (which wasnt the case with VPN PBR iirc, though I might be wrong).

If you care to elaborate on your message with the config and the error thrown, there's a good chance it will get at least looked at, if not fixed. :wink:

0.9.9-14: WebUI updates after service control button clicks. No major updates to the init script.

Hey Trendy,

That is exactly what I figured, I've started a new thread if you wouldn't mind taking a look.

1 Like

great - thank you - happy to help

a bit more information that might be useful. I could get ipv6 working correctly on 0.9.9-10 by adding the v6 addresses explicitly, but something seems to have changed in 0.9.9-14 which causes the rules to generate incorrectly


  1. when ipv6 routing fails, my blocking rule was using just the device name. in this case nft list ruleset gave a stanza for pbr_van_6_src_ip with an empty elements line, while correctly looking up the v4 IP for me.
table inet fw4 {
	set pbr_van_4_src_ip_cfg026ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 192.168.1.208 }
	}

	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
	}
  1. when I explicitly added all the ipv6 addresses of the device to the rule, on 0.9.9-10, then (with masq6 enabled at least) it worked correctly - blocking the device's ipv6 and forcing it onto v4 on the second connection.

  2. however, on 0.9.9-14 the same rule gives an error for each ipv6 address manually added thus

nft 'add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 goto pbr_mark_0x020000 comment "C2VQF93F24"'
nft 'add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1:892c:c654:198e:d3ea goto pbr_mark_0x020000 comment "C2VQF93F24"'
nft 'add rule inet fw4 pbr_prerouting ip saddr fdc9:a26:3e41:0:1c9e:966c:7c49:733d goto pbr_mark_0x020000 comment "C2VQF93F24"'

this in turn seems to correspond to this part of nft list ruleset which is adding the IPs explicitly rather than referencing the @pbr_van defined earlier

	chain pbr_prerouting {
		ip saddr @pbr_van_4_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr @pbr_van_6_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr 2001:8b0:dc1b:1da1:892c:c654:198e:d3ea goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr fdc9:a26:3e41:0:1c9e:966c:7c49:733d goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip saddr @pbr_van_4_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr @pbr_van_6_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
	}

so to summarise - it looks like it will be possible to get v6 to v4 re-routing working, but

  • is it possible that the rule generation code isn't looking up ipv6 addresses of devices correctly, leading to a empty elements tag in @pbr_van_6?
  • was there something changed in 0.9.9-14 which would lead to explicit ipv6 addresses in a rule to cause rule generation to be incorrect?

Not much has changed for pbr.init from an even earlier commit than 0.9.9-10 and 0.9.9-14.

If there are errors or things not working as expected, there's entire section in the README on what you need to provide.

OK - here's the info as requested in the ReadMe - sorry I hadn't spotted that. (Btw - ReadMe says "the output of /etc/init.d/pbr support" but the help for pbr said /etc/init.d/pbr status so I used that.)

Report Info
  • content of /etc/config/dhcp
config dnsmasq
	option domainneeded '1'
	option boguspriv '1'
	option filterwin2k '0'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option nonegcache '0'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option nonwildcard '1'
	option localservice '1'
	option ednspacket_max '1232'
	option cachesize '10000'
	list notinterface 'pppoe-wan'
	list server '127.0.0.1#5053'
	list server '/mask.icloud.com/'
	list server '/mask-h2.icloud.com/'
	list server '/use-application-dns.net/'
	list server '127.0.0.1#5054'
	option doh_backup_noresolv '-1'
	option noresolv '1'
	list doh_backup_server '127.0.0.1#5053'
	list doh_backup_server '/mask.icloud.com/'
	list doh_backup_server '/mask-h2.icloud.com/'
	list doh_backup_server '/use-application-dns.net/'
	list doh_backup_server '127.0.0.1#5054'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	option ra_slaac '1'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'
	option leasetime '2h'
	option ra_default '1'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'
  • content of /etc/config/firewall
config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	list network 'vodem'
	list network 'van'
	list network 'modem'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option masq6 '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config redirect 'dns_int'
	option name 'Intercept-DNS'
	option src 'lan'
	option src_dport '53'
	option proto 'tcp udp'
	option target 'DNAT'
	option family 'all'
	option enabled '1'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/pbr.firewall.include'
  • content of /etc/config/network
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdc9:0a26:3e41::/48'

config device
	option name 'br-lan'
	option type 'bridge'
	option ports 'eth2'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'eth0'
	option proto 'pppoe'
	option username '********'
	option password '********'
	option pppd_options 'debug'
	option ipv6 '1'
	option metric '10'
	option peerdns '0'
	list dns '45.90.28.123'
	list dns '45.90.30.123'

config interface 'wan6'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'
	option peerdns '0'
	list dns '2a07:a8c0::31:527d'
	list dns '2a07:a8c1::31:527d'
	option device 'pppoe-wan'

config interface 'van'
	option device 'eth1'
	option proto 'dhcp'
	option ipv6 '0'
	option peerdns '0'
	list dns '45.90.28.181'
	list dns '45.90.30.181'
	option metric '20'

config interface 'modem'
	option device 'eth0'
	option proto 'static'
	option ipaddr '192.168.2.2'
	option netmask '255.255.255.0'

config interface 'vodem'
	option proto 'static'
	option ipaddr '192.168.100.20'
	option netmask '255.255.255.0'
	option device 'eth1'
  • content of /etc/config/pbr
config pbr 'config'
	option verbosity '2'
	option resolver_set 'none'
	list ignored_interface 'vpnserver'
	list ignored_interface 'wgserver'
	option boot_timeout '30'
	option rule_create_option 'add'
	option procd_reload_delay '1'
	list webui_supported_protocol 'all'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	option strict_enforcement '1'
	option ipv6_enabled '1'
	option enabled '1'

config policy
	option name 'C2VQF93F24'
	option interface 'van'
	option src_addr 'C2VQF93F24 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 2001:8b0:dc1b:1da1:6c61:5686:72b0:4ae1 fdc9:a26:3e41:0:1c9e:966c:7c49:733d 2001:8b0:dc1b:1da1::2d7'
  • the output of /etc/init.d/pbr status
# /etc/init.d/pbr status -d
============================================================
pbr - environment
pbr 0.9.9-14 running on OpenWrt 22.03.2. WAN (IPv4): van/eth1/86.16.56.1. WAN (IPv6): wan6/pppoe-wan/2001:8b0:1111:1111:0:ffff:51bb:a8f9.
============================================================
Dnsmasq version 2.86  Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
	chain pbr_forward {
	}
	chain pbr_input {
	}
	chain pbr_output {
	}
	chain pbr_prerouting {
		ip saddr @pbr_van_4_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr @pbr_van_6_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr 2001:8b0:dc1b:1da1:6c61:5686:72b0:4ae1 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr fdc9:a26:3e41:0:1c9e:966c:7c49:733d goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr 2001:8b0:dc1b:1da1::2d7 goto pbr_mark_0x020000 comment "C2VQF93F24"
	}
	chain pbr_postrouting {
	}
============================================================
pbr chains - marking
	chain pbr_mark_0x010000 {
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000
		return
	}
	chain pbr_mark_0x020000 {
		counter packets 1042 bytes 147893 meta mark set meta mark & 0xff02ffff | 0x00020000
		return
	}
============================================================
pbr nft sets
	set pbr_van_4_src_ip_cfg026ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 192.168.1.208 }
	}
	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 2001:8b0:dc1b:1da1::2d7,
			     2001:8b0:dc1b:1da1:1c50:a07d:4155:2869,
			     2001:8b0:dc1b:1da1:6c61:5686:72b0:4ae1,
			     fdc9:a26:3e41:0:1c9e:966c:7c49:733d }
	}
  • the output of /etc/init.d/pbr reload with verbosity setting set to 2
Activating traffic killswitch [βœ“]
Setting up routing for 'wan/pppoe-wan/81.187.81.187/2001:8b0:1111:1111:0:ffff:51bb:a8f9/128
fe80::3c14:6728:be16:7f79' [βœ“]
Setting up routing for 'van/eth1/86.16.56.1/2001:8b0:1111:1111:0:ffff:51bb:a8f9/128
fe80::3c14:6728:be16:7f79' [βœ“]
Routing 'C2VQF93F24' via van [βœ—]
Deactivating traffic killswitch [βœ“]
pbr 0.9.9-14 monitoring interfaces: wan 
pbr 0.9.9-14 (nft) started with gateways:
wan/pppoe-wan/81.187.81.187/2001:8b0:1111:1111:0:ffff:51bb:a8f9/128
fe80::3c14:6728:be16:7f79 [βœ“]
van/eth1/86.16.56.1/2001:8b0:1111:1111:0:ffff:51bb:a8f9/128
fe80::3c14:6728:be16:7f79
ERROR: nft 'add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 goto pbr_mark_0x020000 comment "C2VQF93F24"'
ERROR: nft 'add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1:6c61:5686:72b0:4ae1 goto pbr_mark_0x020000 comment "C2VQF93F24"'
ERROR: nft 'add rule inet fw4 pbr_prerouting ip saddr fdc9:a26:3e41:0:1c9e:966c:7c49:733d goto pbr_mark_0x020000 comment "C2VQF93F24"'
ERROR: nft 'add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1::2d7 goto pbr_mark_0x020000 comment "C2VQF93F24"'

So I think I'm actually reporting two problems - the first occurred only after the move to 0.9.9-14 but the second is the important one for my use case and exists on both versions.

  1. On 0.9.9-14 when I create a rule explicitly containing the IPv6 addresses of a device, as well as the device name, routing works correctly (the device is forced onto van and IPv4) and nft list ruleset gives a stanza correctly populated with the ipv6 addresses
	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 2001:8b0:dc1b:1da1::2d7,
			     2001:8b0:dc1b:1da1:1c50:a07d:4155:2869,
			     2001:8b0:dc1b:1da1:6c61:5686:72b0:4ae1,
			     fdc9:a26:3e41:0:1c9e:966c:7c49:733d }
	}

However with this rule I also get Service Errors in the luci app of this form for each v6 address

nft 'add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 goto pbr_mark_0x020000 comment "C2VQF93F24"'

Entering this command at the CLI gives

Error: Could not resolve hostname: Name does not resolve
add rule inet fw4 pbr_prerouting ip saddr 2001:8b0:dc1b:1da1::2d7 goto pbr_mark_0x020000 comment "C2VQF93F24"
                                          ^^^^^^^^^^^^^^^^^^^^^^^

I think this is because the generated command should be ip6 saddr not ip saddr?

  1. Contrariwise, if I create a rule just using the device name C2VQF93F24 but no explicit IP addresses, then there are no errors, but the v6 addresses of the device are not added to the firewall rules and the elements section of the v6 stanza is left unpopulated like this:
	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
	}

I think the main problem is the second one - adding a device name to the rule would ideally populate the pbr_van_6_src_ip_cfg026ff5 fully in the same way as it does for v4.

What I think happened is that the CLI has been showing up IPv4 errors for IPv6 resources for a while, but since the WebUI is cached by the browser, it was only after upgrade to 0.9.9-14 that the WebUI updated in your browser and started showing the errors.

Either way, 0.9.9-15 has updated error handling and should no longer display IPv4 errors for IPv6 resources.

For the second issue, please post the config and the output of reload/status with that config.

Super - thanks - yes the errors are gone with 0.9.9-15, thank you.

For the other issue here are the config and status. The status shows that when given the device name, pbr is successfully grabbing the IPv4 address, but not the v6 addresses.

A workaround for now is to use MAC address as the identifier for the machine instead which works well, but am still flagging this in case it's useful.

There's still the question of why pbr is reporting IPv6 addresses on the v4 interface, but that's a different question again I guess - happens with this config though.

Config and Status
  • Content of /etc/config/pbr
config pbr 'config'
	option verbosity '2'
	option resolver_set 'none'
	list ignored_interface 'vpnserver'
	list ignored_interface 'wgserver'
	option boot_timeout '30'
	option rule_create_option 'add'
	option procd_reload_delay '1'
	list webui_supported_protocol 'all'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	option strict_enforcement '1'
	option ipv6_enabled '1'
	option enabled '1'

config policy
	option name 'C2VQF93F24'
	option interface 'van'
	option src_addr 'C2VQF93F24'
  • Output of /etc/init.d/pbr status
============================================================
pbr - environment
pbr 0.9.9-15 running on OpenWrt 22.03.2. WAN (IPv4): van/eth1/86.16.56.1. WAN (IPv6): wan6/pppoe-wan/2001:8b0:1111:1111:0:ffff:51bb:a8f9.
============================================================
Dnsmasq version 2.86  Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
	chain pbr_forward {
	}
	chain pbr_input {
	}
	chain pbr_output {
	}
	chain pbr_prerouting {
		ip saddr @pbr_van_4_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr @pbr_van_6_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
	}
	chain pbr_postrouting {
	}
============================================================
pbr chains - marking
	chain pbr_mark_0x010000 {
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000
		return
	}
	chain pbr_mark_0x020000 {
		counter packets 320 bytes 57284 meta mark set meta mark & 0xff02ffff | 0x00020000
		return
	}
============================================================
pbr nft sets
	set pbr_van_4_src_ip_cfg026ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 192.168.1.81 }
	}
	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
	}

The 0.9.9-16 should resolve the local name to both IPv4 and IPv6 addresses, however each local name encountered in the policy will cause two calls to resolveip so it will be a bit slower to process for now.

In the future I may try to improve performance on this. Speaking of, do you see all the IPv6 addresses for C2VQF93F24 in the /tmp/dhcp.leases file? Or is there any other local file where IPv6 assignments are stored? It would be much faster for me to parse a local file than initiate two resolveip calls.

Thanks, 0.9.9-16 does indeed get the IPv6 addresses of the machine in my setup, as shown in pbr status

============================================================
pbr nft sets
	set pbr_van_4_src_ip_cfg026ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 192.168.1.81 }
	}
	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 2001:8b0:dc1b:1da1::81,
			     fdc9:a26:3e41::81 }
	}

Do you see all the IPv6 addresses for C2VQF93F24 in the /tmp/dhcp.leases file? Or is there any other local file where IPv6 assignments are stored?

Yes the v6 addresses seem to be in /tmp/hosts/odhcpd

# cat /tmp/hosts/odhcpd | grep C2VQF93F24
2001:8b0:dc1b:1da1::81	C2VQF93F24
fdc9:a26:3e41::81	C2VQF93F24
# br-lan 000100012ae99da9c889f3db0be8 0 C2VQF93F24 1667735114 81 128 2001:8b0:dc1b:1da1::81/128 fdc9:a26:3e41::81/128

FWIW the Mac (running Ventura) actually choses to have a few more ipv6 addresses as shown by ifconfig output below (the ones labelled secured), but it doesn't get those by dhcp so I guess there's no way the router can know about them. Something to do with IPv6 Privacy Extensions I think. Furthermore the Mac isn't using one of the ones that odhcpd thinks it is (fdc9:a26:3e41::81)

So in the end, as you say, MAC address are probably a more elegant way to setup rules for combined IPv6 and v4 routing, but thank you for helping track this down.

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether c8:89:f3:db:0b:e8 
	inet6 fe80::80a:8b00:f78c:95c%en0 prefixlen 64 secured scopeid 0xf 
	inet 192.168.1.81 netmask 0xffffff00 broadcast 192.168.1.255
	inet6 2001:8b0:dc1b:1da1:1c50:a07d:4155:2869 prefixlen 64 autoconf secured 
	inet6 2001:8b0:dc1b:1da1:54eb:8f6c:1ad6:1cb7 prefixlen 64 autoconf temporary 
	inet6 fdc9:a26:3e41:0:1c9e:966c:7c49:733d prefixlen 64 autoconf secured 
	inet6 2001:8b0:dc1b:1da1::81 prefixlen 64 dynamic 

Before you switch to using mac address, if you could test 0.9.9-17, I've replaced uses of resolveip to parsing local leases files to obtain IPv4/IPv6 addresses for policies with device names. Even on an x86 box there's a perceivable difference between the two.

I hope it should also stop reporting ipv6 address for your van interface.

I hope it should also stop reporting ipv6 address for your van interface.

Indeed it does!

But sadly elements is empty for v6 again :neutral_face:

# /etc/init.d/pbr status
============================================================
pbr - environment
pbr 0.9.9-17 running on OpenWrt 22.03.2. WAN (IPv4): van/eth1/86.16.56.1. WAN (IPv6): wan6/pppoe-wan/2001:8b0:1111:1111:0:ffff:51bb:a8f9.
============================================================
Dnsmasq version 2.86  Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
	chain pbr_forward {
	}
	chain pbr_input {
	}
	chain pbr_output {
	}
	chain pbr_prerouting {
		ip saddr @pbr_van_4_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
		ip6 saddr @pbr_van_6_src_ip_cfg026ff5 goto pbr_mark_0x020000 comment "C2VQF93F24"
	}
	chain pbr_postrouting {
	}
============================================================
pbr chains - marking
	chain pbr_mark_0x010000 {
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000
		return
	}
	chain pbr_mark_0x020000 {
		counter packets 211 bytes 52919 meta mark set meta mark & 0xff02ffff | 0x00020000
		return
	}
============================================================
pbr nft sets
	set pbr_van_4_src_ip_cfg026ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
		elements = { 192.168.1.81 }
	}
	set pbr_van_6_src_ip_cfg026ff5 {
		type ipv6_addr
		flags interval
		auto-merge
		comment "C2VQF93F24"
	}

The addresses are in /tmp/hosts/odhcpd though...

# cat /tmp/hosts/odhcpd | grep C2VQF93F24
2001:8b0:dc1b:1da1::81	C2VQF93F24
fdc9:a26:3e41::81	C2VQF93F24
# br-lan 000100012ae99da9c889f3db0be8 0 C2VQF93F24 1667773720 81 128 2001:8b0:dc1b:1da1::81/128 fdc9:a26:3e41::81/128