Pbr rules do not work

Installing and Using OpenWrt
1

Hello,

I have an Asus Tuf AX6000 with the latest stable OpenWrt version (OpenWrt 24.10.0 r28427-6df0e3d02a / LuCI openwrt-24.10 branch 25.080.48760~19939dd).

I have installed PBR 1.1.8-r10 and luci-app-pbr.

My default WAN interface is WAN. Proton0 has a lower priority.

Now, here's what I intended to do: Half of my network should be routed through WAN.

The other half should go through the Proton WireGuard VPN.

To achieve this, I created a PBR rule: From 192.168.178.128/25 to proton0.

Then, I set up two DNS policies for:

192.168.178.0/25 to use public DNS

192.168.178.128/25 to use Proton VPN DNS.

The IPs from 128 onward are assigned via DHCP, meaning that every device will use the Proton VPN by default. If I want a device to use WAN directly, I assign it an IP <128.

This setup works fine.

However, I noticed that some websites are incompatible with the VPN for devices that should still be routed through it. I wanted to exclude these websites for VPN clients. So, I thought about replacing my dnsmasq with dnsmasq-full (according to the wiki). Then, I created a rule to route my VPN IP range to WAN when the destination is domain.com. I placed this rule before the other one in PBR.

Now, I'm not even sure if PBR is working correctly. When I run pbr status, all the routing nft rules are displayed, and they seem correct. But when I run ip rule show, it always shows the proton0 interface with a lower value and WAN with a higher one. However, nothing changes, even if I disable all PBR rules. ip rule show stays the same.

I also tried the PBR rule without DNS resolve, using an IP as the destination. Here, too, the rule doesn't work (I tried it with wieistmeineip.de, but it still shows the Proton VPN external IP).

I simply want to route certain destination domains through WAN. Is this not possible? Or is something wrong with my configuration? I sometimes feel like no matter what I set, it doesn't update anything. I've restarted the router, network, and PBR multiple times, but nothing changes. The output of pbr status also shows different priorities than what is shown in ip rule show.

The routing for half of my network (IPs starting from 128) through the VPN and the IPs <128 through WAN works absolutely reliably without DNS leaks.

If more logs are needed, I apologize.

Thanks for the help!

Best regards,
Sebastian

System-Summary:

pbr	1.1.8-r10
dnsmasq-full	2.90-r4
OpenWrt 24.10.0 r28427-6df0e3d02a / LuCI openwrt-24.10 branch 25.080.48760~19939dd
ASUS TUF-AX6000

ip rule show:

0:      from all lookup local
29998:  from all fwmark 0x20000/0xff0000 lookup pbr_proton0
30000:  from all fwmark 0x10000/0xff0000 lookup pbr_wan
32766:  from all lookup main
32767:  from all lookup default

pbr status:

pbr - environment
pbr 1.1.8-r10 running on OpenWrt 24.10.0.

Dnsmasq version 2.90  Copyright (c) 2000-2024 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack no-ipset nftset auth cryptohash DNSSEC no-ID loop-detect inotify dumpfile

pbr fw4 nft file: /usr/share/nftables.d/ruleset-post/30-pbr.nft
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000  mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000  mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add set inet fw4 pbr_wan_4_dst_ip_cfg046ff5 { type ipv4_addr;            auto-merge;                             flags interval;                                 policy performance;                             comment "Ausnahme";}
add rule inet fw4 pbr_prerouting ip saddr { 192.168.178.128/25 } ip daddr @pbr_wan_4_dst_ip_cfg046ff5 tcp sport { 80, 443 } tcp dport { 80, 443 }  goto pbr_mark_0x010000 comment "Ausnahme"
add set inet fw4 pbr_wan_4_dst_ip_cfg046ff5 { type ipv4_addr;            auto-merge;                             flags interval;                                 policy performance;                             comment "Ausnahme";}
add rule inet fw4 pbr_prerouting ip saddr { 192.168.178.128/25 } ip daddr @pbr_wan_4_dst_ip_cfg046ff5 udp sport { 80, 443 } udp dport { 80, 443 }  goto pbr_mark_0x010000 comment "Ausnahme"
add rule inet fw4 pbr_prerouting ip saddr { 192.168.178.128/25 }  goto pbr_mark_0x020000 comment "Use VPN"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.178.128/25 }  meta nfproto ipv4 tcp dport 53 dnat ip to 10.2.0.1:53 comment "VPN DNS"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.178.128/25 }  meta nfproto ipv4 udp dport 53 dnat ip to 10.2.0.1:53 comment "VPN DNS"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.178.0/25 }  meta nfproto ipv4 tcp dport 53 dnat ip to 9.9.9.9:53 comment "Public"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.178.0/25 }  meta nfproto ipv4 udp dport 53 dnat ip to 9.9.9.9:53 comment "Public"

pbr chains - policies
        chain pbr_forward { # handle 37
        }
        chain pbr_input { # handle 38
        }
        chain pbr_output { # handle 39
        }
        chain pbr_postrouting { # handle 41
        }
        chain pbr_prerouting { # handle 40
                ip saddr 192.168.178.128/25 ip daddr @pbr_wan_4_dst_ip_cfg046ff5 tcp sport { 80, 443 } tcp dport { 80, 443 } goto pbr_mark_0x010000 comment "Ausnahme" # handle 10155
                ip saddr 192.168.178.128/25 ip daddr @pbr_wan_4_dst_ip_cfg046ff5 udp sport { 80, 443 } udp dport { 80, 443 } goto pbr_mark_0x010000 comment "Ausnahme" # handle 10158
                ip saddr 192.168.178.128/25 goto pbr_mark_0x020000 comment "Use VPN" # handle 10159
        }
        chain pbr_dstnat { # handle 36
                ip saddr 192.168.178.128/25 meta nfproto ipv4 tcp dport 53 dnat ip to 10.2.0.1:53 comment "VPN DNS" # handle 10160
                ip saddr 192.168.178.128/25 meta nfproto ipv4 udp dport 53 dnat ip to 10.2.0.1:53 comment "VPN DNS" # handle 10161
                ip saddr 192.168.178.0/25 meta nfproto ipv4 tcp dport 53 dnat ip to 9.9.9.9:53 comment "Public" # handle 10162
                ip saddr 192.168.178.0/25 meta nfproto ipv4 udp dport 53 dnat ip to 9.9.9.9:53 comment "Public" # handle 10163
        }

pbr chains - marking
        chain pbr_mark_0x010000 { # handle 9702
                meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 10149
                return # handle 10150
        }
        chain pbr_mark_0x020000 { # handle 9705
                meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 10151
                return # handle 10152
        }

pbr nft sets
        set pbr_wan_4_dst_ip_cfg046ff5 { # handle 9708
                type ipv4_addr
                flags interval
                auto-merge
                comment "Ausnahme"
        }

dnsmasq sets
nftset=/lindt.de/4#inet#fw4#pbr_wan_4_dst_ip_cfg046ff5 # Ausnahme
nftset=/lindt.de/4#inet#fw4#pbr_wan_4_dst_ip_cfg046ff5 # Ausnahme

/etc/config/pbr

config pbr 'config'
        option enabled '1'
        option verbosity '2'
        option strict_enforcement '1'
        option resolver_set 'dnsmasq.nftset'
        list resolver_instance '*'
        option ipv6_enabled '0'
        option boot_timeout '30'
        option rule_create_option 'add'
        option procd_reload_delay '1'
        option webui_show_ignore_target '0'
        option nft_rule_counter '0'
        option nft_set_auto_merge '1'
        option nft_set_counter '0'
        option nft_set_flags_interval '1'
        option nft_set_flags_timeout '0'
        option nft_set_policy 'performance'
        list webui_supported_protocol 'all'
        list webui_supported_protocol 'tcp'
        list webui_supported_protocol 'udp'
        list webui_supported_protocol 'tcp udp'
        list webui_supported_protocol 'icmp'
        option dns_resolve_ip '1'
        list supported_interface 'proton0'
        list supported_interface 'wan'
        list ignored_interface 'vpn_incoming'

config include
        option path '/usr/share/pbr/pbr.user.aws'
        option enabled '0'

config include
        option path '/usr/share/pbr/pbr.user.netflix'
        option enabled '0'

config policy
        option name 'Ausnahme'
        option dest_addr 'lindt.de'
        option interface 'wan'
        option src_port '80 443'
        option dest_port '80 443'
        option src_addr '192.168.178.128/25'

config policy
        option name 'Use VPN'
        option src_addr '192.168.178.128/25'
        option interface 'proton0'

config dns_policy
        option name 'VPN DNS'
        option src_addr '192.168.178.128/25'
        option dest_dns '10.2.0.1'

config dns_policy
        option name 'Public'
        option src_addr '192.168.178.0/25'
        option dest_dns '9.9.9.9 1.1.1.1 8.8.8.8'

/etc/config/network:

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdaa:bfe2:c17a::/48'
        option packet_steering '1'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'
        list ports 'lan5'
        option ipv6 '0'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.178.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option device 'eth1'
        option proto 'dhcp'
        option peerdns '0'
        option metric '1'
        list dns '9.9.9.9'
        list dns '1.1.1.1'
        list dns '8.8.8.8'

config interface 'wan6'
        option device 'eth1'
        option proto 'dhcpv6'

config interface 'proton0'
        option proto 'wireguard'
        option private_key 'xxxxx'
        list addresses '10.2.0.2/32'
        option metric '2'

config wireguard_proton0
        option description 'Asus_Router-DE-319.conf'
        option public_key 'xxxxxx'
        list allowed_ips '0.0.0.0/0'
        option endpoint_host '149.88.19.225'
        option endpoint_port '51820'
        option route_allowed_ips '1'

config interface 'vpn_incoming'
        option proto 'wireguard'
        option private_key 'xxxx'
        option listen_port '553'
        list addresses '192.168.9.1/24'
        list addresses 'fd00:9::1/64'

config wireguard_vpn_incoming 'wgclient'
        option public_key 'xxx'
        option preshared_key 'xxxx'
        list allowed_ips '192.168.9.2/32'
        list allowed_ips 'fd00:9::2/128'
        option description 'Sebastian_VPN'
        option private_key 'xxxx'
        option endpoint_port '553'
        option persistent_keepalive '25'

/etc/config/firewall:

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'

config zone 'lan'
	option name 'wan'
	option input 'DROP'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list device 'wg0'
	list network 'wan'
	list network 'wan6'

config zone 'wan'
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option mtu_fix '1'
	list network 'lan'
	list network 'vpn_incoming'

config rule
	option name 'Kindersicherung Ausnahmen'
	option src 'lan'
	option dest '*'
	option target 'ACCEPT'
	list proto 'all'
	list src_mac '60:CF:84:B0:24:BE'
	list src_mac '0C:CA:FB:6B:DA:F4'
	list src_mac '5C:85:7E:3A:F2:2B'
	list src_mac '50:F2:65:BD:BA:55'
	list src_mac '60:74:F4:97:25:84'
	list src_mac '60:74:F4:EC:0D:A4'
	list src_mac '60:74:F4:F2:56:B0'
	list src_mac '60:74:F4:FA:07:42'
	list src_mac '60:F2:65:CC:20:C6'
	list src_mac '68:F6:3B:23:A5:9D'
	list src_mac '74:56:3C:9B:E2:27'
	list src_mac '78:D8:40:80:0D:18'
	list src_mac '98:B6:E9:FE:53:23'
	list src_mac '98:FA:2E:55:7C:20'
	list src_mac 'A0:85:FC:E3:C3:B2'
	list src_mac 'C8:4A:A0:70:49:B8'
	list src_mac 'D0:C9:07:3B:BB:7A'
	list src_mac 'D8:80:83:9D:40:BB'
	list src_mac 'E8:A7:2F:58:E6:5C'
	list src_mac 'E8:B2:FE:9A:33:C6'
	list src_mac '78:D8:40:80:13:07'
	list src_mac 'D0:C9:07:30:57:74'
	list src_mac '70:2C:09:E9:40:4B'

config rule
	option name 'Kindersicherung Wochenende'
	option src 'lan'
	option dest '*'
	option target 'ACCEPT'
	option start_time '05:20:00'
	option stop_time '23:59:59'
	option weekdays 'Fri Sat'
	list proto 'all'

config rule
	option name 'Kindersicherung'
	option src 'lan'
	option dest '*'
	option target 'REJECT'
	option start_time '23:00:00'
	option stop_time '06:00:00'
	list proto 'all'

config rule 'wg'
	option name 'Allow-WireGuard'
	option src 'wan'
	option dest_port '553'
	option proto 'udp'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config zone
	option name 'vpn'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	list network 'proton0'

config forwarding
	option src 'lan'
	option dest 'vpn'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/firewall.include'

pbr restart output:

Resetting chains and sets [âś“]
Removing routing for 'wan/eth1/95.208.38.1' [âś“]
Removing routing for 'proton0/10.2.0.2' [âś“]
pbr 1.1.8-r10 (fw4 nft file mode) stopped [âś“]
Using wan interface (on_start): wan [âś“]
Found wan gateway (on_start): 95.208.38.1 [âś“]
Setting up routing for 'wan/eth1/95.208.38.1' [âś“]
Setting up routing for 'proton0/10.2.0.2' [âś“]
Routing 'Ausnahme' via wan [âś“]
Routing 'Use VPN' via proton0 [âś“]
Routing 'VPN DNS' DNS to 10.2.0.1 [âś“]
Routing 'Public' DNS to 9.9.9.9 1.1.1.1 8.8.8.8 [âś“]
Installing fw4 nft file [âś“]
Setting interface trigger for wan [âś“]
Setting interface trigger for proton0 [âś“]

pbr 1.1.8-r10 monitoring interfaces: wan proton0
pbr 1.1.8-r10 (fw4 nft file mode) started with gateways:
wan/eth1/95.208.38.1 [âś“]
proton0/10.2.0.2
2

Remove option src_port '80 443' from the rule, restart the pbr service and test again.

1 Like
3

To test add to option dest_addr 'lindt.de' e.g. ipleak.net

Edit: you have a somewhat unusual setup with a metric set on the WireGuard interface (not wrong though).
If you want the default route to stay at the WAN interface then the ususal way to do that (and the one with PBR works best and which is described in the PBR read.me) is to disable Route Allowed IPs. If you do that you can remove the metric on the WireGuard interface.

Furthermore I would recommend to add option mtu_fix '1' to the VPN firewall zone which can be useful when having MTU issues

1 Like
4

Hello,

@pavel:
Removing the source port didn’t help.

@egc
Can you explain exactly what I need to configure on which interface? Should I disable “Route allowed IPs” on the proton0 interface? And should I remove the metric there, while keeping it on wan?

Or should I disable “Route allowed IPs” everywhere and let PBR handle everything?

What about the vpn_incoming interface?

As a test, I just disabled “Route allowed IPs” on proton0 and removed the metric. Now I can’t access the router via the vpn_incoming tunnel anymore. But my wife confirmed that the internet at home is still working (phew). I’ll take a closer look later or tomorrow and report back. Thanks a lot for your help!

Btw: Can you explain why my approach isn’t fully working as expected?

Best regards,
Sebastian

5

Remove this form the PBR config

By removing this PBR will take care of routing the incoming interface always via the WAN (not that it is necessary as you want the WAN to be default)

Remove option metric '2':

Remove option route_allowed_ips '1':

You need two PBR policies and possible a DNS policy.
For the first policy and this has to come first, i added ipleak.mnet so that you can test this as it should show the WAN ip address, use e.g. myip.is to show the VPN IP address:

config policy
        option name 'Ausnahme'
        option dest_addr 'lindt.de ipleak.net'
        option interface 'wan'
        option src_addr '192.168.178.128/25'

The second:

config policy
        option name 'Use VPN'
        option src_addr '192.168.178.128/25'
        option interface 'proton0'

Reboot and wait till everything settles and test again.

Note that if you are going to use DNS policies this is not compatible with nftables in that case you have to disable DNSMasq nftset on the PBR setting Use resolver set support for domains if the domains are resolved to a simple IP address this might work

1 Like
6

Thank you very much. I think it works now for ip. Unfortunately i have no such resolver option in pbr, only nft or disabled. What iam missing?

7

nft is the best option but does not work with DNS policies as those will bypass DNSMasq.
So if you are going to use DNS policies you have no choice and use disabled

8

But when i chose disabled and use domain name instead of ip, it does not work. Hmm.

Edit: sorry typo it does work! But only for ipleak.net. lindt.de is not working. Can you tell me whats different with that domain? The multiple A entries?

9

From my earlier post.

10

Sorry! And thanks again! For other domains with 1-2 A entries it works too.

But lindt.de does not work, even if i try it with all the ips instead of the domain. 405 error. Weird but nevermind

11

You can try to reverse the set up, e.g. by default everything via the VPN and see if that works for you.

To do that Enable Route Allowed IPs on the WireGuard peer.

Reboot and test again, now everything should go over the VPN

The incoming server connection should automatically been taken care off, if you removed the vpn_incoming interface from the ignored_interfaces as advised in my earlier post.
You can check with: ip rule show which should show the listen port 553 via the wan.

Now you do not need a DNS policy for your VPN clients and thus can use the nftset for resolving domains and your VPN clients should now work also with lindt.de.

For the Non VPN clients you need a PBR policy to route those via the wan and if necessary a DNS policy to make sure they do not use the DNS via the VPN.

Not sure if everything will work for you but perhaps worth a try

12

I'm just curious – lindt.de is the only site that doesn't work for me. Even when using no VPN but 9.9.9.9 as DNS, it still fails. It seems like they’re doing something unusual. Their homepage makes almost 500 requests, and even on my desktop, many of them get blocked.

Other sites with multiple A records work just fine, and I’ve also verified that there are no DNS leaks.

I don’t mean to sound rude, but is there a specific reason why I should switch everything to reverse? Just trying to understand better. :slight_smile:

13

So you would be able to resolve lindt.de with nftset but if that is not working anyway then of course just leave it as is :slight_smile:

14

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.