Newb having problems with PBR and Mullvad Wireguard client

Rickety8972 · August 30, 2024, 3:09am

Hi all,

I'm a fairly recent adopter of OpenWrt and not very network savvy so the learning curve has been very steep and I'm not 100% sure of what I'm doing.

I've managed to get OpenWrt (23.05.2) flashed onto a GL.iNet MT-1300 (Beryl) travel router and have that connected to my ISP router which is in bridge mode.

I then installed a Wireguard interface following the instructions provided by Mulvad.

This was all working well, but I encountered problems with some sites not working, particularly with services provided through AppleTV - this, of course, was not acceptable to my wife!

Since then I have installed PBR and am trying to get the Mullvad client tunnel working as a separate. I had various issues with this but managed to get it working to some degree even though DNS was leaking when using Wireguard (showing my ISP DNS).

However the PBR Wireguard route is not working at all after I tired to configure OpenWrt to work with Pi-hole running on our local media server. I have had to bypass using that route otherwise there is no internet access (local access is still working).

Basically I have tied myself in knots, following different advice here & there and not really fully understanding what I'm doing. I have tried searching the forum (and wider) but most of the discussions that seem related are too complicated for my level of understanding. It seems those users have much more background knowledge of OpenWrt and networks in general so the solutions are just not simply enough for me to follow.

So I am not really sure where to start!

Maybe it's something easy, and someone can step me through a fix. Or maybe I need to go and do some homework to get my knowledge up - if this is the case please be specific about what I need to understand to get this working.

Thanks in advance

Network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdbd:d479:4af8::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'wan'
	option proto 'pppoe'
	option username 'username@my.isp'
	option password 'password'
	option ipv6 'auto'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'

config interface 'mullvad'
	option proto 'wireguard'
	option force_link '1'
	option private_key 'removed'
	list addresses '10.xx.xx.xx/32'

config wireguard_mullvad
	option description 'endpoint_name'
	option public_key 'removed'
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'xxx.mullvad.net'
	option endpoint_port '51820'
	option route_allowed_ips '0'

PBR

config pbr 'config'
	option enabled '1'
	option verbosity '2'
	option strict_enforcement '1'
	option resolver_set 'none'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver'
	list ignored_interface 'wgserver'
	option boot_timeout '30'
	option rule_create_option 'add'
	option procd_reload_delay '1'
	option webui_show_ignore_target '0'
	list webui_supported_protocol 'all'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'

config include
	option path '/usr/share/pbr/pbr.user.aws'
	option enabled '0'

config include
	option path '/usr/share/pbr/pbr.user.netflix'
	option enabled '0'

config policy
	option name 'Plex/Emby Local Server'
	option interface 'wan'
	option src_port '8096 8920 32400'
	option enabled '0'

config policy
	option name 'Plex/Emby Remote Servers'
	option interface 'wan'
	option dest_addr 'plex.tv my.plexapp.com emby.media app.emby.media tv.emby.media'
	option enabled '0'

config policy
	option src_addr '192.168.1.125'
	option interface 'mullvad'
	option name 'test_wireguard'
	option enabled '1'

Firewall

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'wan6'
	list network 'mullvad'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/pbr.firewall.include'

config zone
	option name 'wgzone'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'wgzone'
	option dest 'wan'

DHCP

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option cachesize '1000'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'
	list dhcp_option '6,192.168.1.200'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config host
	option name 'printer'
	option ip '192.168.1.192'
	option mac 'xx:xx:xx:xx:xx:xx'

config dhcp 'mullvad'
	option interface 'mullvad'
	option ignore '1'

(edited to fix link to Mullvad blog)

brada4 · November 24, 2024, 12:35pm

Please post outputs of

ubus call system board
wg status

editing away private keys

What is 192.168.1.200 ?

egc · November 24, 2024, 1:50pm

What happens if you remove the above, then reboot and test again?

Your WG and PBR look ok (you can remove the wgzone as that is not used it seems)

stangri · November 24, 2024, 7:02pm

Because you've explicitly stated you're a newb, even tho there're links to the README in both principal package and the luci app, make sure to follow README->Getting Help.

I would venture a guess that the device you're targeting by the ip address in the pbr policy, has a different ip.

Rickety8972 · November 25, 2024, 6:58am

Hi @brada4 ... thanks for the reply

ubus call system board

{
	"kernel": "5.15.137",
	"hostname": "OpenWrt",
	"system": "MediaTek MT7621 ver:1 eco:3",
	"model": "GL.iNet GL-MT1300",
	"board_name": "glinet,gl-mt1300",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "23.05.2",
		"revision": "r23630-842932a63d",
		"target": "ramips/mt7621",
		"description": "OpenWrt 23.05.2 r23630-842932a63d"
	}
}

wg status ... Invalid subcommand: 'status'

wg show

interface: wginterface
  public key: wKDk0LyJPu4Y84nyQrPtgRczoQlJiH3xSZajYR8XIFE=
  private key: (hidden)
  listening port: 32884

peer: z+JG0QA4uNd/wRTpjCqn9rDpQsHKhf493omqQ5rqYAc=
  endpoint: [2404:f780:4:dec::a02f]:51820
  allowed ips: 0.0.0.0/0

Rickety8972 · November 25, 2024, 7:05am

Thanks @egc

I actually thought I wasn't going to get any replies on this because it's been a while ... ended up playing with it again over the weekend, with mixed success. Because of this I've re-posted the relevant configs again in my reply to @stangri below

I have removed the '6,192.168.1.200' ... still having issues.

This was an attempt to have DNS designated by Pihole which is setup on a machine with that IP.

Rickety8972 · November 25, 2024, 7:57am

Hi @stangri ... thanks for your reply

As I said in one of the other replies, I've messed around with this some more and am still having issues. Because I may have changed things compared to my first post I thought I'd post all my configs again, after setting the counters & verbosity as per the 'Getting Help' section of the README (sorry about not following that previously).

dhcp

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option cachesize '1000'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
	list server '10.64.0.1'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'
	option start '100'
	option limit '150'
	option leasetime '12h'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config host
	option name 'SEC30CDA79A16DE'
	option ip '192.168.1.192'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option ip '192.168.1.186'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'umbrel'
	option ip '192.168.1.131'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option ip '192.168.1.156'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option ip '192.168.1.176'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'DEBBIE-DESKTOP'
	option ip '192.168.1.200'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'Living-Room'
	option ip '192.168.1.149'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'stlnx'
	option ip '192.168.1.187'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'iMac'
	option ip '192.168.1.158'
	option mac 'XX:XX:XX:XX:XX:XX'

firewall

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'wan6'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/pbr.firewall.include'

config zone
	option name 'wgzone'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wginterface'

config forwarding
	option src 'wgzone'
	option dest 'wan'

config forwarding
	option src 'lan'
	option dest 'wan'

network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdbd:d479:4af8::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'

config interface 'wginterface'
	option proto 'wireguard'
	option private_key 'removed'
	list addresses '10.70.156.54/32'
	option force_link '1'

config wireguard_wginterface
	option description 'host name removed'
	option public_key 'removed'
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'xxx.relays.mullvad.net'
	option endpoint_port '51820'

pbr

config pbr 'config'
	option enabled '1'
	option verbosity '2'
	option strict_enforcement '1'
	option resolver_set 'none'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver'
	list ignored_interface 'wgserver'
	option boot_timeout '30'
	option rule_create_option 'add'
	option procd_reload_delay '1'
	option webui_show_ignore_target '0'
	list webui_supported_protocol 'all'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	option nft_rule_counter '1'
	option nft_set_counter '1'

config include
	option path '/usr/share/pbr/pbr.user.aws'
	option enabled '0'

config include
	option path '/usr/share/pbr/pbr.user.netflix'
	option enabled '0'

config policy
	option name 'Plex/Emby Local Server'
	option interface 'wan'
	option src_port '8096 8920 32400'
	option enabled '0'

config policy
	option name 'Plex/Emby Remote Servers'
	option interface 'wan'
	option dest_addr 'plex.tv my.plexapp.com emby.media app.emby.media tv.emby.media'
	option enabled '0'

config policy
	option src_addr '192.168.1.125'
	option interface 'mullvad'
	option name 'stlnx_wireguard'
	option enabled '0'

config policy
	option name 'wifi_test'
	option interface 'wginterface'
	option src_addr '192.168.1.159'

/etc/init.d/pbr status

============================================================
pbr - environment
pbr 1.1.1-7 running on OpenWrt 23.05.2. WAN (IPv4): wan/wan/100.82.168.1.
============================================================
Dnsmasq version 2.89  Copyright (c) 2000-2022 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
	chain pbr_forward { # handle 36
	}
	chain pbr_input { # handle 37
	}
	chain pbr_output { # handle 38
	}
	chain pbr_prerouting { # handle 39
		ip saddr @pbr_wginterface_4_src_ip_cfg076ff5 goto pbr_mark_0x020000 comment "wifi_test" # handle 454
	}
	chain pbr_postrouting { # handle 40
	}
============================================================
pbr chains - marking
	chain pbr_mark_0x010000 { # handle 447
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 448
		return # handle 449
	}
	chain pbr_mark_0x020000 { # handle 450
		counter packets 222 bytes 16869 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 451
		return # handle 452
	}
============================================================
pbr nft sets
	set pbr_wginterface_4_src_ip_cfg076ff5 { # handle 453
		type ipv4_addr
		flags interval
		counter
		auto-merge
		comment "wifi_test"
		elements = { 192.168.1.159 counter packets 222 bytes 16869 }
	}
============================================================
IPv4 table 256 route: default via 100.82.168.1 dev wan 
IPv4 table 256 rule(s):
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 route: default via 10.70.156.54 dev wginterface 
IPv4 table 257 rule(s):
30001:	from all fwmark 0x20000/0xff0000 lookup pbr_wginterface

/etc/init.d/pbr reload

Activating traffic killswitch [✓]
Setting up routing for 'wan/100.82.168.1' [✓]
Setting up routing for 'wginterface/10.70.156.54' [✓]
pbr.cfg066ff5.name=stlnx_wireguard validates as string with true
pbr.cfg066ff5.enabled=0 validates as bool with true
pbr.cfg066ff5.interface=mullvad validates as or("ignore", "tor", uci("network", "@interface")) with false
pbr.cfg066ff5.proto is unset and defaults to or(string) (null)
pbr.cfg066ff5.chain is unset and defaults to or("", "forward", "input", "output", "prerouting", "postrouting", "FORWARD", "INPUT", "OUTPUT", "PREROUTING", "POSTROUTING") prerouting
pbr.cfg066ff5.src_addr=192.168.1.125 validates as list(neg(or(host,network,macaddr,string))) with true
pbr.cfg066ff5.src_port is unset and defaults to list(neg(or(portrange,string))) (null)
pbr.cfg066ff5.dest_addr is unset and defaults to list(neg(or(host,network,string))) (null)
pbr.cfg066ff5.dest_port is unset and defaults to list(neg(or(portrange,string))) (null)
Routing 'wifi_test' via wginterface [✓]
Deactivating traffic killswitch [✓]
pbr 1.1.1-7 monitoring interfaces: wan wginterface 
pbr 1.1.1-7 (nft) started with gateways:
wan/100.82.168.1 [✓]
wginterface/10.70.156.54

/etc/init.d/pbr status

============================================================
pbr - environment
pbr 1.1.1-7 running on OpenWrt 23.05.2. WAN (IPv4): wan/wan/100.82.168.1.
============================================================
Dnsmasq version 2.89  Copyright (c) 2000-2022 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
	chain pbr_forward { # handle 36
	}
	chain pbr_input { # handle 37
	}
	chain pbr_output { # handle 38
	}
	chain pbr_prerouting { # handle 39
		ip saddr @pbr_wginterface_4_src_ip_cfg076ff5 goto pbr_mark_0x020000 comment "wifi_test" # handle 464
	}
	chain pbr_postrouting { # handle 40
	}
============================================================
pbr chains - marking
	chain pbr_mark_0x010000 { # handle 457
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 458
		return # handle 459
	}
	chain pbr_mark_0x020000 { # handle 460
		counter packets 0 bytes 0 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 461
		return # handle 462
	}
============================================================
pbr nft sets
	set pbr_wginterface_4_src_ip_cfg076ff5 { # handle 463
		type ipv4_addr
		flags interval
		counter
		auto-merge
		comment "wifi_test"
		elements = { 192.168.1.159 counter packets 0 bytes 0 }
	}
============================================================
IPv4 table 256 route: default via 100.82.168.1 dev wan 
IPv4 table 256 rule(s):
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 route: default via 10.70.156.54 dev wginterface 
IPv4 table 257 rule(s):
30001:	from all fwmark 0x20000/0xff0000 lookup pbr_wginterface

I am now/again using a separate firewall zone 'wgzone' and the wg interface name has changed to 'wginterface'.

I am currently trying to route a single IP address (my phone - I can confirm that the IP 192.168.1.159 is correct, I check it regularly while playing with this and it has a static lease) via the wg tunnel - this is just to try to get this working with pbr without upsetting the rest of the network and my wife!

Thanks all for checking in and replying. Hope you can assist.

egc · November 25, 2024, 8:40am

Consider upgrading to 23.05.5

Your WG does not connect to mullvad as seen by wg show
After you have updated, make a new Mullvad config and import that on your router.
If you want default route via the VPN make sure to enable Route allowed IPs.

You are trying to reach Mullvad on an IPv6 address, do you have a fully functional IPv6 implementation?

Note I did not look at your configs as you first have to establish a working WireGuard connection

Rickety8972 · November 25, 2024, 9:21am

Will do. That, will have to wait until next weekend unfortunately - don't have the brain capacity for that during the week.

I have had the vpn/wg working as the default (and only route) by following Mullvad's blog, but that's not suitable for all devices on the network, particularly when issues arise from using a vpn (gotta keep the wife happy), hence why I am trying pbr.

I didn't mean to use IPv6. Is that determined by the peer endpoint host address? I had previously used the IPv4 address but had changed that to the domain name at some point. I've changed that back to IPv4 now and rebooted but it doesn't appear to have changed anything.

I'll have a look at the docs and sort out backups etc and then do the upgrade. Then I might start with the Mullvad blog again (all traffic via wireguard - no pbr).

If I get all of that working, should I just be able to turn on pbr, make sure "route allowed IPs" is disabled (so that wan is default) and it should just work? Or are there other config changes needed when comparing all traffic via vpn (no pbr) and one route via vpn with pbr?

egc · November 25, 2024, 9:27am

Yes that is how it is supposed to work

stangri · November 25, 2024, 5:35pm

Before the reload it seems that nft is matching some packets to the set. How are you testing if the policy works?

Rickety8972 · November 25, 2024, 8:32pm

That’s my phone IP. I’m simply trying to load external websites in my phone after making changes (apply change, reboot, restart mobile browser, try to load website - mostly Mullvad.net).

Rickety8972 · November 30, 2024, 6:06am

Ok ...

I upgraded to 23.05.5 (ASU) and then revisited the Mullvad Wireguard on a Router guide.

After resetting everything according to the guide I successfully had all traffic running via the wireguard interface (wginterface). I check the pbr settings and noted that the default path was now wginterface.

I then went back and turned off "Route allowed IPs" in the wireguard interface peer settings and then rebooted.

Pbr settings now showed wan as the default path. I re-enabled my "wifi_test" policy, checked my phone's IP was correct and saved the changes. My phone could still connect to the internet via the wireguard tunnel but no other device on the network could access the internet.

Looking at what I'd changed I realised that the lan was being forwarded to the wgzone in firewall settings and thought this may be causing problems for other device on the lan, so I change this back to lan => wan. I figured that the wginterface would still be going via the wgzone as it was selected in covered networks.

After making this change, lan devices could again access the internet but my mobile could not any more! I rebooted to make sure all changes had properly updated but things remained the same.

If lan => wan then all lan devices work ok, but my phone (identified in the routing policy) can not access the internet. If lan => wgzone then the opposite is true - no lan devices reach the internet but the phone can (via the wg tunnel).

Appreciate any thoughts you might have @stangri and @egc

ubus call system board

{
	"kernel": "5.15.167",
	"hostname": "OpenWrt",
	"system": "MediaTek MT7621 ver:1 eco:3",
	"model": "GL.iNet GL-MT1300",
	"board_name": "glinet,gl-mt1300",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "23.05.5",
		"revision": "r24106-10cc5fcd00",
		"target": "ramips/mt7621",
		"description": "OpenWrt 23.05.5 r24106-10cc5fcd00"
	}
}

dhcp

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option cachesize '1000'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
	list server '10.64.0.1'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'
	option start '100'
	option limit '150'
	option leasetime '12h'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config host
	option name 'printer'
	option ip '192.168.1.192'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option ip '192.168.1.186'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'umbrel'
	option ip '192.168.1.131'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option ip '192.168.1.156'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option ip '192.168.1.176'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'DEBBIE-DESKTOP'
	option ip '192.168.1.200'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'Living-Room'
	option ip '192.168.1.149'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'stlnx'
	option ip '192.168.1.187'
	option mac 'XX:XX:XX:XX:XX:XX'

config host
	option name 'iMac'
	option ip '192.168.1.158'
	option mac 'XX:XX:XX:XX:XX:XX'

firewall

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'wan6'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config zone
	option name 'wgzone'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wginterface'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/firewall.include'

config forwarding
	option src 'lan'
	option dest 'wan'

network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdbd:d479:4af8::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'

config interface 'wginterface'
	option proto 'wireguard'
	option private_key 'REMOVED'
	list addresses '10.70.156.54/32'
	option force_link '1'

config wireguard_wginterface
	option description 'REMOVED'
	option public_key 'REMOVED'
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'REMOVED'
	option endpoint_port '51820'

config route
	option interface 'wginterface'
	option target '100.64.0.7/32'

pbr config

config pbr 'config'
	option enabled '1'
	option verbosity '2'
	option strict_enforcement '1'
	option resolver_set 'none'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver'
	list ignored_interface 'wgserver'
	option boot_timeout '30'
	option rule_create_option 'add'
	option procd_reload_delay '1'
	option webui_show_ignore_target '0'
	list webui_supported_protocol 'all'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	option nft_rule_counter '1'
	option nft_set_counter '1'

config include
	option path '/usr/share/pbr/pbr.user.aws'
	option enabled '0'

config include
	option path '/usr/share/pbr/pbr.user.netflix'
	option enabled '0'

config policy
	option name 'Plex/Emby Local Server'
	option interface 'wan'
	option src_port '8096 8920 32400'
	option enabled '0'

config policy
	option name 'Plex/Emby Remote Servers'
	option interface 'wan'
	option dest_addr 'plex.tv my.plexapp.com emby.media app.emby.media tv.emby.media'
	option enabled '0'

config policy
	option src_addr '192.168.1.125'
	option interface 'mullvad'
	option name 'stlnx_wireguard'
	option enabled '0'

config policy
	option name 'wifi_test'
	option interface 'wginterface'
	option src_addr '192.168.1.159'

pbr status

pbr - environment
pbr 1.1.6-22 running on OpenWrt 23.05.5.

Dnsmasq version 2.90  Copyright (c) 2000-2024 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile

pbr fw4 nft file: /usr/share/nftables.d/ruleset-post/30-pbr.nft
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000 counter mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000 counter mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.159 } counter goto pbr_mark_0x020000 comment "wifi_test"

pbr chains - policies
	chain pbr_forward { # handle 37
	}
	chain pbr_input { # handle 38
	}
	chain pbr_output { # handle 39
	}
	chain pbr_postrouting { # handle 41
	}
	chain pbr_prerouting { # handle 40
		ip saddr 192.168.1.159 counter packets 88 bytes 6798 goto pbr_mark_0x020000 comment "wifi_test" # handle 627
	}
	chain pbr_dstnat { # handle 36
	}

pbr chains - marking
	chain pbr_mark_0x010000 { # handle 236
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 623
		return # handle 624
	}
	chain pbr_mark_0x020000 { # handle 239
		counter packets 88 bytes 6798 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 625
		return # handle 626
	}

pbr nft sets

IPv4 table 256 route: default via 100.82.168.1 dev wan 
IPv4 table 256 rule(s):
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 route: default via 10.70.156.54 dev wginterface 
IPv4 table 257 rule(s):
29998:	from all fwmark 0x20000/0xff0000 lookup pbr_wginterface

pbr reload

Using wan interface (on_start): wan 
Found wan gateway (on_start): 100.82.168.1 
Setting up routing for 'wan/100.82.168.1' [✓]
Setting up routing for 'wginterface/10.70.156.54' [✓]
pbr.cfg066ff5.name=stlnx_wireguard validates as string with true
pbr.cfg066ff5.enabled=0 validates as bool with true
pbr.cfg066ff5.interface=mullvad validates as or("ignore", "tor", regex("xray_.*"), uci("network", "@interface")) with false
pbr.cfg066ff5.proto is unset and defaults to or(string) (null)
pbr.cfg066ff5.chain is unset and defaults to or("", "forward", "input", "output", "prerouting", "postrouting", "FORWARD", "INPUT", "OUTPUT", "PREROUTING", "POSTROUTING") prerouting
pbr.cfg066ff5.src_addr=192.168.1.125 validates as list(neg(or(host,network,macaddr,string))) with true
pbr.cfg066ff5.src_port is unset and defaults to list(neg(or(portrange,string))) (null)
pbr.cfg066ff5.dest_addr is unset and defaults to list(neg(or(host,network,string))) (null)
pbr.cfg066ff5.dest_port is unset and defaults to list(neg(or(portrange,string))) (null)
Routing 'wifi_test' via wginterface [✓]
Installing fw4 nft file [✓]
pbr 1.1.6-22 monitoring interfaces: wan wginterface 
pbr 1.1.6-22 (fw4 nft file mode) started with gateways:
wan/100.82.168.1 [✓]
wginterface/10.70.156.54

pbr status

pbr - environment
pbr 1.1.6-22 running on OpenWrt 23.05.5.

Dnsmasq version 2.90  Copyright (c) 2000-2024 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile

pbr fw4 nft file: /usr/share/nftables.d/ruleset-post/30-pbr.nft
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000 counter mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000 counter mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.159 } counter goto pbr_mark_0x020000 comment "wifi_test"

pbr chains - policies
	chain pbr_forward { # handle 37
	}
	chain pbr_input { # handle 38
	}
	chain pbr_output { # handle 39
	}
	chain pbr_postrouting { # handle 41
	}
	chain pbr_prerouting { # handle 40
		ip saddr 192.168.1.159 counter packets 0 bytes 0 goto pbr_mark_0x020000 comment "wifi_test" # handle 713
	}
	chain pbr_dstnat { # handle 36
	}

pbr chains - marking
	chain pbr_mark_0x010000 { # handle 707
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 708
		return # handle 709
	}
	chain pbr_mark_0x020000 { # handle 710
		counter packets 0 bytes 0 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 711
		return # handle 712
	}

pbr nft sets

IPv4 table 256 route: default via 100.82.168.1 dev wan 
IPv4 table 256 rule(s):
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 route: default via 10.70.156.54 dev wginterface 
IPv4 table 257 rule(s):
29998:	from all fwmark 0x20000/0xff0000 lookup pbr_wginterface

As I said, the only thing I am changing to flip the configuration is the firewall forwarding. The above is all with lan => wan. Below are for lan => wgzone.

firewall

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'wan6'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config zone
	option name 'wgzone'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wginterface'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/firewall.include'

config forwarding
	option src 'lan'
	option dest 'wgzone'

pbr status

pbr - environment
pbr 1.1.6-22 running on OpenWrt 23.05.5.

Dnsmasq version 2.90  Copyright (c) 2000-2024 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile

pbr fw4 nft file: /usr/share/nftables.d/ruleset-post/30-pbr.nft
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000 counter mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000 counter mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.159 } counter goto pbr_mark_0x020000 comment "wifi_test"

pbr chains - policies
	chain pbr_forward { # handle 37
	}
	chain pbr_input { # handle 38
	}
	chain pbr_output { # handle 39
	}
	chain pbr_postrouting { # handle 41
	}
	chain pbr_prerouting { # handle 40
		ip saddr 192.168.1.159 counter packets 311 bytes 69976 goto pbr_mark_0x020000 comment "wifi_test" # handle 790
	}
	chain pbr_dstnat { # handle 36
	}

pbr chains - marking
	chain pbr_mark_0x010000 { # handle 707
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 786
		return # handle 787
	}
	chain pbr_mark_0x020000 { # handle 710
		counter packets 311 bytes 69976 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 788
		return # handle 789
	}

pbr nft sets

IPv4 table 256 route: default via 100.82.168.1 dev wan 
IPv4 table 256 rule(s):
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 route: default via 10.70.156.54 dev wginterface 
IPv4 table 257 rule(s):
29998:	from all fwmark 0x20000/0xff0000 lookup pbr_wginterface

pbr reload

Using wan interface (on_start): wan 
Found wan gateway (on_start): 100.82.168.1 
Setting up routing for 'wan/100.82.168.1' [✓]
Setting up routing for 'wginterface/10.70.156.54' [✓]
pbr.cfg066ff5.name=stlnx_wireguard validates as string with true
pbr.cfg066ff5.enabled=0 validates as bool with true
pbr.cfg066ff5.interface=mullvad validates as or("ignore", "tor", regex("xray_.*"), uci("network", "@interface")) with false
pbr.cfg066ff5.proto is unset and defaults to or(string) (null)
pbr.cfg066ff5.chain is unset and defaults to or("", "forward", "input", "output", "prerouting", "postrouting", "FORWARD", "INPUT", "OUTPUT", "PREROUTING", "POSTROUTING") prerouting
pbr.cfg066ff5.src_addr=192.168.1.125 validates as list(neg(or(host,network,macaddr,string))) with true
pbr.cfg066ff5.src_port is unset and defaults to list(neg(or(portrange,string))) (null)
pbr.cfg066ff5.dest_addr is unset and defaults to list(neg(or(host,network,string))) (null)
pbr.cfg066ff5.dest_port is unset and defaults to list(neg(or(portrange,string))) (null)
Routing 'wifi_test' via wginterface [✓]
Installing fw4 nft file [✓]
pbr 1.1.6-22 monitoring interfaces: wan wginterface 
pbr 1.1.6-22 (fw4 nft file mode) started with gateways:
wan/100.82.168.1 [✓]
wginterface/10.70.156.54

pbr status

pbr - environment
pbr 1.1.6-22 running on OpenWrt 23.05.5.

Dnsmasq version 2.90  Copyright (c) 2000-2024 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile

pbr fw4 nft file: /usr/share/nftables.d/ruleset-post/30-pbr.nft
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000 counter mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000 counter mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.159 } counter goto pbr_mark_0x020000 comment "wifi_test"

pbr chains - policies
	chain pbr_forward { # handle 37
	}
	chain pbr_input { # handle 38
	}
	chain pbr_output { # handle 39
	}
	chain pbr_postrouting { # handle 41
	}
	chain pbr_prerouting { # handle 40
		ip saddr 192.168.1.159 counter packets 11 bytes 1001 goto pbr_mark_0x020000 comment "wifi_test" # handle 876
	}
	chain pbr_dstnat { # handle 36
	}

pbr chains - marking
	chain pbr_mark_0x010000 { # handle 870
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 871
		return # handle 872
	}
	chain pbr_mark_0x020000 { # handle 873
		counter packets 11 bytes 1001 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 874
		return # handle 875
	}

pbr nft sets

IPv4 table 256 route: default via 100.82.168.1 dev wan 
IPv4 table 256 rule(s):
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 route: default via 10.70.156.54 dev wginterface 
IPv4 table 257 rule(s):
29998:	from all fwmark 0x20000/0xff0000 lookup pbr_wginterface

Rickety8972 · November 30, 2024, 6:19am

One more thing ...

I just tried forwarding lan to both wgzone and wan. Both my phone and other devices on the network have internet access!

The only issue is that if I visit the Mullvad Connection Check page with my phone it says that I am not using Mullvad VPN, even though it shows the IPv4 address as "103.216.xx.xx (Wireguard)" and also shows the server name as the Mullvad server I've configured in the peer settings.

It also says the phone is leaking the DNS server, but the server details are not the same (other device show the IPv4 and DNS server of my ISP when checked).

When I only have lan => wgzone the connection check says I am using Mullvad VPN,

egc · November 30, 2024, 8:51am

Look at it like this, you have two outgoing interfaces one to the WAN and one to the VPN and from the LAN side where you phone also resides you want to have access to both. So indeed you have to forward from LAN > WAN and from LAN > VPN.
If you phone should be on another separate network e.g. guest or iot you could forward only that network to the VPN.

Mullvad will check your DNS and will inform you that your are not using their network in full as you have a DNS leak.
This is caused by the fact that the default routing is via the WAN, and your router and thus DNSMasq is using the default routing via the WAN.

There are several things you can do about this:
Reverse the situation, so default routing via the VPN and make policies to use the WAN for everything other than your phone.
Or use Split DNS see my notes about this:

The PBR package can help you also with that as it incorporates DNS policies, also described in my notes

Rickety8972 · November 30, 2024, 9:04am

So I kinda just fell backwards into the right solution

Thanks for sharing the information on Split DNS - that sounds like exactly what I'm missing.

And so the (steep) learning curve continues!

Rickety8972 · December 1, 2024, 5:29am

Still not able to resolve the DNS issue.

I followed your notes @egc, specifically the section on PBR DNS Policies. It seemed straight forward but I'm still getting the Leaking DNS notification for Mullvad.

PBR is setup like this:

config policy
	option name 'wifi_test'
	option interface 'wginterface'
	option src_addr '192.168.1.159'

config dns_policy
	option name 'wg_dns'
	option src_addr '192.168.1.159'
	option dest_dns 'wginterface'

And wginterface has the DNS server set to the one in the Mullvad guide:

config interface 'wginterface'
	option proto 'wireguard'
	option private_key 'REMOVED'
	list addresses '10.70.156.54/32'
	option force_link '1'
	list dns '10.64.0.1'

I also tried adding the DNS address directly in the PBR DNS Policy at dest_dns, but this didn't change anything.

I also had a look at Split DNS using option 6 but wasn't sure how that worked as the wginterface doesn't have DHCP enabled so it doesn't show up as an interface in DHCP Config. I tried adding list dhcp_option 'wginterface,6,10.64.0.1' but that didn't seem to change anything.

Thoughts?

egc · December 1, 2024, 10:01am

Remove the wginterface and check on your clients that they have 10.64.0.1 as DNS server instead of the routers address

About DNS policies, do have IPv6 enabled?

If so you have to make two DNS policy rules one for IPv4 and one for IPv6, as your LAN client will have multiple IPv6 address it is sometimes difficult to know what is the preferred/used IPv6 address so better to use the clients MAC address for IPv4 and IPv6 address.
Your interface must also have both an IPv4 and IPv6 DNS server set.

I have updated the information, see: https://github.com/egc112/OpenWRT-egc-add-on/tree/main/stop-dns-leak#split-dns

Furthermore if you have other DNS hijacking rules or use the Force DNS redirect rules from HTTPS-DNS proxy then those are not compatible as they take precedence over the DNS policy rules!

Rickety8972 · December 2, 2024, 7:31am

Is this in relation to the 'tag' option you describe in your notes (specifically, assign 10.64.0.1 to clients via tags)? I was trying to apply the DNS to all traffic that routes via that interface.

OK ... I should probably take a step back and explain what I'm ultimately hoping to do here.

Most devices in the house connect via wifi. I can't have all traffic going via the wireguard/vpn tunnel - some devices like the Apple TV just refuse to work and others, while they work most of the time, need an alternative route available if a particular website is being blocked because of the vpn.

Ideally I'd like to have two separate networks/SSID using the same wifi radio - one that goes via the wireguard tunnel (primary) and the other that doesn't (failsafe). I'm not even sure this is possible but have read some things to suggest it is.

However, as you've seen, I've had problems getting even basic split tunneling set up due to my overall lack of knowledge of how these things work. So starting from scratch (getting all traffic via Mullvad wireguard) and then extended from there seemed like a good idea.

The solutions your suggesting seem to rely on identifying specific devices by IP or MAC address and I'm wondering if that's going to work given my preferred outcome.

I guess the first question is, is what I want to do even possible? And if so, what's the best way to go about it (might not be my 2x network on the same radio delusion)?

In regards to IPv6, it appears it is enabled (to whatever degree) out of the box. I haven't setup up anything specific in that regard as I have very limited understanding of what it is apart from the replacement for IPv4.

Under the OpenWrt Status in Luci I see IPv6 Upstream for wan that looks to be connected to a 2401: address (my ISP?) and some of the devices that have an active DHCP lease also have an active DHCPv6 lease.

Do I even need/want IPv6 enabled and is it better to just disable it globally (if that's even possible)?

Apologies again for all the newbie questions that are likely frustrating to most on this forum. I'm normally pretty good at hacking things together with limited background but my inadequacies are clearly showing here.

As always, I'm very appreciative of any advice you can offer.

egc · December 2, 2024, 4:42pm

Sure make a guest wifi: https://openwrt.org/docs/guide-user/network/wifi/guestwifi/configuration_webinterface

Then you use PBR to route this guest wifi via the VPN.

You do not need IPv6, although it should not hurt but you need some more work to get your PBR setup.
I have IPv6 and works without a problem, mullvad supports IPv6, it works also with PBR.
But you can disable IPv6 if you are not using it:

Disable IPv6
https://3os.org/infrastructure/openwrt/disable-ipv6/
uci set 'network.lan.ipv6=0'
uci set 'network.wan.ipv6=0'
uci set 'dhcp.lan.dhcpv6=disabled'

# Disable RA and DHCPv6 so no IPv6 IPs are handed out
uci -q delete dhcp.lan.dhcpv6
uci -q delete dhcp.lan.ra

# Disable the LAN delegation
uci set network.lan.delegate="0"

# Delete the IPv6 ULA Prefix
uci -q delete network.globals.ula_prefix

# Disable odhcpd
/etc/init.d/odhcpd disable
/etc/init.d/odhcpd stop

# Save changes
uci commit
/etc/init.d/network restart