VPN Policy-Based Routing + Web UI -- Discussion

Post your /etc/config/network file.

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdf7:xxxx:xxxx::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option ipaddr '192.168.0.1'

config device
	option name 'wan'
	option macaddr 'xx:xx:xx:xx:xx:xx'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'
	option metric '1024'
	option hostname '*'
	option peerdns '0'
	list dns 'x.x.x.x'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'
	option auto '0'

config interface 'tun0'
	option proto 'none'
	option device 'tun0'
	option peerdns '0'
	list dns 'x.x.x.x'

Hi back at home base.

Installed 22.03.2 missing the option to resolv hostnames with dnsmasq ipset. I don't have the option available as I mentioned earlier.

root@Router:~# opkg list-installed | grep dnsmasq; opkg list-installed | grep pb
r;
dnsmasq-full - 2.86-14
luci-app-pbr - 0.9.8-18
pbr - 0.9.8-18
root@Router:~# 

Could you please post the output of dnsmasq -v?

And your user file and output of service pbr restart; sleep 3; service pbr status; please.

The user file is exactly the same like your pbr.user.netflix.
I've change in the user file to:

TARGET_SET='pbr_tun0_4_dst_ip'
TARGET_IPSET='pbr_tun0_4_dst_net'
TARGET_ASN='17035'

And now it's working okay.

Thank you stangri.

dnsmasq -v

root@Router:~# dnsmasq -v
Dnsmasq version 2.86  Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC no-ID loop-detect inotify dumpfile

This software comes with ABSOLUTELY NO WARRANTY.
Dnsmasq is free software, and you are welcome to redistribute it
under the terms of the GNU General Public License, version 2 or 3.
root@Router:~#
root@Router:~# service pbr restart; sleep 3; service pbr status;
Activating Traffic Killswitch [✓]
Removing routing for 'wan/eth0.10/xx.xx.56.1' [✓]
Removing routing for 'vpnclient/tun1/10.35.0.3' [✓]
Removing routing for 'wwan/wlan0/192.168.0.1' [✓]
Removing routing for 'wg0/10.2.0.2' [✓]
Deactivating Traffic Killswitch [✓]
pbr 0.9.8-18 (nft) stopped [✓]
Activating Traffic Killswitch [✓]
Setting up routing for 'wan/eth0.10/xx.xx.56.1' [✓]
Setting up routing for 'vpnclient/tun1/10.35.0.3' [✓]
Setting up routing for 'wwan/wlan0/192.168.0.1' [✓]
Setting up routing for 'wg0/10.2.0.2' [✓]
Routing 'ignore local traffic' via ignore [✓]
Routing 'Alle dhcp clients' via wg0 [✓]
Routing 'vpn server clients' via wg0 [✓]
Deactivating Traffic Killswitch [✓]
pbr 0.9.8-18 monitoring interfaces: wan vpnclient wwan wg0 
pbr 0.9.8-18 (nft) started with gateways:
wan/eth0.10/xx.xx.56.1 [✓]
vpnclient/tun1/10.35.0.3
wwan/wlan0/192.168.0.1
wg0/10.2.0.2
============================================================
pbr - environment
pbr 0.9.8-18 running on OpenWrt 22.03.2. WAN (IPv4): wan/eth0.10/82.75.56.1.
============================================================
Dnsmasq version 2.86  Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC no-ID loop-detect inotify dumpfile
============================================================
pbr chains - policies
	chain pbr_forward {
	}
	chain pbr_input {
	}
	chain pbr_output {
	}
	chain pbr_prerouting {
		ip saddr @pbr_ignore_4_src_ip_cfg016ff5 return comment "ignore local traffic"
		ip saddr @pbr_wg0_4_src_ip_cfg036ff5 goto pbr_mark_0x040000 comment "Alle dhcp clients"
		ip saddr @pbr_wg0_4_src_ip_cfg046ff5 goto pbr_mark_0x040000 comment "vpn server clients"
		ip daddr @pbr_wan_4_dst_ip goto pbr_mark_0x010000
		ip saddr @pbr_wan_4_src_ip goto pbr_mark_0x010000
		ether saddr @pbr_wan_4_src_mac goto pbr_mark_0x010000
		ip daddr @pbr_vpnclient_4_dst_ip goto pbr_mark_0x020000
		ip saddr @pbr_vpnclient_4_src_ip goto pbr_mark_0x020000
		ether saddr @pbr_vpnclient_4_src_mac goto pbr_mark_0x020000
		ip daddr @pbr_wwan_4_dst_ip goto pbr_mark_0x030000
		ip saddr @pbr_wwan_4_src_ip goto pbr_mark_0x030000
		ether saddr @pbr_wwan_4_src_mac goto pbr_mark_0x030000
		ip daddr @pbr_wg0_4_dst_ip goto pbr_mark_0x040000
		ip saddr @pbr_wg0_4_src_ip goto pbr_mark_0x040000
		ether saddr @pbr_wg0_4_src_mac goto pbr_mark_0x040000
	}
	chain pbr_postrouting {
	}
============================================================
pbr chains - marking
	chain pbr_mark_0x010000 {
		counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000
		return
	}
	chain pbr_mark_0x020000 {
		counter packets 0 bytes 0 meta mark set meta mark & 0xff02ffff | 0x00020000
		return
	}
	chain pbr_mark_0x030000 {
		counter packets 0 bytes 0 meta mark set meta mark & 0xff03ffff | 0x00030000
		return
	}
	chain pbr_mark_0x040000 {
		counter packets 49 bytes 5262 meta mark set meta mark & 0xff04ffff | 0x00040000
		return
	}
============================================================
pbr nft sets
	set pbr_ignore_4_src_ip_cfg016ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "ignore local traffic: 10.2.0.0/24"
		elements = { 10.2.0.0/24 }
	}
	set pbr_wg0_4_src_ip_cfg036ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "Alle dhcp clients: 192.168.1.100/30"
		elements = { 192.168.1.100/30, 192.168.1.104/29,
			     192.168.1.112/28, 192.168.1.128/25 }
	}
	set pbr_wg0_4_src_ip_cfg046ff5 {
		type ipv4_addr
		flags interval
		auto-merge
		comment "vpn server clients: 192.168.200.2/31"
		elements = { 192.168.200.2/31, 192.168.200.4/30,
			     192.168.200.8/29, 192.168.200.16/28,
			     192.168.200.32/28, 192.168.200.48/31,
			     192.168.200.50 }
	}
	set pbr_wan_4_dst_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wan_4_src_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wan_4_src_mac {
		type ether_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_vpnclient_4_dst_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_vpnclient_4_src_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_vpnclient_4_src_mac {
		type ether_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wwan_4_dst_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wwan_4_src_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wwan_4_src_mac {
		type ether_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wg0_4_dst_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wg0_4_src_ip {
		type ipv4_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
	set pbr_wg0_4_src_mac {
		type ether_addr
		policy memory
		flags interval
		auto-merge
		comment ""
	}
root@Router:~# 

Do you have an ipset package installed?

hmm only this one;

root@Router:~# opkg list-installed | grep ipset
kmod-ipt-ipset - 5.10.146-1
root@Router:~# 

I was pretty sure that I read this and act like it but... I think not.

which requires the dnsmasq-full package with ipset support

Sorry for using your time, just forgot to install the package in my imagebuild ;-(

root@Router:~# opkg install ipset
Installing ipset (7.15-2) to root...
Downloading https://downloads.openwrt.org/releases/22.03.2/packages/aarch64_cortex-a72/base/ipset_7.15-2_aarch64_cortex-a72.ipk
Installing libipset13 (7.15-2) to root...
Downloading https://downloads.openwrt.org/releases/22.03.2/packages/aarch64_cortex-a72/base/libipset13_7.15-2_aarch64_cortex-a72.ipk
Configuring libipset13.
Configuring ipset.
root@Router:~# opkg list-installed | grep ipset
ipset - 7.15-2
kmod-ipt-ipset - 5.10.146-1
libipset13 - 7.15-2
root@Router:~# 

It works. thanks for the help offcourse.

I may append the README and/or code at some point, but there are no hard requirements on old/legacy packages if you install pbr, they are only present for pbr-iptables. Maybe dnsmasq-full used to depend on ipset so it would get installed automatically.

1 Like

ooh, It now failed to setup any gateway with "dnsmasq ipset". activated?

When I choose "disabled' it sets up the gateways. (but resolving domains is not working offcourse)

root@Router:~# service pbr restart; sleep 3; service pbr status;
Activating Traffic Killswitch [✗]
Setting up routing for 'wan/eth0.10/82.75.56.1' [✗]
Setting up routing for 'vpnclient/tun1/10.35.0.3' [✗]
Setting up routing for 'wwan/wlan0/192.168.0.1' [✗]
Setting up routing for 'wg0/10.2.0.2' [✗]
Routing 'ignore local traffic' via ignore [✗]
Routing 'Alle dhcp clients' via wg0 [✗]
Routing 'vpn server clients' via wg0 [✗]
Routing 'google' via wan [✗]
Routing 'netflix' via wan [✗]
Routing 'Afas' via wan [✗]
Deactivating Traffic Killswitch [✗]
pbr 0.9.8-18 monitoring interfaces: wan vpnclient wwan wg0 
ERROR: Failed to set up 'wan/eth0.10/82.75.56.1'
ERROR: Failed to set up 'vpnclient/tun1/10.35.0.3'
ERROR: Failed to set up 'wwan/wlan0/192.168.0.1'
ERROR: Failed to set up 'wg0/10.2.0.2'
ERROR: iptables -t mangle -A PBR_PREROUTING -j RETURN  -s 10.2.0.0/24 -m comment --comment ignore_local_traffic

ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.1.100/30 -m comment --comment Alle_dhcp_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.1.104/29 -m comment --comment Alle_dhcp_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.1.112/28 -m comment --comment Alle_dhcp_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.1.128/25 -m comment --comment Alle_dhcp_clients

ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.2/31 -m comment --comment vpn_server_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.4/30 -m comment --comment vpn_server_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.8/29 -m comment --comment vpn_server_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.16/28 -m comment --comment vpn_server_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.32/28 -m comment --comment vpn_server_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.48/31 -m comment --comment vpn_server_clients
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x040000  -s 192.168.200.50/32 -m comment --comment vpn_server_clients

ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x010000 -m set  --match-set pbr_wan_4_dst_ip_cfg056ff5 dst -m comment --comment google
ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x010000 -m set  --match-set pbr_wan_4_dst_ip_cfg056ff5 dst -m comment --comment google

ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x010000 -m set  --match-set pbr_wan_4_dst_ip_cfg066ff5 dst -m comment --comment netflix

ERROR: iptables -t mangle -A PBR_PREROUTING -g PBR_MARK_0x010000 -m set  --match-set pbr_wan_4_dst_ip_cfg076ff5 dst -m comment --comment Afas

ERROR: failed to set up any gateway!
pbr 0.9.8-18 running on OpenWrt 22.03.2. WAN (IPv4): wan/eth0.10/82.75.56.1.
============================================================
Dnsmasq version 2.86  Copyright (c) 2000-2021 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC no-ID loop-detect inotify dumpfile
============================================================
Routes/IP Rules
default         82-75-56-1.cabl 0.0.0.0         UG    10     0        0 eth0.10
default         192.168.0.1     0.0.0.0         UG    20     0        0 wlan0

IPv4 Table 201: default via 82.75.56.1 dev eth0.10 
10.0.10.0/24 dev eth0 proto kernel scope link src 10.0.10.1 
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1 
192.168.180.0/24 dev eth0.30 proto kernel scope link src 192.168.180.1 
192.168.200.0/24 dev tun0 proto kernel scope link src 192.168.200.1 
IPv4 Table 201 Rules:
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan

IPv4 Table 202: default via 10.35.0.3 dev tun1 
10.0.10.0/24 dev eth0 proto kernel scope link src 10.0.10.1 
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1 
192.168.180.0/24 dev eth0.30 proto kernel scope link src 192.168.180.1 
192.168.200.0/24 dev tun0 proto kernel scope link src 192.168.200.1 
IPv4 Table 202 Rules:
29999:	from all fwmark 0x20000/0xff0000 lookup pbr_vpnclient

IPv4 Table 203: default via 192.168.0.1 dev wlan0 
10.0.10.0/24 dev eth0 proto kernel scope link src 10.0.10.1 
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1 
192.168.180.0/24 dev eth0.30 proto kernel scope link src 192.168.180.1 
192.168.200.0/24 dev tun0 proto kernel scope link src 192.168.200.1 
IPv4 Table 203 Rules:
29998:	from all fwmark 0x30000/0xff0000 lookup pbr_wwan

IPv4 Table 204: default via 10.2.0.2 dev wg0 
10.0.10.0/24 dev eth0 proto kernel scope link src 10.0.10.1 
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1 
192.168.180.0/24 dev eth0.30 proto kernel scope link src 192.168.180.1 
192.168.200.0/24 dev tun0 proto kernel scope link src 192.168.200.1 
IPv4 Table 204 Rules:
29997:	from all fwmark 0x40000/0xff0000 lookup pbr_wg0
============================================================
Current ipsets
create pbr_ignore_4_src_net_cfg016ff5 hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_src_net_cfg036ff5 hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_src_net_cfg046ff5 hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_dst_ip_cfg056ff5 hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_dst_ip_cfg066ff5 hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_dst_ip_cfg076ff5 hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_dst_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_src_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_dst_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_src_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wan_4_src_mac hash:mac hashsize 1024 maxelem 65536 comment
create pbr_vpnclient_4_dst_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_vpnclient_4_src_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_vpnclient_4_dst_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_vpnclient_4_src_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_vpnclient_4_src_mac hash:mac hashsize 1024 maxelem 65536 comment
create pbr_wwan_4_dst_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wwan_4_src_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wwan_4_dst_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wwan_4_src_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wwan_4_src_mac hash:mac hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_dst_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_src_ip hash:ip family inet hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_dst_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_src_net hash:net family inet hashsize 1024 maxelem 65536 comment
create pbr_wg0_4_src_mac hash:mac hashsize 1024 maxelem 65536 comment
============================================================
DNSMASQ sets
ipset=/google.nl/pbr_wan_4_dst_ip_cfg056ff5 # google: google.nl
ipset=/google.com/pbr_wan_4_dst_ip_cfg056ff5 # google: google.com
ipset=/netflix.com/pbr_wan_4_dst_ip_cfg066ff5 # netflix: netflix.com
ipset=/afasinsite.nl/pbr_wan_4_dst_ip_cfg076ff5 # Afas: afasinsite.nl
============================================================
Your support details have been logged to '/var/pbr-support'. [✓]
root@Router:~# 

Please refer to the README section on dnsmasq-full installation with ipset support, there are additional packages mentioned there.

1 Like

Ok, I thought that I installed all needed packages, I check all. Thanks for the help :wink:

Had to install iptables. I thought somehow that this was not needed on 22.03.x

When I use the Custom User Files, /usr/share/pbr/pbr.user.netflix do I need to edit the TARGET_ASN line for my location?

Because when I use the default, the content on my notebook (vpn) is different then on my settop box (no vpn)

If you're not in CONUS, yes, Netflix may use other networks on other areas of the world. Those scripts were user-contributed and tested to some degree of success to work in CONUS only afaik.

If you want to use Netflix on your smart TV/media playback device, you may want to completely exclude it from using VPN by IP/mac address.

1 Like

The discussion for the new package (pbr) is here: Policy-Based-Routing (pbr) package discussion

@tmomas or other mods, please lock this thread.