VPN Policy-Based Routing + Web UI -- Discussion

You can use -m iprange to do the trick. It's included in OpenWrt opkg source (iptables-mod-iprange)

Hi guys,

I have a minor issue about when I reboot my router. I have everything working great, just when I reboot, I have to manually restart my Wireguard VPN interface for VPR to pick it up.

I notice that, after a reboot, in the VPR status output it is missing the default route via the VPN interface:

vpn-policy-routing 0.0.4-4 running on OpenWrt 18.06.1. WAN (IPv4): wan/dev/77.248.XX.X. WAN (IPv6): wan6/dev6/fe80::/64.
============================================================
Dnsmasq version 2.80  Copyright (c) 2000-2018 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC no-ID loop-detect inotify dumpfile
============================================================
Routes/IP Rules
default         dhcp-077-248-02 0.0.0.0         UG    0      0        0 eth1
IPv4 Table 201:
IPv4 Table 201 Rules:
32765:	from all fwmark 0x10000 lookup 201

After I have restarted the wg interface, the default route is populated correctly:

============================================================
Routes/IP Rules
default         dhcp-077-248-02 0.0.0.0         UG    0      0        0 eth1
IPv4 Table 201: default via 192.168.0.2 dev wg
IPv4 Table 201 Rules:
32765:	from all fwmark 0x10000 lookup 201
============================================================

Here's my config:

config vpn-policy-routing 'config'
	option boot_timeout '30'
	option verbosity '2'
	option ipv6_enabled '1'
	option strict_enforcement '1'
	option dnsmasq_enabled '1'
	option enabled '1'
	list supported_interface 'wg'
	list supported_interface 'wg6'
	list ignored_interface 'wan'
	list ignored_interface 'wan6'

Any ideas?

Thanks,

Dan

Please share as much as possible publicly, so I could add it to the README potentially. I've been slowly building up the "recipes" section of it.

The goal is to force VPR to set the trigger for WAN interface monitoring, so yeah, it's a complete opposite of what you've had before.

1 Like

I've also just noticed that I don't have an IPv6 table created, despite having IPv6 support enabled. ipv6tables rules are added as are ipsets but not the routing table:

============================================================
Routes/IP Rules
default         dhcp-077-248-02 0.0.0.0         UG    0      0        0 eth1
IPv4 Table 201: default via 192.168.0.2 dev wg
IPv4 Table 201 Rules:
32765:	from all fwmark 0x10000 lookup 201
============================================================
IP Tables PREROUTING
-N VPR_PREROUTING
-A VPR_PREROUTING -m set --match-set wg dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
IP6 Tables PREROUTING
-N VPR_PREROUTING
-A VPR_PREROUTING -m set --match-set wg6 dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================

I believe the status output is incorrect. Could you please post the output of:
ip rule list | grep -c 'fwmark'
ip -6 rule list | grep -c 'fwmark'
ip -6 route show

Here are my configs. My set up is:

  • a guest network directed over my wan
  • a 'family' network which uses dnsmasq+unbound with cleanbrowsing.org's 'family filter' (over TLS) with traffic directed over my VPN provider
  • my lan interface which uses dnsmasq+stubby (possibly of interest to @rainmaverick; see my dhcp config) for DNS over TLS and which directs traffic over my VPN provider (with a couple of clients excluded)
  • my 'phone home' wireguard server is in my lan firewall zone, and traffic is directed over the wan interface (useful if I need to appear from my ISP-provided (static) IP address.

@risc I think the key bits for running both wireguard interfaces alongside each other are:
option append_local_rules '! -d 192.168.0.0/16' in config vpn-policy-routing 'config'

and ensuring that route_allowed_ips is removed or set to 0 in config wireguard_wgserver

@stangri I'll roll-back to the older build this evening and populate the list of supported interfaces, as requested

/etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd2c:dd09:d634::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth1.1'
	option proto 'static'
	option ipaddr '192.168.100.1'
	option ip6assign '60'
	option netmask '255.255.255.0'

config interface 'wan'
	option ifname 'eth0.2'
	option proto 'pppoe'
	option username '<redacted>'
	option password '<redacted>'
	option ipv6 'auto'
	option peerdns '0'
	option dns '84.200.69.80 84.200.70.40'

config interface 'wan6'
	option ifname 'eth0.2'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'
	option peerdns '0'
	option dns '2001:1608:10:25::1c04:b12f 2001:1608:10:25::9249:d69b'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '1 2 3 4 6t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '5 0t'

config interface 'streaming'
	option proto 'static'
	option ipaddr '192.168.3.1'
	option netmask '255.255.255.0'
	option type 'bridge'

config interface 'guest'
	option proto 'static'
	option netmask '255.255.255.0'
	option ipaddr '10.0.0.1'
	option type 'bridge'

config interface 'wgserver'
	option proto 'wireguard'
	option private_key '<redacted>'
	option listen_port '52000'
	list addresses '192.168.99.1/24'

config interface 'mullvad'
	option proto 'wireguard'
	option private_key '<redacted>'
	option force_link '1'
	list addresses '10.99.57.166/32'

config wireguard_wgserver
	option public_key '<redacted>'
	list allowed_ips '192.168.99.3/32'
	option persistent_keepalive '25'
	option description 'pixel'

config wireguard_mullvad
	list allowed_ips '0.0.0.0/0'
	option persistent_keepalive '25'
	option public_key '<redacted>'
	option description 'gb2'
	option endpoint_host '185.16.85.130'

config interface 'MODEM'
	option proto 'static'
	option ifname 'eth0.2'
	option ipaddr '192.168.2.2'
	option netmask '255.255.255.0'

config interface 'family'
	option proto 'static'
	option ipaddr '172.16.1.1'
	option netmask '255.255.255.0'
	option type 'bridge'


/etc/config/firewall

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config rule
	option target 'ACCEPT'
	option proto 'udp'
	option dest_port '67-68'
	option name 'guest_dhcp'
	option src 'guest'

config rule
	option target 'ACCEPT'
	option proto 'tcp udp'
	option dest_port '53'
	option name 'guest_dns'
	option src 'guest'

config rule
	option target 'ACCEPT'
	option proto 'tcp udp'
	option dest_port '53'
	option name 'family_dns'
	option src 'family'

config rule
	option target 'ACCEPT'
	option proto 'udp'
	option dest_port '67-68'
	option name 'family_dhcp'
	option src 'family'

config rule
	option src 'wan'
	option target 'ACCEPT'
	option proto 'udp'
	option dest_port '52000'
	option name 'Allow-Wireguard-Inbound'

config rule
	option src 'guest'
	option dest_ip '192.168.2.1'
	option target 'DROP'
	option name 'Disable Modem Access Guest'
	option dest 'wan'

config rule
	option src 'family'
	option name 'Disable Modem Access Family'
	option dest_ip '192.168.2.1'
	option target 'DROP'
	option dest 'wan'

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option network 'lan streaming wgserver'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option network 'mullvad wan wan6 MODEM'

config include
	option path '/etc/firewall.user'

config zone
	option forward 'REJECT'
	option output 'ACCEPT'
	option name 'guest'
	option input 'REJECT'
	option network 'guest'

config forwarding
	option dest 'wan'
	option src 'guest'

config forwarding
	option dest 'wan'
	option src 'lan'

config zone
	option forward 'REJECT'
	option output 'ACCEPT'
	option name 'family'
	option input 'REJECT'
	option network 'family'

config forwarding
	option dest 'wan'
	option src 'family'

config redirect 'adblock_dns'
	option name 'Adblock DNS'
	option src 'lan'
	option proto 'tcp udp'
	option src_dport '53'
	option dest_port '53'
	option target 'DNAT'
	option dest_ip '192.168.100.1'

config redirect 'adblock_dns_guest'
	option name 'Adblock DNS Guest'
	option src 'guest'
	option proto 'tcp udp'
	option src_dport '53'
	option dest_port '53'
	option target 'DNAT'
	option dest_ip '10.0.0.1'

config redirect 'adblock_dns_family'
	option name 'Adblock DNS Family'
	option src 'family'
	option proto 'tcp udp'
	option src_dport '53'
	option dest_port '53'
	option target 'DNAT'
	option dest_ip '172.16.1.1'

/etc/config/dhcp

config dnsmasq 'main'
	option domainneeded '1'
	option localise_queries '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option serversfile '/tmp/adb_list.overall'
	option nonwildcard '1'
	option localservice '1'
	list notinterface 'guest'
	list notinterface 'family'
	option dnssec '1'
	option dnsseccheckunsigned '1'
	option rebind_protection '1'
	option noresolv '1'
	list server '127.0.0.1#5453'

config dnsmasq 'guest'
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/guest/'
	option domain 'guest'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases.guest'
	option serversfile '/tmp/adb_list.overall'
	option nonwildcard '1'
	option localservice '1'
	option noresolv '1'
	list interface 'guest'
	list notinterface 'lo'
	option dnssec '1'
	option dnsseccheckunsigned '1'
	list server '185.228.168.9'
	list server '185.228.169.9'

config dnsmasq 'family'
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/family/'
	option domain 'family'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases.family'
	option serversfile '/tmp/adb_list.overall'
	option nonwildcard '1'
	option localservice '1'
	option noresolv '1'
	list interface 'family'
	list notinterface 'lo'
	option dnssec '1'
	option dnsseccheckunsigned '1'
	list server '127.0.0.1#53535'

config dhcp 'lan'
	option instance 'main'
	option interface 'lan'
	option leasetime '12h'
	option dhcpv6 'server'
	option ra 'server'
	option ra_management '1'
	option start '100'
	option limit '150'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'streaming'
	option leasetime '12h'
	option instance 'main'
	option interface 'streaming'
	option start '100'
	option limit '150'

config dhcp 'guests'
	option start '100'
	option limit '150'
	option instance 'guest'
	option interface 'guest'
	option leasetime '1h'

config dhcp 'familysafe'
	option start '100'
	option leasetime '12h'
	option limit '150'
	option instance 'family'
	option interface 'family'

config host
	option name 'HDHomerun'
	option dns '1'
	option mac '<redacted>'
	option ip '192.168.100.101'

config host
	option name 'TV'
	option dns '1'
	option mac '<redacted>'
	option ip '192.168.100.100'

config host
	option name 'Hello'
	option dns '1'
	option mac '<redacted>'
	option ip '192.168.100.102'

config host
	option name 'Hue'
	option dns '1'
	option mac '<redacted>'
	option ip '192.168.100.119'


/etc/config/vpr-policy-routing

config policy
	option chain 'PREROUTING'
	option interface 'wan'
	option name 'TV'
	option local_address '192.168.100.100'
	option proto 'tcp udp'

config policy
	option chain 'PREROUTING'
	option interface 'wan'
	option name 'Hello'
	option local_address '192.168.100.102'
	option proto 'tcp udp'

config policy
	option chain 'PREROUTING'
	option interface 'wan'
	option name 'Hue'
	option local_address '192.168.100.119'
	option proto 'tcp udp'

config policy
	option chain 'PREROUTING'
	option name 'Private'
	option interface 'mullvad'
	option local_address '192.168.100.1/24'
	option proto 'tcp udp'

config policy
	option chain 'PREROUTING'
	option name 'Family'
	option local_address '172.16.1.1/24'
	option interface 'mullvad'
	option proto 'tcp udp'

config vpn-policy-routing 'config'
	option verbosity '2'
	option ipv6_enabled '0'
	option ipset_enabled '1'
	option dnsmasq_enabled '0'
	option strict_enforcement '1'
	list ignored_interface 'wgserver'
	option append_local_rules '! -d 192.168.0.0/16'
	option enabled '1'
	option boot_timeout '120'


root@nl-rt0:~# ip rule list | grep -c 'fwmark'
1
root@nl-rt0:~# ip -6 rule list | grep -c 'fwmark'
1
root@nl-rt0:~# ip -6 route show
2a00:b980:XXX::/127 dev wg proto kernel metric 256 pref medium
2a00:b980:XXX:44::/64 dev br-lan proto kernel metric 256 pref medium
unreachable fdc5:207f:4365::/48 dev lo proto static metric 2147483647 error -113 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium

Ignore my first reply, I'd disabled IPv6 whilst testing. Above output is with IPv6 enabled in VPR.

Hey, @stangri.

Done. Back on 0.0.4-4 and have two entries under the list of supported interfaces. So-far-so-good, but will keep an eye on it over the next week.

All the best.

In the end I managed to get things working with vpn-policy-routing, many thanks to tectonic.

I did however run in to some real quirks of uci / stubby / vpn-policy-routing, for example:

  • Use working setup of dnsmasq+stubby.
  • Add stangri repo and install vpn-policy-routing
  • Set it up. All works well.
  • Reboot
  • Stubby goes bananas for some weird uci bootup reason seemingly caused by vpn-policy-routing
  • Reinstall stubby+getdns. And all works well again.
  • Reboot
  • Now vpn-policy-routing won't work (at boot) until manual reload.

Rebooting will randomly break stubby again.. :smile:

Anyway I got annoyed with complexities and asked myself for a different solution. And I found what worked well for me while keeping things simple was the following setup (without using vpn-policy-routing):

(Be aware that LUCI wont support most of these settings, UCI/ssh only (18.06), better stick with vpn-policy-routing if this seems complicated)
In /etc/config/networking, I added:

config rule
        option src '10.0.0.128/25'
        option lookup '44'

config rule
        option src '10.0.10.128/25'
        option lookup '44'

config rule
        option mark '0x44'
        option lookup '44'

config route 'mullvad_wireguard'
        option interface 'mullvad'
        option target '0.0.0.0'
        option netmask '0.0.0.0'
        option table '44'

And in /etc/firewall.user, the following

iptables -t mangle -A PREROUTING -m mac --mac-source 00:11:22:33:44:55 ! -d 10.0.0.0/8 -j MARK --set-mark 0x44

If desired the firewall rule can be added to /etc/config/firewall instead (but lacking a fix for async routing), like this:

config rule
  option src 'lan'
  option src_mac '00:11:22:33:44:55'
  option proto 'all'
  option target 'MARK'
  option set_mark '0x44'

This allows me to manually assign IP's via DHCP or manually to use VPN or not. I limit dhcp scope to lower part of 10.0.0.0/24 subnet (10.0.0.2-10.0.0.127), and manually add hosts above 10.0.0.128 for VPN use. And I can also select custom mac addresses via firewall.user to be routed via VPN if needed.

I followed this guide here for expressvpn:

Note specifically the firewall section. If I disable OpenVPN I get no internet. I think this configuration is flowing on to VPN Policy Based Routing, if I for instance add my PC's IP to the config and no domains, so that everything should bypass VPN, I get no internet connectivity. Likewise with specific pages that I add, they can't be loaded. I added Google's DNS servers to the LAN interface, otherwise pages won't load through the VPN. So I'm guessing I need to do something at the firewall for this to work, any advice would be appreciated, I might post the same question over on the stichroads blog too considering he's already familiar with VPN Bypass..

EDIT: RESOLVED - stichroad's reply on his blog worked:

Imgur

You've created a kill switch, preventing traffic from being routed over the WAN

EDIT: ah, sorry; just seen your edit saying you've resolved it.

I am getting "Started without PROCD support" in the VPN Policy-Based Routing steup page. Is this normal?

In the past it listed the OpenVPN instances that were started, I use 2.


1 Like

No, it should only happen on Turris routers running obsolete OpenWrt 15.05.

Hey. Im using OpenVPN on my router with VPN Policy Based routing (software is amazing btw.) to route couple devices through OpenVPN and everything is working great except port forwarding. I want to power on my device with Wake On Lan from WAN and it's working great only when there is no rule set up (device is not using VPN at this time). When I create rule for device to use VPN, it work's except port forwarding.

I was doing some testing and I discovered that port forward is working when I make rule like this: https://cdn3.imggmi.com/uploads/2019/4/24/42dfbbeef7314280ca6a5f35dccd9474-full.png Port 450 is this WOL service.

I am wondering... Everything is (i think) working fine.Traceroute goes through VPN also HTTP traffic etc. within that range i have set up. Port forwarding is working also fine. Only pings from what i seen are going through WAN but I think this is expected behavior because of Network Layer? Can this affect privacy, DNS leaks or included KillSwitch? Maybe there is any other (proper) way to forward port from WAN while using VPN service?

Thanks and sorry for my English.

I am using davidc502' latest build "OpenWrt SNAPSHOT r9886-399aa0b933 / LuCI Master (f138fc93)" on a WRT3200ACM.

How do I correct this problem?

EDIT: checked my VPR config file and all seems OK. VPR seems to run OK also but just the strange status message.

Will wait for an updated VPR build to see if error remains.

Please confirm versions of both VPR and the luci app you have installed.

I have the same issue and I am running OpenWrt SNAPSHOT r9901-6e7e2f4421 / LuCI Master (f138fc93) on BT HomeHub 5A with minimal config.

root@AhmarRouter:~# opkg list-installed *vpn-policy-routing
luci-app-vpn-policy-routing - git-19.113.55792-3c7b3fa-38
vpn-policy-routing - 0.0.4-6

Although it seems fine with SSH:

root@AhmarRouter:~# service vpn-policy-routing stop
vpn-policy-routing 0.0.4-6 stopped [✓]
root@AhmarRouter:~# service vpn-policy-routing start
Creating table 'wan/182.176.1.48' [✓]
Creating table 'vpn/10.211.1.34' [✓]
Routing 'Microsoft' via vpn [✓]
Routing 'Hub' via vpn [✓]
vpn-policy-routing 0.0.4-6 started on wan/182.176.1.48 vpn/10.211.1.34 [✓]
vpn-policy-routing 0.0.4-6 monitoring interfaces: wan vpn [✓]

Sorry, I should have included it in the first post:

  1. VPR = 0.0.4-6
  2. Luci App VPR = git-19.113.55792-3c7b3fa-38

@ahmar16 -- thank you for confirming @FCS001FCS's earlier report.

It must have something to do with luci API to access ubus in the snapshot.

I'll try to test the snapshot on my router sometime in early May and fix this.

I don't know if it matters but I use the basic "Bootstrap" theme for the GUI.

+++++++++++++++++++++++++++++++++++
EDIT: Maybe it is linked to the permissions issue in https://forum.openwrt.org/t/buggy-builds-luci-statistics-error-while-showing-graphs/35547/23?u=fcs001fcs

I also had the Statistics problem and reinstalled it to coorect the problem.

Just a thought.

1 Like