Will do. Away from my network for a couple of days, but will let you know during the week
@rainmaverick I'm running multiple dnsmasq instances: one with Stubby, one with Unbound (one with neither) and haven't seen any problems in conjunction with VPR.
Happy to help by sharing my configs, but probably best to shift it from this thread, as @stangri suggested.
1 Like
I've been trying to figure out the following:
My router has two wireguard interfaces, first one is for my VPN provider and using vpn-policy-routing to assign certain hosts/subnets to it.
The second wireguard interface is used for my mobile devices to access my network at home.
This mostly works with one exception, a device connected to the second wireguard interface will not be able to talk to any host/subnet that has been set up to be policed by vpn-policy-routing.
Some packet dumps/traces shows that packets sent to the hosts policed by vpn-policy-routing will arrive there, but their replies is then sent out the interface configured in vpn-policy-routing, instead of returning to where it came from (the second wireguard interface).
I'm no expert at this, so maybe this is normal? But it would be nice if vpn-policy-routing could be set to not route targets in a local (rfc1918) over the outgoing wan/vpn interface? Perhaps I missed some other setting. Is there anything else I should try? I've been trying to edit the iptables rules to add '! -d 10.0.0.0/8' or similar but haven't so far managed to get things working.
Instead of doing this, try setting (under VPR advanced configuration settings) "Append local IP Tables rules" to "! -d 10.0.0.1/8
[EDIT] also, make sure your second wireguard interface (the one that lets you 'phone home') is in your lan firewall zone.
@stangri I'm still 'on the road', but my VPR config has always had list ignored_interface specified. Appreciate this is the opposite of that which you're asking me to check, but perhaps the effect is equivalent?
No luck unfortunately.
The second wireguard interface already shares the lan firewall zone, and modifying the iptables remote rule in vpn-policy-routing leads to: 'ERROR: iptables -t mangle -I VPR_PREROUTING 1 -j MARK --set-xmark 0x020000/0xff0000 -s 10.0.1.1/24 -d 0.0.0.0/0 ! -d 10.0.0.1/8 -m comment --comment VPN'
Something like 'iptables -t mangle -I VPR_PREROUTING 1 -j MARK --set-xmark 0x020000/0xff0000 -s 10.0.1.1/24 ! -d 10.0.0.1/8' would probably work, but would need editing the vpn-policy-routing service itself.
I must have missed something.. it seems there should be a better way.
I've got the same setup as you: one wireguard interface (in my wan zone) for my VPN provider; and one in my lan zone to allow me to 'phone home'. I'll share my configs with you via DM in the next couple of days once I'm back at home.
Appreciate that, thanks! I'll keep trying meanwhile..
You can use -m iprange to do the trick. It's included in OpenWrt opkg source (iptables-mod-iprange)
Hi guys,
I have a minor issue about when I reboot my router. I have everything working great, just when I reboot, I have to manually restart my Wireguard VPN interface for VPR to pick it up.
I notice that, after a reboot, in the VPR status output it is missing the default route via the VPN interface:
vpn-policy-routing 0.0.4-4 running on OpenWrt 18.06.1. WAN (IPv4): wan/dev/77.248.XX.X. WAN (IPv6): wan6/dev6/fe80::/64.
============================================================
Dnsmasq version 2.80 Copyright (c) 2000-2018 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC no-ID loop-detect inotify dumpfile
============================================================
Routes/IP Rules
default dhcp-077-248-02 0.0.0.0 UG 0 0 0 eth1
IPv4 Table 201:
IPv4 Table 201 Rules:
32765: from all fwmark 0x10000 lookup 201
After I have restarted the wg interface, the default route is populated correctly:
============================================================
Routes/IP Rules
default dhcp-077-248-02 0.0.0.0 UG 0 0 0 eth1
IPv4 Table 201: default via 192.168.0.2 dev wg
IPv4 Table 201 Rules:
32765: from all fwmark 0x10000 lookup 201
============================================================
Here's my config:
config vpn-policy-routing 'config'
option boot_timeout '30'
option verbosity '2'
option ipv6_enabled '1'
option strict_enforcement '1'
option dnsmasq_enabled '1'
option enabled '1'
list supported_interface 'wg'
list supported_interface 'wg6'
list ignored_interface 'wan'
list ignored_interface 'wan6'
Any ideas?
Thanks,
Dan
Please share as much as possible publicly, so I could add it to the README potentially. I've been slowly building up the "recipes" section of it.
The goal is to force VPR to set the trigger for WAN interface monitoring, so yeah, it's a complete opposite of what you've had before.
1 Like
I've also just noticed that I don't have an IPv6 table created, despite having IPv6 support enabled. ipv6tables rules are added as are ipsets but not the routing table:
============================================================
Routes/IP Rules
default dhcp-077-248-02 0.0.0.0 UG 0 0 0 eth1
IPv4 Table 201: default via 192.168.0.2 dev wg
IPv4 Table 201 Rules:
32765: from all fwmark 0x10000 lookup 201
============================================================
IP Tables PREROUTING
-N VPR_PREROUTING
-A VPR_PREROUTING -m set --match-set wg dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
IP6 Tables PREROUTING
-N VPR_PREROUTING
-A VPR_PREROUTING -m set --match-set wg6 dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
I believe the status output is incorrect. Could you please post the output of:
ip rule list | grep -c 'fwmark'
ip -6 rule list | grep -c 'fwmark'
ip -6 route show
Here are my configs. My set up is:
- a guest network directed over my wan
- a 'family' network which uses dnsmasq+unbound with cleanbrowsing.org's 'family filter' (over TLS) with traffic directed over my VPN provider
- my lan interface which uses dnsmasq+stubby (possibly of interest to @rainmaverick; see my dhcp config) for DNS over TLS and which directs traffic over my VPN provider (with a couple of clients excluded)
- my 'phone home' wireguard server is in my lan firewall zone, and traffic is directed over the wan interface (useful if I need to appear from my ISP-provided (static) IP address.
@risc I think the key bits for running both wireguard interfaces alongside each other are:
option append_local_rules '! -d 192.168.0.0/16' in config vpn-policy-routing 'config'
and ensuring that route_allowed_ips is removed or set to 0 in config wireguard_wgserver
@stangri I'll roll-back to the older build this evening and populate the list of supported interfaces, as requested
/etc/config/network
config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fd2c:dd09:d634::/48'
config interface 'lan'
option type 'bridge'
option ifname 'eth1.1'
option proto 'static'
option ipaddr '192.168.100.1'
option ip6assign '60'
option netmask '255.255.255.0'
config interface 'wan'
option ifname 'eth0.2'
option proto 'pppoe'
option username '<redacted>'
option password '<redacted>'
option ipv6 'auto'
option peerdns '0'
option dns '84.200.69.80 84.200.70.40'
config interface 'wan6'
option ifname 'eth0.2'
option proto 'dhcpv6'
option reqaddress 'try'
option reqprefix 'auto'
option peerdns '0'
option dns '2001:1608:10:25::1c04:b12f 2001:1608:10:25::9249:d69b'
config switch
option name 'switch0'
option reset '1'
option enable_vlan '1'
config switch_vlan
option device 'switch0'
option vlan '1'
option ports '1 2 3 4 6t'
config switch_vlan
option device 'switch0'
option vlan '2'
option ports '5 0t'
config interface 'streaming'
option proto 'static'
option ipaddr '192.168.3.1'
option netmask '255.255.255.0'
option type 'bridge'
config interface 'guest'
option proto 'static'
option netmask '255.255.255.0'
option ipaddr '10.0.0.1'
option type 'bridge'
config interface 'wgserver'
option proto 'wireguard'
option private_key '<redacted>'
option listen_port '52000'
list addresses '192.168.99.1/24'
config interface 'mullvad'
option proto 'wireguard'
option private_key '<redacted>'
option force_link '1'
list addresses '10.99.57.166/32'
config wireguard_wgserver
option public_key '<redacted>'
list allowed_ips '192.168.99.3/32'
option persistent_keepalive '25'
option description 'pixel'
config wireguard_mullvad
list allowed_ips '0.0.0.0/0'
option persistent_keepalive '25'
option public_key '<redacted>'
option description 'gb2'
option endpoint_host '185.16.85.130'
config interface 'MODEM'
option proto 'static'
option ifname 'eth0.2'
option ipaddr '192.168.2.2'
option netmask '255.255.255.0'
config interface 'family'
option proto 'static'
option ipaddr '172.16.1.1'
option netmask '255.255.255.0'
option type 'bridge'
/etc/config/firewall
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option src_ip 'fc00::/6'
option dest_ip 'fc00::/6'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option target 'ACCEPT'
option proto 'udp'
option dest_port '67-68'
option name 'guest_dhcp'
option src 'guest'
config rule
option target 'ACCEPT'
option proto 'tcp udp'
option dest_port '53'
option name 'guest_dns'
option src 'guest'
config rule
option target 'ACCEPT'
option proto 'tcp udp'
option dest_port '53'
option name 'family_dns'
option src 'family'
config rule
option target 'ACCEPT'
option proto 'udp'
option dest_port '67-68'
option name 'family_dhcp'
option src 'family'
config rule
option src 'wan'
option target 'ACCEPT'
option proto 'udp'
option dest_port '52000'
option name 'Allow-Wireguard-Inbound'
config rule
option src 'guest'
option dest_ip '192.168.2.1'
option target 'DROP'
option name 'Disable Modem Access Guest'
option dest 'wan'
config rule
option src 'family'
option name 'Disable Modem Access Family'
option dest_ip '192.168.2.1'
option target 'DROP'
option dest 'wan'
config defaults
option syn_flood '1'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option network 'lan streaming wgserver'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
option network 'mullvad wan wan6 MODEM'
config include
option path '/etc/firewall.user'
config zone
option forward 'REJECT'
option output 'ACCEPT'
option name 'guest'
option input 'REJECT'
option network 'guest'
config forwarding
option dest 'wan'
option src 'guest'
config forwarding
option dest 'wan'
option src 'lan'
config zone
option forward 'REJECT'
option output 'ACCEPT'
option name 'family'
option input 'REJECT'
option network 'family'
config forwarding
option dest 'wan'
option src 'family'
config redirect 'adblock_dns'
option name 'Adblock DNS'
option src 'lan'
option proto 'tcp udp'
option src_dport '53'
option dest_port '53'
option target 'DNAT'
option dest_ip '192.168.100.1'
config redirect 'adblock_dns_guest'
option name 'Adblock DNS Guest'
option src 'guest'
option proto 'tcp udp'
option src_dport '53'
option dest_port '53'
option target 'DNAT'
option dest_ip '10.0.0.1'
config redirect 'adblock_dns_family'
option name 'Adblock DNS Family'
option src 'family'
option proto 'tcp udp'
option src_dport '53'
option dest_port '53'
option target 'DNAT'
option dest_ip '172.16.1.1'
/etc/config/dhcp
config dnsmasq 'main'
option domainneeded '1'
option localise_queries '1'
option local '/lan/'
option domain 'lan'
option expandhosts '1'
option authoritative '1'
option readethers '1'
option leasefile '/tmp/dhcp.leases'
option serversfile '/tmp/adb_list.overall'
option nonwildcard '1'
option localservice '1'
list notinterface 'guest'
list notinterface 'family'
option dnssec '1'
option dnsseccheckunsigned '1'
option rebind_protection '1'
option noresolv '1'
list server '127.0.0.1#5453'
config dnsmasq 'guest'
option domainneeded '1'
option localise_queries '1'
option rebind_protection '1'
option rebind_localhost '1'
option local '/guest/'
option domain 'guest'
option expandhosts '1'
option authoritative '1'
option readethers '1'
option leasefile '/tmp/dhcp.leases.guest'
option serversfile '/tmp/adb_list.overall'
option nonwildcard '1'
option localservice '1'
option noresolv '1'
list interface 'guest'
list notinterface 'lo'
option dnssec '1'
option dnsseccheckunsigned '1'
list server '185.228.168.9'
list server '185.228.169.9'
config dnsmasq 'family'
option domainneeded '1'
option localise_queries '1'
option rebind_protection '1'
option rebind_localhost '1'
option local '/family/'
option domain 'family'
option expandhosts '1'
option authoritative '1'
option readethers '1'
option leasefile '/tmp/dhcp.leases.family'
option serversfile '/tmp/adb_list.overall'
option nonwildcard '1'
option localservice '1'
option noresolv '1'
list interface 'family'
list notinterface 'lo'
option dnssec '1'
option dnsseccheckunsigned '1'
list server '127.0.0.1#53535'
config dhcp 'lan'
option instance 'main'
option interface 'lan'
option leasetime '12h'
option dhcpv6 'server'
option ra 'server'
option ra_management '1'
option start '100'
option limit '150'
config dhcp 'wan'
option interface 'wan'
option ignore '1'
config odhcpd 'odhcpd'
option maindhcp '0'
option leasefile '/tmp/hosts/odhcpd'
option leasetrigger '/usr/sbin/odhcpd-update'
option loglevel '4'
config dhcp 'streaming'
option leasetime '12h'
option instance 'main'
option interface 'streaming'
option start '100'
option limit '150'
config dhcp 'guests'
option start '100'
option limit '150'
option instance 'guest'
option interface 'guest'
option leasetime '1h'
config dhcp 'familysafe'
option start '100'
option leasetime '12h'
option limit '150'
option instance 'family'
option interface 'family'
config host
option name 'HDHomerun'
option dns '1'
option mac '<redacted>'
option ip '192.168.100.101'
config host
option name 'TV'
option dns '1'
option mac '<redacted>'
option ip '192.168.100.100'
config host
option name 'Hello'
option dns '1'
option mac '<redacted>'
option ip '192.168.100.102'
config host
option name 'Hue'
option dns '1'
option mac '<redacted>'
option ip '192.168.100.119'
/etc/config/vpr-policy-routing
config policy
option chain 'PREROUTING'
option interface 'wan'
option name 'TV'
option local_address '192.168.100.100'
option proto 'tcp udp'
config policy
option chain 'PREROUTING'
option interface 'wan'
option name 'Hello'
option local_address '192.168.100.102'
option proto 'tcp udp'
config policy
option chain 'PREROUTING'
option interface 'wan'
option name 'Hue'
option local_address '192.168.100.119'
option proto 'tcp udp'
config policy
option chain 'PREROUTING'
option name 'Private'
option interface 'mullvad'
option local_address '192.168.100.1/24'
option proto 'tcp udp'
config policy
option chain 'PREROUTING'
option name 'Family'
option local_address '172.16.1.1/24'
option interface 'mullvad'
option proto 'tcp udp'
config vpn-policy-routing 'config'
option verbosity '2'
option ipv6_enabled '0'
option ipset_enabled '1'
option dnsmasq_enabled '0'
option strict_enforcement '1'
list ignored_interface 'wgserver'
option append_local_rules '! -d 192.168.0.0/16'
option enabled '1'
option boot_timeout '120'
root@nl-rt0:~# ip rule list | grep -c 'fwmark'
1
root@nl-rt0:~# ip -6 rule list | grep -c 'fwmark'
1
root@nl-rt0:~# ip -6 route show
2a00:b980:XXX::/127 dev wg proto kernel metric 256 pref medium
2a00:b980:XXX:44::/64 dev br-lan proto kernel metric 256 pref medium
unreachable fdc5:207f:4365::/48 dev lo proto static metric 2147483647 error -113 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
Ignore my first reply, I'd disabled IPv6 whilst testing. Above output is with IPv6 enabled in VPR.
Hey, @stangri.
Done. Back on 0.0.4-4 and have two entries under the list of supported interfaces. So-far-so-good, but will keep an eye on it over the next week.
All the best.
In the end I managed to get things working with vpn-policy-routing, many thanks to tectonic.
I did however run in to some real quirks of uci / stubby / vpn-policy-routing, for example:
- Use working setup of dnsmasq+stubby.
- Add stangri repo and install vpn-policy-routing
- Set it up. All works well.
- Reboot
- Stubby goes bananas for some weird uci bootup reason seemingly caused by vpn-policy-routing
- Reinstall stubby+getdns. And all works well again.
- Reboot
- Now vpn-policy-routing won't work (at boot) until manual reload.
Rebooting will randomly break stubby again.. 
Anyway I got annoyed with complexities and asked myself for a different solution. And I found what worked well for me while keeping things simple was the following setup (without using vpn-policy-routing):
(Be aware that LUCI wont support most of these settings, UCI/ssh only (18.06), better stick with vpn-policy-routing if this seems complicated)
In /etc/config/networking, I added:
config rule
option src '10.0.0.128/25'
option lookup '44'
config rule
option src '10.0.10.128/25'
option lookup '44'
config rule
option mark '0x44'
option lookup '44'
config route 'mullvad_wireguard'
option interface 'mullvad'
option target '0.0.0.0'
option netmask '0.0.0.0'
option table '44'
And in /etc/firewall.user, the following
iptables -t mangle -A PREROUTING -m mac --mac-source 00:11:22:33:44:55 ! -d 10.0.0.0/8 -j MARK --set-mark 0x44
If desired the firewall rule can be added to /etc/config/firewall instead (but lacking a fix for async routing), like this:
config rule
option src 'lan'
option src_mac '00:11:22:33:44:55'
option proto 'all'
option target 'MARK'
option set_mark '0x44'
This allows me to manually assign IP's via DHCP or manually to use VPN or not. I limit dhcp scope to lower part of 10.0.0.0/24 subnet (10.0.0.2-10.0.0.127), and manually add hosts above 10.0.0.128 for VPN use. And I can also select custom mac addresses via firewall.user to be routed via VPN if needed.
I followed this guide here for expressvpn:
Note specifically the firewall section. If I disable OpenVPN I get no internet. I think this configuration is flowing on to VPN Policy Based Routing, if I for instance add my PC's IP to the config and no domains, so that everything should bypass VPN, I get no internet connectivity. Likewise with specific pages that I add, they can't be loaded. I added Google's DNS servers to the LAN interface, otherwise pages won't load through the VPN. So I'm guessing I need to do something at the firewall for this to work, any advice would be appreciated, I might post the same question over on the stichroads blog too considering he's already familiar with VPN Bypass..
EDIT: RESOLVED - stichroad's reply on his blog worked:
You've created a kill switch, preventing traffic from being routed over the WAN
EDIT: ah, sorry; just seen your edit saying you've resolved it.
