Because you reject the input of the guest zone, so you have to accept the dhcp and dns for hosts to work.
11-14 are not running on the router, so there is no point to allow the input.
iptables-save -c -t mangle; ip -4 ro li table all
1 Like
root@OpenWrt:~# iptables-save -c -t mangle; ip -4 ro li table all
# Generated by iptables-save v1.8.3 on Fri Jan 15 13:21:19 2021
*mangle
:PREROUTING ACCEPT [12622785:14057512844]
:INPUT ACCEPT [4809343:7093849390]
:FORWARD ACCEPT [7812961:6963539438]
:OUTPUT ACCEPT [3846656:478769225]
:POSTROUTING ACCEPT [11659596:7442308247]
:QOS_MARK_eth1 - [0:0]
:QOS_MARK_mullvad - [0:0]
:VPR_FORWARD - [0:0]
:VPR_INPUT - [0:0]
:VPR_OUTPUT - [0:0]
:VPR_PREROUTING - [0:0]
[0:0] -A PREROUTING -i vtun+ -p tcp -j MARK --set-xmark 0x2/0xff
[4371:4166836] -A PREROUTING -i mullvad -m dscp ! --dscp 0x00 -j DSCP --set-dscp 0x00
[46:1656] -A PREROUTING -i eth1 -m dscp ! --dscp 0x00 -j DSCP --set-dscp 0x00
[12622788:14057512997] -A PREROUTING -m mark --mark 0x0/0xff0000 -j VPR_PREROUTING
[4807919:7093662375] -A INPUT -m mark --mark 0x0/0xff0000 -j VPR_INPUT
[3967527:6762113442] -A FORWARD -m mark --mark 0x0/0xff0000 -j VPR_FORWARD
[134:8432] -A FORWARD -o eth1 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[46:2760] -A FORWARD -i eth1 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[655:39668] -A FORWARD -o mullvad -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone mullvad MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[275:16420] -A FORWARD -i mullvad -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone mullvad MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[3070:265957] -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP --set-dscp 0x24
[3846658:478769366] -A OUTPUT -m mark --mark 0x0/0xff0000 -j VPR_OUTPUT
[3854764:479999164] -A POSTROUTING -o eth1 -m mark --mark 0x0/0xff -g QOS_MARK_eth1
[3847383:201886633] -A POSTROUTING -o mullvad -m mark --mark 0x0/0xff -g QOS_MARK_mullvad
[3854764:479999164] -A QOS_MARK_eth1 -j MARK --set-xmark 0x2/0xff
[0:0] -A QOS_MARK_eth1 -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
[2:293] -A QOS_MARK_eth1 -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
[0:0] -A QOS_MARK_eth1 -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
[294:20860] -A QOS_MARK_eth1 -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
[297:21088] -A QOS_MARK_eth1 -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff
[3847383:201886633] -A QOS_MARK_mullvad -j MARK --set-xmark 0x2/0xff
[0:0] -A QOS_MARK_mullvad -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
[191:26315] -A QOS_MARK_mullvad -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
[49:3724] -A QOS_MARK_mullvad -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
[164:12464] -A QOS_MARK_mullvad -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
[170:12920] -A QOS_MARK_mullvad -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff
[3845469:201443081] -A VPR_PREROUTING -s 10.19.89.0/24 -m comment --comment Lan -j MARK --set-xmark 0x20000/0xff0000
[1873:294099] -A VPR_PREROUTING -s 10.19.90.0/24 -m comment --comment Guest -j MARK --set-xmark 0x10000/0xff0000
COMMIT
# Completed on Fri Jan 15 13:21:19 2021
default via 172.17.17.1 dev eth1 table 201
10.19.90.0/24 dev br-guest table 201 proto kernel scope link src 10.19.90.1
default via X.X.X.X dev mullvad table 202
10.19.90.0/24 dev br-guest table 202 proto kernel scope link src 10.19.90.1
default dev mullvad proto static scope link
10.19.89.0/24 dev br-lan proto kernel scope link src 10.19.89.1
10.19.90.0/24 dev br-guest proto kernel scope link src 10.19.90.1
172.17.17.0/24 dev eth1 proto kernel scope link src 172.17.17.2
193.27.14.146 via 172.17.17.1 dev eth1 proto static
broadcast 10.19.89.0 dev br-lan table local proto kernel scope link src 10.19.89.1
local 10.19.89.1 dev br-lan table local proto kernel scope host src 10.19.89.1
broadcast 10.19.89.255 dev br-lan table local proto kernel scope link src 10.19.89.1
broadcast 10.19.90.0 dev br-guest table local proto kernel scope link src 10.19.90.1
local 10.19.90.1 dev br-guest table local proto kernel scope host src 10.19.90.1
broadcast 10.19.90.255 dev br-guest table local proto kernel scope link src 10.19.90.1
local X.X.X.X dev mullvad table local proto kernel scope host src X.X.X.X
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast 172.17.17.0 dev eth1 table local proto kernel scope link src 172.17.17.2
local 172.17.17.2 dev eth1 table local proto kernel scope host src 172.17.17.2
broadcast 172.17.17.255 dev eth1 table local proto kernel scope link src 172.17.17.2
The routes are still not populated for some reason.
What does /etc/init.d/vpn-policy-routing status say?
root@OpenWrt:~# /etc/init.d/vpn-policy-routing status
Syntax: /etc/init.d/vpn-policy-routing [command]
Available commands:
start Start the service
stop Stop the service
restart Restart the service
reload Reload configuration files (or restart if service does not implement reload)
enable Enable service autostart
disable Disable service autostart
support Generates output required to troubleshoot routing issues
Use '-d' option for more detailed output
Use '-p' option to automatically upload data under VPR paste.ee account
WARNING: while paste.ee uploads are unlisted, they are still publicly available
List domain names after options to include their lookup in report
root@OpenWrt:~# /etc/init.d/vpn-policy-routing support
ERROR: DNSMASQ ipset support is enabled in vpn-policy-routing, but DNSMASQ is either not installed or installed DNSMASQ does not support ipsets!
vpn-policy-routing 0.2.1-13 running on OpenWrt 19.07.5. WAN (IPv4): wan/dev/172.17.17.1.
============================================================
Dnsmasq version 2.80 Copyright (c) 2000-2018 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-DNSSEC no-ID loop-detect inotify dumpfile
============================================================
Routes/IP Rules
default * 0.0.0.0 U 0 0 0 mullvad
IPv4 Table 201: default via 172.17.17.1 dev eth1
10.19.90.0/24 dev br-guest proto kernel scope link src 10.19.90.1
IPv4 Table 201 Rules:
32741: from all fwmark 0x10000/0xff0000 lookup 201
IPv4 Table 202: default via X.X.X.X dev mullvad
10.19.90.0/24 dev br-guest proto kernel scope link src 10.19.90.1
IPv4 Table 202 Rules:
32740: from all fwmark 0x20000/0xff0000 lookup 202
============================================================
IP Tables PREROUTING
-N VPR_PREROUTING
-A VPR_PREROUTING -s 10.19.89.0/24 -m comment --comment Lan -c 3850645 202373006 -j MARK --set-xmark 0x20000/0xff0000
-A VPR_PREROUTING -s 10.19.90.0/24 -m comment --comment Guest -c 2143 325205 -j MARK --set-xmark 0x10000/0xff0000
============================================================
IP Tables FORWARD
-N VPR_FORWARD
============================================================
IP Tables INPUT
-N VPR_INPUT
============================================================
IP Tables OUTPUT
-N VPR_OUTPUT
============================================================
Current ipsets
============================================================
Your support details have been logged to '/var/vpn-policy-routing-support'. [✓]
root@OpenWrt:~#
Our configs are almost identical, I am using version 0.2.1-14 and I have the status option available. Also in every routing table created by vpn-pbr I have all my local routes, so there is no limitation in intranet traffic regardless of the gateway.
@stangri maybe you have an idea what's wrong here and the routing tables are not filled in properly?
I'm also wondering whats wrong here, since dnsmasq is installed 
root@OpenWrt:~# opkg list-installed | grep ^dnsmasq
dnsmasq - 2.80-16.1
You need dnsmasq-full for that, but it is not necessary if you don't use ipsets.
uci set vpn-policy-routing.@policy[0].dest_addr="!10.19.88.0/22"
uci commit vpn-policy-routing
/etc/init.d/vpn-policy-routing restart
Sorry, can you maybe highlight somehow what is expected to be in the routing tables but isn't present?
Sure! The OP has only one entry for the 10.19.90.0/24 from guest interface.
I have routes for all my interfaces, lan, guest, iot, plus some more static routes there are in main routing table:
Routes/IP Rules
default blah.blah 0.0.0.0 UG 10 0 0 pppoe-wan
default * 0.0.0.0 U 90 0 0 tun2
IPv4 Table 201: default via 95.152.X.X dev pppoe-wan
10.0.0.0/19 via 10.0.20.1 dev tun0 proto zebra metric 20
unreachable 10.0.0.0/8 proto static metric 240
10.0.2.0/24 dev eth0.4 proto kernel scope link src 10.0.2.1
10.0.3.0/24 via 10.0.10.3 dev roadwarrior proto zebra metric 20
10.0.10.0/24 dev roadwarrior proto kernel scope link src 10.0.10.1
10.0.20.0/30 dev tun0 proto kernel scope link src 10.0.20.2
10.0.20.2 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.20.4/30 dev elvetias proto kernel scope link src 10.0.20.5
10.0.20.10 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.20.14 via 10.0.20.1 dev tun0 proto zebra metric 20
unreachable 169.254.0.0/16 proto static metric 240
unreachable 172.16.0.0/12 proto static metric 240
172.17.17.0/24 dev eth0.2 proto kernel scope link src 172.17.17.1
172.30.30.0/24 dev eth0.3 proto kernel scope link src 172.30.30.1
unreachable 192.168.0.0/16 proto static metric 240
192.168.1.0/24 via 10.0.20.6 dev elvetias proto zebra metric 20
192.168.8.0/24 dev tun1 proto kernel scope link src 192.168.8.1
IPv4 Table 201 Rules:
...
The question is why we have different output in the routing tables with the same configuration? The only differences I found were some webgui cosmetics, apart from the slightly different version. And none has excluded any networks like you suggested.
root@magiatiko / > uci show vpn-policy-routing
vpn-policy-routing.@policy[0]=policy
vpn-policy-routing.@policy[0].src_addr='10.0.2.5/32'
vpn-policy-routing.@policy[0].proto='all'
vpn-policy-routing.@policy[0].interface='proton'
vpn-policy-routing.@policy[0].name='rockpi proton'
vpn-policy-routing.@policy[1]=policy
vpn-policy-routing.@policy[1].interface='wan'
vpn-policy-routing.@policy[1].name='rockpi local'
vpn-policy-routing.@policy[1].src_addr='10.0.2.5/32'
vpn-policy-routing.@policy[1].dest_addr='10.0.0.0/19'
vpn-policy-routing.@policy[2]=policy
vpn-policy-routing.@policy[2].interface='wan'
vpn-policy-routing.@policy[2].name='roadwarrior'
vpn-policy-routing.@policy[2].src_port='1200'
vpn-policy-routing.@policy[2].chain='OUTPUT'
vpn-policy-routing.@policy[2].proto='udp'
vpn-policy-routing.@policy[2].enabled='0'
vpn-policy-routing.config=vpn-policy-routing
vpn-policy-routing.config.ipv6_enabled='0'
vpn-policy-routing.config.supported_interface=''
vpn-policy-routing.config.boot_timeout='30'
vpn-policy-routing.config.webui_sorting='1'
vpn-policy-routing.config.webui_supported_protocol='tcp' 'udp' 'tcp udp' 'icmp' 'all'
vpn-policy-routing.config.webui_enable_column='1'
vpn-policy-routing.config.webui_protocol_column='1'
vpn-policy-routing.config.webui_chain_column='1'
vpn-policy-routing.config.iprule_enabled='0'
vpn-policy-routing.config.strict_enforcement='1'
vpn-policy-routing.config.enabled='1'
vpn-policy-routing.config.iptables_rule_option='append'
vpn-policy-routing.config.dest_ipset='dnsmasq.ipset'
vpn-policy-routing.config.src_ipset='0'
vpn-policy-routing.config.ignored_interface='vpnserver wgserver' 'roadwarrior' 'elvetias' 'vps'
vpn-policy-routing.config.verbosity='0'
vpn-policy-routing.@include[0]=include
vpn-policy-routing.@include[0].path='/etc/vpn-policy-routing.netflix.user'
vpn-policy-routing.@include[0].enabled='0'
vpn-policy-routing.@include[1]=include
vpn-policy-routing.@include[1].path='/etc/vpn-policy-routing.aws.user'
vpn-policy-routing.@include[1].enabled='0'
It looks like br-lan is ignored for some reason, so the issue affects all bridge lan* interfaces.
2 Likes
@vgaetera beat me to it. This is a piece of code which copies the entries from main routing table to the VPR tables: https://github.com/stangri/source.openwrt.melmac.net/blob/master/vpn-policy-routing/files/vpn-policy-routing.init#L627-L632.
If between the two of you, you can come up with a better code/idea, please do let me know/send PR. You have way more experience here.
As there seems to be no straightforward solution with PBR, I've played around with VLANs.
After I found out that my NAS can be in configured to be in multiple VLAN's at once (How To Configure Multiple VLANs on one Synology Bond) I now have a working solution. 
If some has a solution with port forwarding I'm happy to test it. Thank you all 
Actually, it works for me by excluding local networks as destination from the policy scope.
You're referring to this, right? I've tried it but I still can't get a connection.
Is there a reason to ignore the br-lan interface? This might work if there are no other interfaces and the internal traffic in lan is not hitting the router. If there are other interfaces and we want inter-interface traffic, they must be in the both in the custom routing tables.
For example br-lan is 192.168.1.0/24 and br-iot is 192.168.2.0/24, we make a policy for lan via wan and iot via vpn. They will be assigned to different routing tables, but this was there won't be communication between them.
1 Like
Got the same suggestion from @vgaetera. Could you gentlemen please help test this version, which has the suggested fix: https://repo.openwrt.melmac.net/vpn-policy-routing_0.3.2-2_all.ipk
If you're using WebUI you may have to update that too: https://repo.openwrt.melmac.net/luci-app-vpn-policy-routing_git-20.358.64958-f3417f2_all.ipk. Since the luci packages naming was changed recently, you'll have to manually grab the IPK as the opkg update from my repo may not work.
2 Likes
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.