Incoming VPN with Outgoing VPN

[aal7]

I have an ER-X router running OpenWrt 23.05.

My basic setup is this:

On the LAN side I have four VLANS, called PIGEON (192.168.76.x), IOT (192.168.20.x), GUEST (192.168.30.x) and WORK (192.168.40.x). There also "lan" (192.168.10.x) which I'm not using (and I think this is where I differ from the example I quote below).

On the WAN side I have two interfaces. One is a PPPoE interface to my ISP, called PLUSNET (although its firewall zone name is wan), and the other is a WireGuard Outgoing VPN to SurfShark, called WGVPN.

PIGEON uses WGVPN.
IOT, GUEST and WORK use PLUSNET.

Upto this point, everything's working and I'm happy.

However, on PIGEON, I have two VPN servers for incoming VPNs. A Raspberry Pi running piVPN (Wireguard) and a Synology NAS running its own VPN server (Open VPN). I have corresponding apps on my phone and with my previous router setup (not OpenWrt) those apps were able to tunnel into my network and give me access to things on PIGEON (192.168.76.x). The VPN apps are set up to enter my network at my static ip address ie via PLUSNET, but PLUSNET doesn't have access to PIGEON as standard. Essentially what I think I'm trying to do is allow traffic on my incoming VPN (but only trafic on my incoming VPN) on PLUSNET to get to PIGEON so it can access the Pi and the NAS. However, try as I might I can't get either of them to connect with my OpenWrt router setup. Also, pbr doesn't always stay running for very long. It often stops.

My problem seems very similar to this example:

VPN to home network - port forwarding

Hello, (Please let me know what information is missing, I will append it asap.) The goal Reach my home network via wireguard vpn from anywhere. (Second mission: create two or three networks, and reach each network with a different tunnel.) Current setup I use a Linksys WRT1900AC running OpenWrt 21.02.3. It is connected to the internet via PPPoE, the IP address from my ISP is directly at the WAN interface: [image] On address 192.168.10.132 I have a Raspberry Pi sitting, running PiHole and…

so I know the solution is PBR. The problem is that that page doesn't really go into great detail about how to implement the PBR. I've done loads of reading, and experimenting but I've never got to a solution that works.

I'll include some files:

network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fda8:94ac:14b8::/48'
	option packet_steering '1'

config device
	option name 'eth0'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth1'
	list ports 'eth2'
	list ports 'eth3'
	list ports 'eth4'
	option igmp_snooping '1'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '10'
	option name 'br-lan.10'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '20'
	option name 'br-lan.20'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '30'
	option name 'br-lan.30'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '40'
	option name 'br-lan.40'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '50'
	option name 'br-lan.50'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'eth1'
	list ports 'eth2'
	list ports 'eth3'
	list ports 'eth4'

config bridge-vlan
	option device 'br-lan'
	option vlan '20'
	list ports 'eth1:t'
	list ports 'eth3:t'
	list ports 'eth4:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '30'
	list ports 'eth1:t'
	list ports 'eth4:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '40'
	list ports 'eth3:t'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.10.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'PLUSNET'
	option proto 'pppoe'
	option device 'eth0'
	option username 'xxxxxxxxxxxxxx'
	option password 'xxxxxxxxxxxxxx'
	option ipv6 'auto'
	option peerdns '0'
	list dns 'xxxxxxxxxxxx'
	list dns 'xxxxxxxxxxxx'

config interface 'WGVPN'
	option proto 'wireguard'
	option private_key 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
	list addresses '10.14.0.2/16'

config wireguard_WGVPN
	option description 'SurfShark London'
	option public_key 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'uk-lon.prod.surfshark.com'
	option endpoint_port '51820'
	option persistent_keepalive '0'

config interface 'PIGEON_VLAN'
	option proto 'static'
	option device 'br-lan.10'
	option ipaddr '192.168.76.1'
	option netmask '255.255.255.0'

config interface 'IOT_VLAN'
	option proto 'static'
	option device 'br-lan.20'
	option ipaddr '192.168.20.1'
	option netmask '255.255.255.0'

config interface 'GUEST_VLAN'
	option proto 'static'
	option device 'br-lan.30'
	option ipaddr '192.168.30.1'
	option netmask '255.255.255.0'

config interface 'WORK_VLAN'
	option proto 'static'
	option device 'br-lan.40'
	option ipaddr '192.168.40.1'
	option netmask '255.255.255.0'

config rule
	option in 'PIGEON_VLAN'
	option dest '192.168.20.0/24'
	option lookup 'main'

config rule
	option in 'PIGEON_VLAN'
	option dest '192.168.30.0/24'
	option lookup 'main'

config rule
	option in 'PIGEON_VLAN'
	option lookup '1742'

config route
	option interface 'WGVPN'
	option target '0.0.0.0'
	option netmask '0.0.0.0'
	option table '1742'

firewall (in this file you can see some of the port forwards and traffic rules I've tried along the way, mainly before I tried PBR at all. Anything PiVPN related or NASVPN related are things I've tried).

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'
	option flow_offloading_hw '1'

config zone 'wan'
	option name 'wan'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option input 'REJECT'
	list network 'PLUSNET'

config zone
	option name 'WGVPN_Zone'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'WGVPN'

config zone 'lan'
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'
	option mtu_fix '1'

config zone
	option name 'Pigeon_Zone'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'PIGEON_VLAN'

config zone
	option name 'IOT_Zone'
	option output 'ACCEPT'
	option forward 'REJECT'
	option input 'ACCEPT'
	list network 'IOT_VLAN'

config zone
	option name 'Guest_Zone'
	option output 'ACCEPT'
	option forward 'REJECT'
	option input 'REJECT'
	list network 'GUEST_VLAN'

config zone
	option name 'Work_Zone'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'WORK_VLAN'

config rule
	option name 'Allow IOT DHCP and DNS'
	option src 'IOT_Zone'
	option dest_port '53 67 68'
	option target 'ACCEPT'

config rule
	option name 'Allow Guest DHCP and DNS'
	option src 'Guest_Zone'
	option dest_port '53 67 68'
	option target 'ACCEPT'

config rule
	option name 'Allow Work DHCP and DNS'
	option src 'Work_Zone'
	option dest_port '53 67 68'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'
	option enabled '0'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config rule 'wg'
	option name 'Allow-WireGuard'
	option src 'wan'
	option dest_port '51822'
	option proto 'udp'
	option target 'ACCEPT'
	option enabled '0'

config rule
	option name 'Allow-NASVPN'
	list proto 'udp'
	option src 'wan'
	option dest_port '1195'
	option target 'ACCEPT'

config rule
	option name 'Allow-PiVPN'
	list proto 'udp'
	option src 'wan'
	option dest_port '1196'
	option target 'ACCEPT'

config rule
	option name 'Allow-OWL'
	option src 'wan'
	option dest '*'
	option dest_port '5100'
	option target 'ACCEPT'

config rule
	option name 'Accept SSH (22)'
	option src 'wan'
	option dest_port '22'
	option target 'ACCEPT'
	list proto 'tcp'

config rule
	option name 'Accept SSH (80)'
	list proto 'tcp'
	option dest_port '80'
	option target 'ACCEPT'
	option src 'wan'

config rule
	option name 'Allow xxxto see xxx'
	option src 'IOT_Zone'
	list src_ip '192.168.20.50'
	option dest 'Pigeon_Zone'
	list dest_ip '192.168.76.202'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'
	list proto 'icmp'

config rule
	option name 'Allow mDNS'
	list proto 'udp'
	option src '*'
	option src_port '5353'
	list dest_ip '224.0.0.251'
	option dest_port '5353'
	option target 'ACCEPT'

config rule
	option name 'Allow xxx to be casted to by required devices'
	option src 'IOT_Zone'
	option dest 'Pigeon_Zone'
	option target 'ACCEPT'
	list src_ip '192.168.20.7'
	list proto 'all'
	list dest_ip '192.168.76.3'
	list dest_ip '192.168.76.6'
	list dest_ip '192.168.76.14'
	list dest_ip '192.168.76.15'

config rule
	option name 'Allow xxx to see xxx'
	option src 'IOT_Zone'
	list src_ip '192.168.20.7'
	option dest 'Pigeon_Zone'
	list dest_ip '192.168.76.202'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'
	list proto 'icmp'
	list proto 'igmp'
	list proto 'esp'
	option enabled '0'

config rule
	option name 'Allow xxx to see xxx'
	option src 'IOT_Zone'
	list src_ip '192.168.20.3'
	list src_ip '192.168.20.4'
	option dest 'Pigeon_Zone'
	list dest_ip '192.168.76.202'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'
	list proto 'icmp'
	list proto 'igmp'
	list proto 'esp'

config forwarding
	option src 'lan'
	option dest 'wan'

config forwarding
	option src 'Pigeon_Zone'
	option dest 'WGVPN_Zone'

config forwarding
	option src 'Pigeon_Zone'
	option dest 'IOT_Zone'

config forwarding
	option src 'Pigeon_Zone'
	option dest 'Guest_Zone'

config forwarding
	option src 'IOT_Zone'
	option dest 'wan'

config forwarding
	option src 'Guest_Zone'
	option dest 'wan'

config forwarding
	option src 'Work_Zone'
	option dest 'wan'

config redirect
	option dest 'Pigeon_Zone'
	option target 'DNAT'
	option name 'NASVPN'
	option src 'wan'
	option src_dport '1195'
	option dest_port '1195'
	option dest_ip '192.168.76.202'

config redirect
	option dest 'Pigeon_Zone'
	option target 'DNAT'
	option name 'PiVPN'
	option src 'wan'
	option src_dport '1196'
	option dest_ip '192.168.76.50'
	option dest_port '1196'

config redirect
	option dest 'Pigeon_Zone'
	option target 'DNAT'
	option name 'OWL'
	list proto 'udp'
	option src 'wan'
	option src_dport '5100'
	option dest_ip '192.168.76.2'
	option dest_port '5100'
	option enabled '0'

config redirect 'adblock_Guest_Zone53'
	option name 'Adblock DNS (Guest_Zone, 53)'
	option src 'Guest_Zone'
	option proto 'tcp udp'
	option src_dport '53'
	option dest_port '53'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.30.1'
	option dest_ip '192.168.30.1'

config redirect 'adblock_Guest_Zone853'
	option name 'Adblock DNS (Guest_Zone, 853)'
	option src 'Guest_Zone'
	option proto 'tcp udp'
	option src_dport '853'
	option dest_port '853'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.30.1'
	option dest_ip '192.168.30.1'

config redirect 'adblock_Guest_Zone5353'
	option name 'Adblock DNS (Guest_Zone, 5353)'
	option src 'Guest_Zone'
	option proto 'tcp udp'
	option src_dport '5353'
	option dest_port '5353'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.30.1'
	option dest_ip '192.168.30.1'

config redirect 'adblock_IOT_Zone53'
	option name 'Adblock DNS (IOT_Zone, 53)'
	option src 'IOT_Zone'
	option proto 'tcp udp'
	option src_dport '53'
	option dest_port '53'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.20.1'
	option dest_ip '192.168.20.1'

config redirect 'adblock_IOT_Zone853'
	option name 'Adblock DNS (IOT_Zone, 853)'
	option src 'IOT_Zone'
	option proto 'tcp udp'
	option src_dport '853'
	option dest_port '853'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.20.1'
	option dest_ip '192.168.20.1'

config redirect 'adblock_IOT_Zone5353'
	option name 'Adblock DNS (IOT_Zone, 5353)'
	option src 'IOT_Zone'
	option proto 'tcp udp'
	option src_dport '5353'
	option dest_port '5353'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.20.1'
	option dest_ip '192.168.20.1'

config redirect 'adblock_Pigeon_Zone53'
	option name 'Adblock DNS (Pigeon_Zone, 53)'
	option src 'Pigeon_Zone'
	option proto 'tcp udp'
	option src_dport '53'
	option dest_port '53'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.76.1'
	option dest_ip '192.168.76.1'

config redirect 'adblock_Pigeon_Zone853'
	option name 'Adblock DNS (Pigeon_Zone, 853)'
	option src 'Pigeon_Zone'
	option proto 'tcp udp'
	option src_dport '853'
	option dest_port '853'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.76.1'
	option dest_ip '192.168.76.1'

config redirect 'adblock_Pigeon_Zone5353'
	option name 'Adblock DNS (Pigeon_Zone, 5353)'
	option src 'Pigeon_Zone'
	option proto 'tcp udp'
	option src_dport '5353'
	option dest_port '5353'
	option target 'DNAT'
	option family 'ipv4'
	option src_ip '!192.168.76.1'
	option dest_ip '192.168.76.1'

config rule
	option name 'PiVPN-IN'
	option src 'wan'
	option src_port '1196'
	option dest 'Pigeon_Zone'
	list dest_ip '192.168.76.50'
	option dest_port '1196'
	option target 'ACCEPT'
	option enabled '0'

config rule
	option name 'PiVPN-OUT'
	option src 'Pigeon_Zone'
	list src_ip '192.168.76.50'
	option src_port '1196'
	option dest 'wan'
	option dest_port '1196'
	option target 'ACCEPT'
	option enabled '0'

config rule
	option name 'NASVPN-IN'
	option src 'wan'
	option src_port '1195'
	option dest 'Pigeon_Zone'
	list dest_ip '192.168.76.202'
	option dest_port '1195'
	option target 'ACCEPT'
	option enabled '0'

config rule
	option name 'NASVPN-OUT'
	option src 'Pigeon_Zone'
	list src_ip '192.168.76.202'
	option src_port '1195'
	option dest 'wan'
	option dest_port '1195'
	option target 'ACCEPT'
	option enabled '0'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/firewall.include'

pbr
I read here

Welcome to docs.openwrt.melmac.net!

that you have to set option procd_wan_interface and option procd_lan_interface if you've used non standard names, so I've done that as you'll see.

I've also added PLUSNET as a supported interface, as it wasn't listed in the options in luci interface.

Finally, I've experimented with enabling the pbr.user.wg_server_and_client rule (came as an example with the pbr software) as I do indeed have a wireguard client (WGVPN to Surfshark) and wireguard servers (piVPN and NASVPN) running in my network (albeit the servers aren't on the router).

config pbr 'config'
	option enabled '0'
	option verbosity '2'
	option strict_enforcement '1'
	option resolver_set 'none'
	list resolver_instance '*'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver'
	option boot_timeout '30'
	option rule_create_option 'add'
	option procd_boot_delay '0'
	option procd_reload_delay '1'
	option procd_wan_interface 'PLUSNET'
	option procd_lan_interface 'PIGEON_VPN'
	option webui_show_ignore_target '0'
	option nft_rule_counter '0'
	option nft_set_auto_merge '1'
	option nft_set_counter '0'
	option nft_set_flags_interval '1'
	option nft_set_flags_timeout '0'
	option nft_set_policy 'performance'
	list webui_supported_protocol 'all'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	list supported_interface 'PLUSNET'

config include
	option path '/usr/share/pbr/pbr.user.wg_server_and_client'
	option enabled '1'

config policy
	option name 'Ignore Local Requests'
	option interface 'ignore'
	option dest_addr '10.0.0.0/24 10.0.1.0/24 192.168.100.0/24 192.168.1.0/24'
	option enabled '0'

config policy
	option name 'PiVPN'
	option src_addr '192.168.76.50'
	option interface 'PLUSNET'
	option dest_port '1196'

config policy
	option name 'NASVPN'
	option src_addr '192.168.76.202'
	option interface 'PLUSNET'
	option dest_port '1195'

I've made some progress - my pbr luci app looks like this:

But I just can't get the VPN apps on my phone to connect.

Can anyone tell me which of the port forwards, traffic rules and pbr policies are required and where I've gone wrong with the ones that should be there?

Many thanks for your help. (I'll be away for this work for a few days, so please don't think I'm ignoring any responses you come back with!)

[egc]

@aal7

Essentially what I think I'm trying to do is allow traffic on my incoming VPN (but only trafic on my incoming VPN) on PLUSNET to get to PIGEON so it can access the Pi and the NAS

If that is what you want you do not need PBR.
PBR is for when you cannot connect to your VPN server but that is not the problem if I understand you.

The problem is to allow access for your VPN clients to the PIGEON subnet, or did I misunderstood that?

[aal7]

Yes, under normal circumstances PIGEON connects with the outside world via WGVPN (Surfshark VPN) and IOT, GUEST and WORK connect with the outside world via PLUSNET (no VPN). When I'm away from home I need access to PIGEON through a VPN client on my phone and a VPN server in my network (in PIGEON). (As far as I know) I can only come in from outside on PLUSNET, and so for that VPN (and only that VPN) I want to link PLUSNET and PIGEON together. Hopefully that's cleared thing up. Thanks for your help.