Remote VPN network unreachable to router but OK for internal LAN

pedropaislopes · March 16, 2021, 5:42pm

Hi folks!

(Sorry for any double post)

After a lot of work (and learning a lot of things!) I've managed to connect two almost identical network topologies using Wireguard VPN, VPN Policy Routing and firewall rules.

Network topology and names

The following image describe names, interfaces (IF) and addresses with netmask. My setup can be divided in two: the OpenWRT1 side with its own LAN and LANEXA networks, and OpenWRT2 with its own LAN and LANEXA networks.

Wireguard VPN is working fine but it is not persistent after power on line failures. A small script (below) takes care of reseting VPN connection (installed on both sides):

/root/check-vpn

#!/bin/sh

ping -W 3 -c 3 -q -A 192.168.9.2 > /dev/null 2>&1
if [ $? -ne 0 ]; then
  ifup vpn
fi

/etc/crontabs/root

* * * * * /usr/bin/wireguard_watchdog
* * * * * /root/checa-vpn

Firewall permissions

Basically identical on both sides:

LAN can access LANEXA, OpenWRT and INTERNET. Should access the other OpenWRT and LANEXA using VPN link.
LANEXA can access OpenWRT and INTERNET. Should access the other OpenWRT and LANEXA using VPN link.
One LAN should not access the other side LAN.
LANEXA should not access any LAN.

The following OpenWRT firewall zones permissions table is identical on both sides:

Every Zone has a network interface attached with the same name (i.e., lan zone has LAN interface). wan Zone has two interfaces: wan and wan6.

Configuration files

On OpenWRT1:

/etc/config/vpn-policy-routing

config vpn-policy-routing 'config'
 	option enabled '1'
 	option strict_enforcement '0'
 	option dest_ipset 'dnsmasq.ipset'
 	option src_ipset '0'
 	option ipv6_enabled '0'
 	option iptables_rule_option 'append'
 	option iprule_enabled '0'
 	option webui_enable_column '0'
 	option webui_protocol_column '0'
	option webui_chain_column '0'
 	option webui_sorting '1'
 	list webui_supported_protocol 'tcp'
 	list webui_supported_protocol 'udp'
 	list webui_supported_protocol 'tcp udp'
 	list webui_supported_protocol 'icmp'
 	list webui_supported_protocol 'all'
 	option verbosity '2'
 	option boot_timeout '30'
 
 config policy
 	option name 'lanexasjc'
 	option interface 'vpn'
 	option dest_addr '192.168.33.0/24 192.168.1.0/24'

/etc/config/network

config interface 'loopback'
   option ifname 'lo'
   option proto 'static'
   option ipaddr '127.0.0.1'
   option netmask '255.0.0.0'

config globals 'globals'
   option ula_prefix 'fdf0:ed02:a420::/48'

config device 'lan_eth0_1_dev'
   option name 'eth0.1'
   option macaddr '48:8f:xxxxx'

config interface 'lan'
   option ifname 'eth0.1'
   option proto 'static'
   option netmask '255.255.255.0'
   option ip6assign '60'
   option ipaddr '192.168.143.1'

config device 'wan_eth0_2_dev'
   option name 'eth0.2'
   option macaddr '48:8f:xxxxx'

config interface 'wan'
   option ifname 'eth0.2'
   option proto 'pppoe'
   option password 'xxxxx'
   option ipv6 'auto'
   option username 'xxxxx@xxxxx'

config interface 'wan_modem'
   option ifname 'eth0.2'
   option proto 'static'
   option ipaddr '192.168.15.2'
   option netmask '255.255.255.0'

config switch
   option name 'switch0'
   option reset '1'
   option enable_vlan '1'

config switch_vlan
   option device 'switch0'
   option vlan '1'
   option ports '1 2 3 4 6t'

config switch_vlan
   option device 'switch0'
   option vlan '2'
   option ports '0 6t'

config interface 'vpn'
   option proto 'wireguard'
   option private_key 'xxxxx'
   option listen_port '51820'
   list addresses '192.168.9.1/24'
   list addresses 'fdf1:e8a1:8d3f:9::1/64'

config wireguard_vpn 'wgclient'
   option public_key 'xxxxx'
   option preshared_key 'xxxxx
   list allowed_ips '0.0.0.0/0'
   list allowed_ips '::0/0'
   option description 'vpnexa'

config interface 'lanexa'
   option ifname 'veth4'
   option proto 'static'
   option ipaddr '192.168.34.1'
   option netmask '255.255.255.0'

/etc/config/firewall

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone 'lan'
	option name 'lan'
	option output 'ACCEPT'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'

config zone
	option name 'lanexa'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	option output 'ACCEPT'
	option network 'lanexa'

config zone
	option name 'vpn'
	option network 'vpn'
	option output 'ACCEPT'
	option forward 'REJECT'
	option input 'ACCEPT'

config zone 'wan'
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option network 'wan wan_modem'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config rule 'wg'
	option name 'Allow-WireGuard'
	option src 'wan'
	option dest_port '51820'
	option proto 'udp'
	option target 'ACCEPT'

config forwarding
	option dest 'wan'
	option src 'lanexa'

config forwarding
	option dest 'lanexa'
	option src 'lan'

config forwarding
	option dest 'vpn'
	option src 'lan'

config forwarding
	option dest 'vpn'
	option src 'lanexa'

config rule
	option src '*'
	option name 'Allow-vpn-other-side'
	option dest 'lanexa'
	option target 'ACCEPT'
	list proto 'all'

On OpenWRT2:

/etc/config/vpn-policy-routing

config policy
	option name 'lanexasp'
	option interface 'vpn'
	option dest_addr '192.168.34.0/24 192.168.143.0/24'

config vpn-policy-routing 'config'
	option verbosity '2'
	option src_ipset '0'
	option dest_ipset 'dnsmasq.ipset'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver wgserver'
	option boot_timeout '30'
	option iptables_rule_option 'append'
	option iprule_enabled '0'
	option webui_enable_column '0'
	option webui_protocol_column '0'
	option webui_chain_column '0'
	option webui_sorting '1'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	list webui_supported_protocol 'all'
	option strict_enforcement '0'
	option enabled '1'

/etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fded:f63d:4e07::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth0.1'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option igmp_snooping '1'
	option ip6assign '64'

config device 'lan_eth0_1_dev'
	option name 'eth0.1'
	option macaddr '48:8f:xxxxxx'

config interface 'wan'
	option ifname 'eth0.2'
	option proto 'static'
	option ip6assign '64'
	option netmask '255.255.255.0'
	list dns '8.8.8.8'
	option ipaddr '192.168.100.10'
	option gateway '192.168.100.1'

config device 'wan_eth0_2_dev'
	option name 'eth0.2'
	option macaddr '48:8f:xxxxx'

config interface 'wan6'
	option ifname 'eth0.2'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '1 2 3 6t'
	option vid '1'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '0 6t'
	option vid '2'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option ports '4 6t'
	option vid '3'

config interface 'LANEXA'
	option ifname 'eth0.3'
	option proto 'static'
	option netmask '255.255.255.0'
	option type 'bridge'
	option ipaddr '192.168.33.1'
	option ip6assign '64'

config interface 'vpn'
	option proto 'wireguard'
	option private_key 'xxxxxxx'
	list addresses '192.168.9.2/24'
	list addresses 'fdf1:e8a1:8d3f:9::2/64'

config wireguard_vpn 'wgserver'
	option public_key 'xxxxxxx'
	option preshared_key 'xxxxxxx'
	option endpoint_host 'OpenWRT1 static IP'
	option endpoint_port '51820'
	option persistent_keepalive '25'
	list allowed_ips '0.0.0.0/0'
	list allowed_ips '::/0'
	option route_allowed_ips '0'

/etc/config/firewall

config defaults
	option forward 'REJECT'
	option flow_offloading '1'
	option synflood_protect '1'
	option input 'REJECT'
	option output 'REJECT'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'lanexa'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	option network 'LANEXA'
	option output 'ACCEPT'

config zone
	option name 'vpn'
	option input 'ACCEPT'
	option forward 'REJECT'
	option network 'vpn'
	option output 'ACCEPT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config forwarding
	option dest 'wan'
	option src 'lanexa'

config forwarding
	option dest 'lanexa'
	option src 'lan'

config rule
	option src 'wan'
	option name 'Allow-IPv6-Forward'
	option dest 'lanexa'
	option target 'ACCEPT'
	list proto 'all'
	option family 'ipv6'
	option enabled '0'

config forwarding
	option dest 'vpn'
	option src 'lan'

config forwarding
	option dest 'vpn'
	option src 'lanexa'

config rule
	option src '*'
	option name 'Allow-vpn-other-side'
	option target 'ACCEPT'
	list proto 'all'
	option dest 'lanexa'

Actual status

Everything works fine. All permissions described above works.

Issues

One little issue is a firewall traffic rule called "Allow-vpn-other-side". After a lot of try-and-error only with this rule all traffic permissions works. If anyone can point me why I need this rule I appreciated.

The major issue (a new functionality) is driving me crazy. OpenWRT router itself (a connection from its box) should access LANEXA on the other side, using VPN connection for this to work (of course). And I really can't find where is the issue - traffic rule, firewall zones, VPN Policy Routing, or other piece of configuration.

Any thoughts? Ideas?

Cheers!

trendy · March 16, 2021, 6:07pm

VPN-PBR doesn't look necessary to me. You can add a couple of static routes, or make the allowed_ips list in WG peer configuration more specific and routeable.
Also the firewall is incomplete. For example you allow lan->vpn forwarding, but on the other side you don't want to allow one lan to connect to the other. The 'Allow-vpn-other-side' will allow everything from everywhere to lanexa.

No route.

vgaetera · March 17, 2021, 12:04am

The client and server configs can be simplified:

Replace the current allowed_ips with the respective peer's subnets/IPs.
Enable route_allowed_ips on both peers.

It's basically described in the wiki:

https://openwrt.org/docs/guide-user/services/vpn/wireguard/extras#site-to-site

This way you don't need to add any extra routes.

pedropaislopes · March 17, 2021, 1:59pm

(sorry for late answer)

@trendy thanks for the quick response!
@vgaetera thanks too!

First, I've disabled VPN-PBR. Then routing is made as suggested using allowed_ips on VPN Wireguard, enabling routing too. To be specific:

On OpenWRT1, allowed_ips are 192.168.33.0/24, 192.168.1.0/24 and 192.168.9.0/24.

Open OpenWRT2, allowed_ips are 192.168.34.0/24, 192.168.143.0/24 and 192.168.9.0/24.

The traffic rule 'Allow-vpn-other-side' was removed and the new Zones permissions is:

As @trendy pointed the VPN to lanexa Zone->Forwarding replaces (in a much more better way) the traffic rule added.

Thank you very much!

Cheers!

system · March 27, 2021, 2:00pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.