Reaching a Device on a VPN Bypass Subnet

marcusus · August 6, 2021, 3:41pm

Hi folks. I've been trawling the forum for ages to try and get my config going, and believe I'm almost there but am just missing that final step...

The end result I'm after is that I want my Apple TV device to bypass the VPN, but I also want to still be able to reach the device from the rest of the network for AirPlay purposes.

What I've currently done:

OpenVPN is installed and working.
There are two VLANs. lan and non_vpn_lan. Their subnets are 192.168.1.0/24 and 192.168.2.0/24 respectively.
I have the VPN Bypass package installed, and have the Local IP Addresses to Bypass field populated with 192.168.2.0/24.
lan connects correctly to the internet via the VPN.
non_vpn_lan connects correctly to the internet bypassing the VPN.
Zone forwardings are as per the following screenshot:
image962×300 20.9 KB
I have the avahi-utils package installed to facilitate Apple's Bonjour service across the VLANs.

My Apple devices on the 192.168.1.0 subnet can see the Apple TV device, but when I attempt to AirPlay, I get an error. If I stop VPN Bypass, AirPlay works but the Apple TV can no longer reach the internet.

I've also tested with a computer connected to both subnets via physical ethernet.
If VPN Bypass is enabled, both computers can reach the internet but are unable to ping each other.
If VPN Bypass is disabled, the computer on subnet 192.168.1.0 can reach the internet, the computer on subnet 192.168.2.0 cannot reach the internet, and both computers can ping each other.

Ideally it would be good to keep the two VLANs, but if it's easier to put the Apple TV back on the lan VLAN and configure the one device to bypass the VPN whilst allowing connectivity to it from the rest of the network, I'm OK with that.

/etc/config/network is as follows:

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'xxxx:xxxx:xxxx::/48'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ifname 'eth0.17'

config interface 'wan'
        option proto 'dhcp'
        option ifname 'eth1.7'

config interface 'wan6'
        option proto 'dhcpv6'
        option ifname 'eth1.7'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option vid '17'
        option ports '2 3 5t'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6t'
        option vid '7'

config switch_vlan
        option device 'switch0'
        option vlan '3'
        option vid '27'
        option ports '0 1 5t'

config interface 'non_vpn_lan'
        option ifname 'eth0.27'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.2.1'
        option type 'bridge'

/etc/config/firewall is as follows:

config defaults
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan wan6'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config include
        option path '/etc/firewall.user'

config zone
        option input 'ACCEPT'
        option forward 'ACCEPT'
        option output 'ACCEPT'
        option name 'non_vpn_lan'
        option network 'non_vpn_lan'

config forwarding
        option src 'lan'
        option dest 'non_vpn_lan'

config forwarding
        option dest 'lan'
        option src 'non_vpn_lan'

config zone
        option input 'REJECT'
        option forward 'REJECT'
        option name 'wan_via_vpn'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        list device 'tun0'

config forwarding
        option dest 'wan_via_vpn'
        option src 'lan'

config forwarding
        option dest 'wan'
        option src 'non_vpn_lan'

Appreciate any help, and thanks in advance!

iplaywithtoys · August 6, 2021, 5:08pm

I'm unfamiliar with the "VPN Bypass" package you mention. I can guess what it's supposed to do, from your description of your problem and the name of the package.

My first thought when I read your description was, "Why not just use policy-based routing?" However, you've not included your routing table/configuration so one can only speculate.

Configure the router to send traffic from 192.168.2.0/24 out the front door, and traffic from 192.168.1.0/24 down the VPN tunnel.

The routing decision method usually runs from most-specific to least-specific. The two Directly Connected routes for 192.168.1.0/24 and .2.0/24 are most-specific so have the highest priority, which means traffic between the subnets should always flow irrespective of any VPN (subject of course to the firewall rules).

The two default routes to the Internet (0.0.0.0/0) are least-specific so have the lowest priority (unless you've adjusted the metric values...)

marcusus · August 7, 2021, 12:51am

Are you referring to the contents of /etc/iproute2/rt_tables ? If so, see the below:

#
# reserved values
#
128     prelocal
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
201 lan
202 wan
203 non_vpn_lan

I thought the above was just a mapping of the table number to a human readable name?

None of the metrics have been adjusted that I'm aware of.

stangri · August 7, 2021, 3:53am

I'd recommend you switch to vpn-policy-routing and use IGNORE target, I believe the VPR README has an example for this specific use case.

marcusus · August 7, 2021, 4:08am

Thanks @stangri for the suggestion. I'll do some reading and see if I can work it out. And a special thank you as well for creating both the packages!

vgaetera · August 7, 2021, 5:36am

marcusus · August 7, 2021, 7:05am

Thanks for the link. I tried what was suggested, but it broke internet to both subnets.

I added the line pull-filter ignore "redirect-gateway" to my ovpn file, and that breaks the internet for the .1/24 subnet.

I then did the following:

/etc/init.d/rpcd restart
uci -q delete vpn-policy-routing.config.dest_ipset
uci set vpn-policy-routing.config.enabled="1"
uci set vpn-policy-routing.config.webui_show_ignore_target="1"
while uci -q delete vpn-policy-routing.@policy[0]; do :; done
uci add vpn-policy-routing policy
uci set vpn-policy-routing.@policy[-1].dest_addr="192.168.1.0/24 192.168.2.0/24"
uci set vpn-policy-routing.@policy[-1].interface="ignore"
uci add vpn-policy-routing policy
uci set vpn-policy-routing.@policy[-1].src_addr="192.168.2.0/24"
uci set vpn-policy-routing.@policy[-1].interface="non_vpn_lan"
uci commit vpn-policy-routing
/etc/init.d/vpn-policy-routing restart

And that's where I end up with internet being broken for both. When the above configuration is active, I can ping devices on the .2/24 subnet from the .1/24 subnet, and vice versa.

What would be involved in putting everything back on a single subnet, and using policy routing to route the single device outside of the VPN? This should still maintain connectivity to the device, right?

vgaetera · August 7, 2021, 7:09am

marcusus · August 8, 2021, 1:50am

OK I've taken things back a step, and am going to try and get a single device at 192.168.1.235 to not use the VPN.

I've followed the instructions here for setting up OpenVPN:

I did have to add tun0 to the Covered Devices section to get connectivity via the VPN for the .1/24 subnet.

When I add the 192.168.1.235 device to VPN Policy Routing and set it to the WAN interface, it continues to show the VPN IP address when I use dnsleaktest.com

I just need to work out how to get that one device to use the WAN properly, and it should do what I need it to do.

Configs are below.

/etc/config/vpn-policy-routing

config vpn-policy-routing 'config'
        option verbosity '2'
        option src_ipset '0'
        option dest_ipset '0'
        option resolver_ipset 'dnsmasq.ipset'
        option ipv6_enabled '0'
        list ignored_interface 'vpnserver wgserver'
        option boot_timeout '30'
        option iptables_rule_option 'append'
        option procd_reload_delay '1'
        option webui_sorting '1'
        list webui_supported_protocol 'tcp'
        list webui_supported_protocol 'udp'
        list webui_supported_protocol 'tcp udp'
        list webui_supported_protocol 'icmp'
        list webui_supported_protocol 'all'
        option enabled '1'
        list supported_interface 'vpnc'
        option webui_enable_column '1'
        option webui_protocol_column '1'
        option webui_chain_column '1'
        option webui_show_ignore_target '1'
        option strict_enforcement '0'

config include
        option path '/etc/vpn-policy-routing.netflix.user'
        option enabled '0'

config include
        option path '/etc/vpn-policy-routing.aws.user'
        option enabled '0'

config policy
        option name 'test'
        option src_addr '192.168.1.235'
        option interface 'wan'

/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'xxxx:xxxx:xxxx::/48'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ifname 'eth0.1 eth0.17'

config interface 'wan'
        option proto 'dhcp'
        option ifname 'eth1.7'

config interface 'wan6'
        option proto 'dhcpv6'
        option ifname 'eth1.7'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 5t'
        option vid '17'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6t'
        option vid '7'

config interface 'vpnc'
        option proto 'none'
        option ifname 'ovpnc0'

/etc/config/firewall

config defaults
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'

config zone
        option name 'wan'
        list network 'wan'
        list network 'wan6'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'

### Default 'Allow' rules here.  Removed for readability.

config include
        option path '/etc/firewall.user'

config zone
        option name 'vpnc'
        option network 'vpnc'
        option input 'REJECT'
        option forward 'REJECT'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        list device 'tun0'

config forwarding
        option src 'lan'
        option dest 'vpnc'

/etc/config/openvpn

config openvpn 'vpnc'
        option config '/etc/openvpn/openvpn.ovpn'
        option enabled '1'
        option client '1'
        option dev_type 'tun'
        option dev 'ovpnc0'

/etc/openvpn/openvpn.ovpn

dev tun
fast-io
persist-key
persist-tun
nobind
remote xxxx.com 1195

remote-random
pull
comp-lzo no
tls-client
verify-x509-name Server name-prefix
remote-cert-tls server
key-direction 1
route-method exe
route-delay 2
tun-mtu 1500
fragment 1300
mssfix 1200
verb 3
cipher AES-256-CBC
auth SHA512
sndbuf 524288
rcvbuf 524288
auth-user-pass /etc/openvpn/xxx.auth

### Certificate information follows

vgaetera · August 8, 2021, 8:05am

uci set network.vpnc.ifname="tun0"
uci commit network
/etc/init.d/network restart
sleep 10
uci -q delete openvpn.vpnc.client
uci -q delete openvpn.vpnc.dev_type
uci -q delete openvpn.vpnc.dev
uci commit openvpn
/etc/init.d/openvpn restart

marcusus · August 8, 2021, 11:50am

I reckon that's done it! Thanks so much!

For future reference, the final configurations are below.

/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'xxxx:xxxx:xxxx::/48'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ifname 'eth0.1 eth0.17'

config interface 'wan'
        option proto 'dhcp'
        option ifname 'eth1.7'

config interface 'wan6'
        option proto 'dhcpv6'
        option ifname 'eth1.7'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 5t'
        option vid '17'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6t'
        option vid '7'

config interface 'vpnc'
        option proto 'none'
        option ifname 'tun0'

/etc/config/openvpn

config openvpn 'vpnc'
        option config '/etc/openvpn/openvpn.ovpn'
        option enabled '1'

Thanks again to all that helped. I'll keep tweaking to see if I can get it back to the two subnet system, but at least I have a known working config now. Much appreciated!

marcusus · August 8, 2021, 12:43pm

In the interests of understanding what has been configured, am I right in the following statements?

The way the OpenVPN file is configured, once it connects, all traffic is routed via the VPN. If this is not desired, the configuration would need to be changed to include pull-filter ignore "redirect-gateway" as a starting point, followed by additional configuration.
In the Firewall configuration, the lan zone is forwarded to both the vpnc zone and the wan zone. Since OpenVPN is running, all traffic routes through the VPN, and the wan route here effectively is overridden.
In vpn-policy-routing, when we specify the source address to route to the wan zone, it overrides the OpenVPN routes, however it requires the lan to wan forwarding in the Firewall configuration. Otherwise the missing firewall rule would prevent the source address (which resides in the lan interface) from reaching the wan.

vgaetera · August 8, 2021, 1:04pm

You are mostly correct, but missing some corner cases:

The VPN connection can fail at any moment and all your traffic will go to WAN.
The VPN server is not actually obligated to push redirect-gateway.

Both issues can be solved with an explicit routing policy.

marcusus · August 11, 2021, 11:59am

Apologies for the late reply, and thanks for confirming.

Understood regarding the VPN going through WAN if the VPN goes down.

Can you explain what you mean by the server not being obligated to push redirect-gateway? What is happening when pull-filter ignore "redirect-gateway" is being used?

vgaetera · August 11, 2021, 12:08pm

The server can be intentionally or unintentionally configured to not push redirect-gateway.

marcusus · August 11, 2021, 12:13pm

So when redirect-gateway is pushed from the OpenVPN connection, it instructs the router to route all traffic through the VPN?

And if I specify pull-filter ignore "redirect-gateway" in the config, it will result in the redirect gateway command being ignored? This would then give the ability to specify which connections go through OpenVPN, and which ones to not go through OpenVPN?

system · August 21, 2021, 12:13pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.