One device not accessible via routed OpenVPN connection

jonathan.holvey · December 12, 2018, 9:55pm

I have an OpenVPN server running on my OpenWrt router. I can access most devices on the LAN subnet (192.168.0.0/24) when I'm not at home, via the VPN from it's subnet (192.168.10.0/24) with forwarding rules in the firewall.

$ ping 192.168.0.10
PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data.
64 bytes from 192.168.0.10: icmp_seq=1 ttl=63 time=54.3 ms
64 bytes from 192.168.0.10: icmp_seq=2 ttl=63 time=82.4 ms

$ traceroute 192.168.0.10
traceroute to 192.168.0.10 (192.168.0.10), 30 hops max, 60 byte packets
 1  192.168.10.1 (192.168.10.1)  98.760 ms  98.281 ms  98.268 ms
 2  192.168.0.10 (192.168.0.10)  98.267 ms  101.425 ms  101.502 ms

My main computer, however, can't be accessed. Ping and traceroute requests timeout.

$ ping 192.168.0.20
PING 192.168.0.20 (192.168.0.20) 56(84) bytes of data.
^C
--- 192.168.0.20 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 7175ms

$ traceroute 192.168.0.20
traceroute to 192.168.0.20 (192.168.0.20), 30 hops max, 60 byte packets
 1  192.168.10.1 (192.168.10.1)  136.147 ms  243.007 ms  400.680 ms
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *

Both these machines are configured in the same way with DHCP, and are running Debian-based operating systems.

What can I do to troubleshoot this issue?

Firewall config
Network config
OpenVPN server config
The client is configured using network-manager-openvpn, with the option Use this connection only for resources on its network checked.

psherman · December 12, 2018, 10:46pm

Is the main computer Windows based? I believe that the windows firewall, by default, blocks communication with RFC1918 addresses that are outside the windows network environment (so a windows machine on the 192.168.0.0/24 network would not be able to communicate with a device on any other network in the RFC1918 defined scopes). I don't know how to change it, but I know that it this can be overridden.

mpa · December 13, 2018, 12:18pm

Run tcpdump or wireshark on 192.168.0.20 to see if the traceroute and ping probes arrive.
Maybe a personal firewall is active on the target machine?

jonathan.holvey · December 23, 2018, 8:40am

Yes, tcpdump shows that the ping packets do in fact arrive at the problem computer. Any idea what might need to change if the firewall is blocking it? I'm running Debian Buster with basically an out-of-the-box configuration.

Interestingly, pings from the problem computer to VPN clients on the 192.168.10.0/42 subnet also fail, but I haven't yet checked to see if the packets arrive or not.

jonathan.holvey · January 10, 2019, 10:28am

After a little more investigation, I can see that the problem computer is in fact receiving and replying to the pings, but the response doesn't make it back to the pinging computer on the VPN subnet.

Pings from the problem computer never reach clients on the VPN subnet either, and traceroute ends up on the WAN by the looks of it, so I'm pretty sure it's a routing problem.

 1  router.lan (192.168.0.1)  0.347 ms  0.479 ms  0.553 ms
 2  static.belong.com.au (xxx.xxx.xxx.xxx)  17.200 ms  18.940 ms  19.540 ms
 3  static.belong.com.au (141.168.8.37)  23.524 ms  24.219 ms  26.149 ms
 4  Bundle-Ether21.cha-edge901.brisbane.telstra.net (110.145.218.177)  25.420 ms  26.355 ms  27.326 ms
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *

What's going wrong here, and how can I fix it? Why isn't this a problem for other computers on the network?

vgaetera · January 10, 2019, 10:52am

Add route to VPN-network via VPN-gateway either on default gateway or on that PC.

jonathan.holvey · January 14, 2019, 11:33am

I tried adding the route to the problem computer using ip, but it returns an error:

$ sudo ip route add 192.168.10.0/24 via 192.168.10.1 dev eth0
Error: Nexthop has invalid gateway.

Is there a problem with the command I'm using?

vgaetera · January 14, 2019, 12:22pm

@openwrt-router:

ip -4 a; ip -4 r; ip -4 ru

@192.168.0.20:

ip -4 a; ip -4 r

jonathan.holvey · January 22, 2019, 8:08am

Please could you explain what this does?

trendy · January 22, 2019, 11:33am

They are commands that show the ipv4 address, routes and rules.
Also post here the output of iptables -L -vn on the debian. You might need to run it with sudo. This is the firewall configuration.

jonathan.holvey · January 29, 2019, 9:51am

From 192.168.0.20:

$ ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 192.168.0.20/24 brd 192.168.0.255 scope global dynamic noprefixroute enp7s0
       valid_lft 86115sec preferred_lft 86115sec
4: br-34df59bd6d99: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-34df59bd6d99
       valid_lft forever preferred_lft forever
5: br-63f05d604645: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    inet 172.20.0.1/16 brd 172.20.255.255 scope global br-63f05d604645
       valid_lft forever preferred_lft forever
6: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
7: br-f25fecbfce9f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    inet 172.19.0.1/16 brd 172.19.255.255 scope global br-f25fecbfce9f
       valid_lft forever preferred_lft forever

$ ip -4 r
default via 192.168.0.1 dev enp7s0 
default via 192.168.0.1 dev enp7s0 proto dhcp metric 100 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 
172.18.0.0/16 dev br-34df59bd6d99 proto kernel scope link src 172.18.0.1 linkdown 
172.19.0.0/16 dev br-f25fecbfce9f proto kernel scope link src 172.19.0.1 
172.20.0.0/16 dev br-63f05d604645 proto kernel scope link src 172.20.0.1 linkdown 
192.168.0.0/24 dev enp7s0 proto kernel scope link src 192.168.0.20 metric 100 

$ sudo iptables -L -vn
[sudo] password for jon: 
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 1522 1145K DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 1522 1145K DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
  906 1098K ACCEPT     all  --  *      br-f25fecbfce9f  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-f25fecbfce9f  0.0.0.0/0            0.0.0.0/0           
  616 46901 ACCEPT     all  --  br-f25fecbfce9f !br-f25fecbfce9f  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-f25fecbfce9f br-f25fecbfce9f  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      br-63f05d604645  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-63f05d604645  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-63f05d604645 !br-63f05d604645  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-63f05d604645 br-63f05d604645  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      br-34df59bd6d99  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-34df59bd6d99  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-34df59bd6d99 !br-34df59bd6d99  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-34df59bd6d99 br-34df59bd6d99  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (4 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !br-f25fecbfce9f br-f25fecbfce9f  0.0.0.0/0            172.19.0.2           tcp dpt:80

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
  616 46901 DOCKER-ISOLATION-STAGE-2  all  --  br-f25fecbfce9f !br-f25fecbfce9f  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-63f05d604645 !br-63f05d604645  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-34df59bd6d99 !br-34df59bd6d99  0.0.0.0/0            0.0.0.0/0           
 1522 1145K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (4 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-f25fecbfce9f  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-63f05d604645  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-34df59bd6d99  0.0.0.0/0            0.0.0.0/0           
  616 46901 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 1522 1145K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0    

$ sudo iptables -t nat -L -vn
[sudo] password for jon: 
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   24  2172 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
   55  3411 MASQUERADE  all  --  *      !br-f25fecbfce9f  172.19.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  all  --  *      !br-63f05d604645  172.20.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  all  --  *      !br-34df59bd6d99  172.18.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  tcp  --  *      *       172.19.0.2           172.19.0.2           tcp dpt:80

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    1    52 RETURN     all  --  br-f25fecbfce9f *       0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  br-63f05d604645 *       0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  br-34df59bd6d99 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       tcp  --  !br-f25fecbfce9f *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:172.19.0.2:80

From the router

# ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 192.168.0.1/24 brd 192.168.0.255 scope global br-lan
       valid_lft forever preferred_lft forever
11: tun1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
    inet 192.168.10.1/24 brd 192.168.10.255 scope global tun1
       valid_lft forever preferred_lft forever
104: pppoe-wan: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN group default qlen 3
    inet xxx.xxx.xxx.xxx peer xxx.xxx.xxx.xxx/32 scope global pppoe-wan
       valid_lft forever preferred_lft forever
105: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
    inet 172.21.26.8/23 brd 172.21.27.255 scope global tun0
       valid_lft forever preferred_lft forever

# ip -4 r
default via xxx.xxx.xxx.xxx dev pppoe-wan proto static 
xxx.xxx.xxx.xxx dev pppoe-wan proto kernel scope link src xxx.xxx.xxx.xxx 
172.21.26.0/23 dev tun0 proto kernel scope link src 172.21.26.8 
192.168.0.0/24 dev br-lan proto kernel scope link src 192.168.0.1 
192.168.10.0/24 dev tun1 proto kernel scope link src 192.168.10.1 

# ip -4 ru
0:	from all lookup local 
32676:	from all fwmark 0x30000 lookup 203 
32677:	from all fwmark 0x20000 lookup 202 
32678:	from all fwmark 0x10000 lookup 201 
32766:	from all lookup main 
32767:	from all lookup default

trendy · January 29, 2019, 9:57am

on Openwrt?

Also on the router: iptables -L -vn , iptables -t nat -L -vn and iptables -t mangle -L -vn

jonathan.holvey · January 29, 2019, 10:46am

Thanks for pointing that out. Please see my edited message above.

trendy · January 29, 2019, 11:24am

These look fishy.
Post also the iptables on the router from my previous post as well as the contents of these routing tables:

ip ro list table 201
ip ro list table 202
ip ro list table 203

jonathan.holvey · February 4, 2019, 10:58am

# iptables -L -vn
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   48  3840 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
 381K   57M input_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom input rule chain */
 372K   56M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED /* !fw3 */
 1431 58160 syn_flood  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x17/0x02 /* !fw3 */
 6681  792K zone_lan_input  all  --  br-lan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
 2810  129K zone_wan_input  all  --  pppoe-wan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_wan_input  all  --  eth1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    1    40 zone_vpnclient_input  all  --  tun0   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnserver_input  all  --  tun1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
1981K 1655M forwarding_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom forwarding rule chain */
1964K 1653M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED /* !fw3 */
16852 1774K zone_lan_forward  all  --  br-lan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_wan_forward  all  --  pppoe-wan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_wan_forward  all  --  eth1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnclient_forward  all  --  tun0   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnserver_forward  all  --  tun1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 reject     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   48  3840 ACCEPT     all  --  *      lo      0.0.0.0/0            0.0.0.0/0            /* !fw3 */
 662K  859M output_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom output rule chain */
 661K  859M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED /* !fw3 */
    9  3036 zone_lan_output  all  --  *      br-lan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */
 1320 88584 zone_wan_output  all  --  *      pppoe-wan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_wan_output  all  --  *      eth1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnclient_output  all  --  *      tun0    0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnserver_output  all  --  *      tun1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain forwarding_lan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain forwarding_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain forwarding_vpnclient_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain forwarding_vpnserver_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain forwarding_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain input_lan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain input_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain input_vpnclient_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain input_vpnserver_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain input_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain output_lan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain output_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain output_vpnclient_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain output_vpnserver_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain output_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain reject (8 references)
 pkts bytes target     prot opt in     out     source               destination         
 2476  102K REJECT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */ reject-with tcp-reset
  319 26357 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */ reject-with icmp-port-unreachable

Chain syn_flood (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 1431 58160 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x17/0x02 limit: avg 25/sec burst 50 /* !fw3 */
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_lan_dest_ACCEPT (5 references)
 pkts bytes target     prot opt in     out     source               destination         
    9  3036 ACCEPT     all  --  *      br-lan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_lan_forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
16852 1774K forwarding_lan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom lan forwarding rule chain */
16852 1774K zone_wan_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Zone lan to wan forwarding policy */
13518 1541K zone_vpnclient_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Zone lan to vpnclient forwarding policy */
    0     0 zone_vpnserver_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Zone lan to vpnserver forwarding policy */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port forwards */
    0     0 zone_lan_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_lan_input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 6681  792K input_lan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom lan input rule chain */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port redirections */
 6681  792K zone_lan_src_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_lan_output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    9  3036 output_lan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom lan output rule chain */
    9  3036 zone_lan_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_lan_src_ACCEPT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 6681  792K ACCEPT     all  --  br-lan *       0.0.0.0/0            0.0.0.0/0            ctstate NEW,UNTRACKED /* !fw3 */

Chain zone_vpnclient_dest_ACCEPT (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    1   292 DROP       all  --  *      tun0    0.0.0.0/0            0.0.0.0/0            ctstate INVALID /* !fw3: Prevent NAT leakage */
13517 1541K ACCEPT     all  --  *      tun0    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnclient_dest_REJECT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 reject     all  --  *      tun0    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnclient_forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 forwarding_vpnclient_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnclient forwarding rule chain */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port forwards */
    0     0 zone_vpnclient_dest_REJECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnclient_input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    40 input_vpnclient_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnclient input rule chain */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port redirections */
    1    40 zone_vpnclient_src_REJECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnclient_output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 output_vpnclient_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnclient output rule chain */
    0     0 zone_vpnclient_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnclient_src_REJECT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    40 reject     all  --  tun0   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnserver_dest_ACCEPT (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      tun1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnserver_dest_REJECT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 reject     all  --  *      tun1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnserver_forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 forwarding_vpnserver_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnserver forwarding rule chain */
    0     0 zone_lan_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Zone vpnserver to lan forwarding policy */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port forwards */
    0     0 zone_vpnserver_dest_REJECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnserver_input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 input_vpnserver_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnserver input rule chain */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port redirections */
    0     0 zone_vpnserver_src_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnserver_output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 output_vpnserver_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnserver output rule chain */
    0     0 zone_vpnserver_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnserver_src_ACCEPT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  tun1   *       0.0.0.0/0            0.0.0.0/0            ctstate NEW,UNTRACKED /* !fw3 */

Chain zone_wan_dest_ACCEPT (2 references)
 pkts bytes target     prot opt in     out     source               destination         
  299 17574 DROP       all  --  *      pppoe-wan  0.0.0.0/0            0.0.0.0/0            ctstate INVALID /* !fw3: Prevent NAT leakage */
 4355  304K ACCEPT     all  --  *      pppoe-wan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 DROP       all  --  *      eth1    0.0.0.0/0            0.0.0.0/0            ctstate INVALID /* !fw3: Prevent NAT leakage */
    0     0 ACCEPT     all  --  *      eth1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_wan_dest_REJECT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 reject     all  --  *      pppoe-wan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 reject     all  --  *      eth1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_wan_forward (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 forwarding_wan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom wan forwarding rule chain */
    0     0 zone_lan_dest_ACCEPT  esp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Allow-IPSec-ESP */
    0     0 zone_lan_dest_ACCEPT  udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:500 /* !fw3: Allow-ISAKMP */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port forwards */
    0     0 zone_wan_dest_REJECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_wan_input (2 references)
 pkts bytes target     prot opt in     out     source               destination         
 2810  129K input_wan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom wan input rule chain */
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:68 /* !fw3: Allow-DHCP-Renew */
   15   838 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 8 /* !fw3: Allow-Ping */
    0     0 ACCEPT     2    --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Allow-IGMP */
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:1194 /* !fw3: Allow-VPN-Server-Connections */
    1    42 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:1194 /* !fw3: Allow-VPN-Server-Connections */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate DNAT /* !fw3: Accept port redirections */
 2794  128K zone_wan_src_REJECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_wan_output (2 references)
 pkts bytes target     prot opt in     out     source               destination         
 1320 88584 output_wan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom wan output rule chain */
 1320 88584 zone_wan_dest_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_wan_src_REJECT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 2794  128K reject     all  --  pppoe-wan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 reject     all  --  eth1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

# iptables -t nat -L -vn
Chain PREROUTING (policy ACCEPT 23498 packets, 2563K bytes)
 pkts bytes target     prot opt in     out     source               destination         
23498 2563K prerouting_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom prerouting rule chain */
21982 2494K zone_lan_prerouting  all  --  br-lan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
 1516 69325 zone_wan_prerouting  all  --  pppoe-wan *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_wan_prerouting  all  --  eth1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnclient_prerouting  all  --  tun0   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnserver_prerouting  all  --  tun1   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain INPUT (policy ACCEPT 3363 packets, 225K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1316 packets, 89988 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 23 packets, 2236 bytes)
 pkts bytes target     prot opt in     out     source               destination         
16760 1703K postrouting_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom postrouting rule chain */
    3   996 zone_lan_postrouting  all  --  *      br-lan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */
 5714  349K zone_wan_postrouting  all  --  *      pppoe-wan  0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_wan_postrouting  all  --  *      eth1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */
11023 1351K zone_vpnclient_postrouting  all  --  *      tun0    0.0.0.0/0            0.0.0.0/0            /* !fw3 */
    0     0 zone_vpnserver_postrouting  all  --  *      tun1    0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain postrouting_lan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain postrouting_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain postrouting_vpnclient_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain postrouting_vpnserver_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain postrouting_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain prerouting_lan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain prerouting_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain prerouting_vpnclient_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain prerouting_vpnserver_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain prerouting_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain zone_lan_postrouting (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    3   996 postrouting_lan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom lan postrouting rule chain */

Chain zone_lan_prerouting (1 references)
 pkts bytes target     prot opt in     out     source               destination         
21982 2494K prerouting_lan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom lan prerouting rule chain */

Chain zone_vpnclient_postrouting (1 references)
 pkts bytes target     prot opt in     out     source               destination         
11023 1351K postrouting_vpnclient_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnclient postrouting rule chain */
11023 1351K MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_vpnclient_prerouting (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 prerouting_vpnclient_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnclient prerouting rule chain */

Chain zone_vpnserver_postrouting (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 postrouting_vpnserver_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnserver postrouting rule chain */

Chain zone_vpnserver_prerouting (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 prerouting_vpnserver_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom vpnserver prerouting rule chain */
    0     0 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:9 /* !fw3: WOL */ to:192.168.0.254:9

Chain zone_wan_postrouting (2 references)
 pkts bytes target     prot opt in     out     source               destination         
 5714  349K postrouting_wan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom wan postrouting rule chain */
 5714  349K MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Chain zone_wan_prerouting (2 references)
 pkts bytes target     prot opt in     out     source               destination         
 1516 69325 prerouting_wan_rule  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* !fw3: Custom wan prerouting rule chain */

# iptables -t mangle -L -vn
Chain PREROUTING (policy ACCEPT 2370K packets, 1714M bytes)
 pkts bytes target     prot opt in     out     source               destination         
2370K 1714M VPR_PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           [goto]  mark match 0x0/0xff0000

Chain INPUT (policy ACCEPT 382K packets, 57M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 1983K packets, 1656M bytes)
 pkts bytes target     prot opt in     out     source               destination         
 2819  169K TCPMSS     tcp  --  *      pppoe-wan  0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
    0     0 TCPMSS     tcp  --  *      eth1    0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
 3271  196K TCPMSS     tcp  --  *      tun0    0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone vpnclient MTU fixing */ TCPMSS clamp to PMTU

Chain OUTPUT (policy ACCEPT 663K packets, 859M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 2646K packets, 2515M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain VPR_PREROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            212.58.224.0/19      /* iPlayer */ MARK xset 0x20000/0xff0000
    0     0 MARK       all  --  *      *       0.0.0.0/0            178.79.192.0/18      /* iPlayer */ MARK xset 0x20000/0xff0000
 587K  809M MARK       all  --  *      *       192.168.0.30         0.0.0.0/0            /* Deluge */ MARK xset 0x20000/0xff0000
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set vpnserver dst MARK xset 0x30000/0xff0000
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set vpnclient dst MARK xset 0x20000/0xff0000
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set wan dst MARK xset 0x10000/0xff0000

# ip ro list table 201
default via xxx.xxx.xxx.xxx dev pppoe-wan 

# ip ro list table 202
default via 172.21.28.4 dev tun0 

# ip ro list table 203
default via 192.168.10.1 dev tun1

trendy · February 5, 2019, 8:52am

For testing, could you temporarily remove the 3 rules that you have and try again to ping? It doesn't look like there is a mark for .20 to use a different routing table, but we need to rule things out.
Also a tcpdump on the router just to verify the packet headers.

jonathan.holvey · February 27, 2019, 10:50am

I can't explain this, but when I checked today, I was able to access the problem computer through the VPN without issue.

I was investigating the IP tables you flagged as being potential problems, and traced two of them to a VPN policy routing package I'm using. Disabling this removed the tables 102 and 103 from the output of ip -4 ru, and when I checked, access over the VPN was possible. I re-enabled the package and rebooted, and access is still possible. However, I didn't try accessing the problem computer immediately before disabling the package, so I can't be sure if it had an effect or not.

The only thing I haven't done yet that might break things again is reboot the problem computer. I'll try this and see how it goes over a few days and report back. Fingers crossed...