Stunnel/openvpn routing issues

mercuzio · August 3, 2019, 1:48pm

Probably a newbie issue, but still banging my head on the wall ...

Set up stunnel/openvpn with OpenWRT as a client. Went more or less fine with only 1 bump: had to set a static route on the client config file 'route 255.255.255.255 net_gateway'.
So OpenWRT is perfectly able to act as a client on stunnel/openvpn and reach the Internet.

The biggest issue is that I cannot have the computers connected to OpenWRT to go on the Internet.

Local network (lan) is 192.168.0.0/24.
OpenVPN network is pretty standard, 10.8.0.0/24.

root@OpenWrt:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.8.0.5        0.0.0.0         UG    0      0        0 tun0
10.8.0.1        10.8.0.5        255.255.255.255 UGH   0      0        0 tun0
10.8.0.5        *               255.255.255.255 UH    0      0        0 tun0
VPNSERVERIP   192.168.1.1     255.255.255.255 UGH   0      0        0 eth0.2
127.0.0.1       192.168.1.1     255.255.255.255 UGH   0      0        0 eth0.2
192.168.0.0     *               255.255.255.0   U     0      0        0 br-lan
192.168.1.0     *               255.255.255.0   U     0      0        0 eth0.2

root@OpenWrt:~# ip route
default via 10.8.0.5 dev tun0
10.8.0.1 via 10.8.0.5 dev tun0
10.8.0.5 dev tun0 proto kernel scope link src 10.8.0.6
VPNSERVERIP via 192.168.1.1 dev eth0.2
127.0.0.1 via 192.168.1.1 dev eth0.2
192.168.0.0/24 dev br-lan proto kernel scope link src 192.168.0.1
192.168.1.0/24 dev eth0.2 proto kernel scope link src 192.168.1.50
root@OpenWrt:~#

I even turned off the firewall, but still the clients connected on 'lan' OpenWRT cannot access the Internet.

Any help would be appreciated

iplaywithtoys · August 3, 2019, 2:55pm

Your route mask is for a single host, not for all hosts. Try adjusting the route mask.

mercuzio · August 3, 2019, 4:04pm

thanks @iplaywithtoys for answering.
Which route mask should I change? The 'VPNSERVERIP' one?

iplaywithtoys · August 3, 2019, 4:28pm

My mistake. I misread the initial post.

What are the contents of /etc/config/firewall, /etc/config/network, and /etc/config/openvpn ?

mercuzio · August 3, 2019, 5:40pm

Firewall

root@OpenWrt:~# cat /etc/config/firewall

config defaults
option syn_flood '1'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'

config zone
option name 'lan'
list network 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option masq '1'
option mtu_fix '1'

config zone
option name 'wan'
list network 'wan'
list network 'wan6'
option output 'ACCEPT'
option masq '1'
option mtu_fix '1'
option input 'ACCEPT'
option forward 'ACCEPT'

config forwarding
option src 'lan'
option dest 'wan'

config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'

config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'

config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'

config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option src_ip 'fc00::/6'
option dest_ip 'fc00::/6'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'

config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'

config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'

config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'

config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'

config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'

config include
option path '/etc/firewall.user'

config forwarding
option dest 'lan'
option src 'wan'

Network

root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'

config globals 'globals'
option ula_prefix 'fdd7:1bad:3b3a::/48'

config interface 'lan'
option type 'bridge'
option proto 'static'
option ipaddr '192.168.0.1'
option netmask '255.255.255.0'
option ip6assign '60'
option ifname 'eth0.1'

config interface 'wan'
option ifname 'eth0.2'
option proto 'dhcp'

config device 'wan_dev'
option name 'eth0.2'
option macaddr 'd8:0d:17:c0:30:cb'

config interface 'wan6'
option ifname 'eth0.2'
option proto 'dhcpv6'
option auto '0'

config switch
option name 'switch0'
option reset '1'
option enable_vlan '1'

config switch_vlan
option device 'switch0'
option vlan '1'
option ports '2 3 4 5 0t'

config switch_vlan
option device 'switch0'
option vlan '2'
option ports '1 0t'

root@OpenWrt:~#

OpenVPN

oot@OpenWrt:~# cat /etc/config/openvpn
package openvpn

#################################################

Sample to include a custom config file.
#################################################

config openvpn custom_config

    # Set to 1 to enable this instance:
    option enabled 1

    # Include OpenVPN configuration
    option config /etc/openvpn/vpnclient.conf

root@OpenWrt:~# cat /etc/openvpn/vpnclient.conf
client
dev tun
proto tcp
remote 127.0.0.1 1194
resolv-retry infinite
nobind
user nobody
group nogroup
persist-key
persist-tun
remote-cert-tls server
tls-auth ta.key 1
cipher AES-128-CBC
auth SHA256
key-direction 1
verb 3
askpass /etc/openvpn/vpnclient.auth
auth-nocache
route VPNSERVERIP 255.255.255.255 net_gateway

vgaetera · August 3, 2019, 5:47pm

sed -i -r -e "s/^(dev\s).*/\1tun0/" /etc/openvpn/vpnclient.conf
service openvpn restart
uci set firewall.@zone[1].device="tun0"
uci commit firewall
service firewall restart

iplaywithtoys · August 3, 2019, 5:50pm

Quick question: why have you got localhost routed via your WAN gateway? There's no need to send traffic to 127/8 via any remote host.

mercuzio · August 3, 2019, 6:24pm

did not help.
First this notif popped

Sat Aug  3 18:19:23 2019 daemon.warn openvpn(custom_config)[2123]: NOTE: unable to redirect default gateway -- Cannot read current default gateway from system

Then ping did not work anymore. So I re-added manually a route and it worked at least from tun0

# route add default gw 10.8.0.5

root@OpenWrt:~# ping www.google.com
PING www.google.com (172.217.19.164): 56 data bytes
ping: sendto: Network unreachable
root@OpenWrt:~#

root@OpenWrt:~# ping -I tun0 www.google.com
PING www.google.com (172.217.19.164): 56 data bytes
64 bytes from 172.217.19.164: seq=0 ttl=53 time=228.104 ms
64 bytes from 172.217.19.164: seq=1 ttl=53 time=227.825 ms

mercuzio · August 3, 2019, 6:25pm

@iplaywithtoys shouldn't that be a requirement for stunnelling?

mercuzio · August 3, 2019, 6:29pm

@iplaywithtoys , you're right it was not needed. I removed it but still not being able to route.
Here's the updated route table

root@OpenWrt:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.8.0.5        0.0.0.0         UG    0      0        0 tun0
10.8.0.1        10.8.0.5        255.255.255.255 UGH   0      0        0 tun0
10.8.0.5        *               255.255.255.255 UH    0      0        0 tun0
VPNSERVERIP   192.168.1.1     255.255.255.255 UGH   0      0        0 eth0.2
192.168.0.0     *               255.255.255.0   U     0      0        0 br-lan
192.168.1.0     *               255.255.255.0   U     0      0        0 eth0.2
root@OpenWrt:~#

Bernd · August 3, 2019, 6:34pm

@mercuzio which VPN provider do you use?

mercuzio · August 3, 2019, 6:38pm

It's my own vpn server.
But the VPN is successfully established. I can access the internet through the tunnel from openwrt.
It's the clients connected to openwrt that cannot access the internet (they are in the 192.168.0.1/24 network)

vgaetera · August 3, 2019, 6:38pm

service openvpn stop
service network restart
sleep 10; ip route show
service openvpn start
sleep 10; ip route show

What does it say?

vgaetera · August 3, 2019, 6:43pm

Because of that:

mercuzio · August 3, 2019, 6:46pm

But that's necessary for stunnel

iplaywithtoys · August 3, 2019, 6:48pm

vgaetera:

iplaywithtoys:

Quick question: why have you got localhost routed via your WAN gateway?

Because of that:
mercuzio:
# cat /etc/openvpn/vpnclient.conf
remote 127.0.0.1 1194

The "remote" directive only specifies a remote endpoint for the VPN; it says nothing about routing to that endpoint.

vgaetera · August 3, 2019, 6:48pm

iplaywithtoys · August 3, 2019, 6:50pm

The OP uses a VPS from Digital Ocean. Presumably the OP has probably installed OpenVPN on that VPS.

mercuzio · August 3, 2019, 6:52pm

root@OpenWrt:~# sleep 10; ip route show
default via 192.168.1.1 dev eth0.2  src 192.168.1.4
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1
192.168.1.0/24 dev eth0.2 scope link  src 192.168.1.4
root@OpenWrt:~# service openvpn start
root@OpenWrt:~# sleep 10; ip route show
0.0.0.0/1 via 10.8.0.5 dev tun0
default via 192.168.1.1 dev eth0.2  src 192.168.1.4
10.8.0.1 via 10.8.0.5 dev tun0
10.8.0.5 dev tun0 scope link  src 10.8.0.6
VPNSERVERIP via 192.168.1.1 dev eth0.2
127.0.0.1 via 192.168.1.1 dev eth0.2
128.0.0.0/1 via 10.8.0.5 dev tun0
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1
192.168.1.0/24 dev eth0.2 scope link  src 192.168.1.4
root@OpenWrt:~#

I had to fix the routes in the following way to at least have openwrt to connect to the internet
(still the clients connected to openwrt cannot access the internet)

root@OpenWrt:~# ip route show
10.8.0.1 via 10.8.0.5 dev tun0
10.8.0.5 dev tun0 scope link  src 10.8.0.6
VPNSERVERIP via 192.168.1.1 dev eth0.2
127.0.0.1 via 192.168.1.1 dev eth0.2
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1
192.168.1.0/24 dev eth0.2 scope link  src 192.168.1.4
root@OpenWrt:~#

iplaywithtoys · August 3, 2019, 6:55pm

Prove it.

From one of the LAN clients, ping an Internet-based IP address which is known to respond. Some suitable examples include Cloudflare's and Google's DNS servers.

While the ping is running, kick off tcpdump on every network interface, both physical and logical/virtual, between the LAN client and the egress point from your Digital Ocean VPS. This includes running tcpdump on the DO VPS as well. Find out where the failure occurs.

Does the traffic from the LAN client even go down the VPN tunnel in the first place?
Does any return traffic get generated?
Does the DO VPS have a return route from itself back to your LAN clients?
Do the above three questions trigger any other ideas?

Routes are not stateful, not in the same way that stateful firewall rules are. Just because I have a route from A to B, does not automatically mean that an equivalent route exists from B back to A. As a genuine real-world example, only last week I found a customer firewall was being pinged every second from a management station, but did not have a route in its routing table for the management station's IP address. So it was receving regular ICMP ECHO REQUEST packets, and it was trying to send ICMP ECHO REPLY packets in response but didn't know where to send them. There was a route from A to B, but not from B to A.