I am triying to setup a VPN using Wireguard and I am having some issues with Openwrt.
Some of them have been solved, but for other I need some guidance. I will describe my setup and solved issues in case that this helps sombedy. And I am asking for help to solve the remaining issue.
The following is my Layer 3 network setup:
- The main router is a VPS server (Amazon AWS running Ubuntu). It is configured as a wireguard peer. It performs inter-LAN routing (interconnects Home, Office and road warriors and provides default route to internet).
- The home router is Openwrt 18.06.2 and requires selective routing. It is configured as wireguard peer and uses vpn-selective-routing to selectively route the traffic based on source IP on the home LAN. Most of the traffic is routed via the VPN, except for the smart TVs, that are routed via WAN interface (or Netflix will be blocked).
- The office router is Openwrt 18.06.2 and routes all private traffic via VPN interface (traffic to home LAN), but all toher traffic is directly connected to the Internet also using vpn-selective-routing. It is configured as wireguard peer.
- Road warriors route all the traffic via the wireguard interface to provide security when browsing internet in public spaces (cafes, hotels, etc) and to allow remote access to Home and Office Lans.
From a wireguard perspective, Client/Server roles are irrelevant, but I diferentiate both roles from a configuration perspective. In this configuration with road warriors, the Clients are the peers that will initiate the connection (handshake) with the Server peer and therefore the server needs to have an open ports to listen to incoming wireguard traffic. For enhacend security clients do not need to have any opened port as long as they initiate the handshake with the server.
The following is the wireguard config file on the Ubuntu server (/etc/wireguard/wg0.conf)
[Interface]
Address = 10.10.10.1/24
SaveConfig = true
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
ListenPort = 5555
PrivateKey = <my server private key>
[Peer]
PublicKey = <openwrt home router public key>
AllowedIPs = 10.10.10.2/32, 192.168.1.0/24
[Peer]
PublicKey = <openwrt office router public key>
AllowedIPs = 10.10.10.3/32, 192.168.2.0/24
[Peer]
PublicKey = <road warrior 1 public key>>
AllowedIPs = 10.10.10.4/32
Road warriors (android phones) connected straight forward and were quickly browsing the Internet via VPN. However Openwrt clients took longer to setup.
The first issue that I have noticed is Openwrt creates automatic routes, but they are incorrect routes. When a wireguard interface is added the following routes were automatically established in my home (and offce) router:
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 <home public IP> 0.0.0.0 UG 0 0 0 eth1
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 wgHome
<Server public IP> 192.168.1.1 255.255.255.255 UGH 0 0 0 br-lan
<home public IP> 0.0.0.0 255.255.254.0 U 0 0 0 eth1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
This automatic routing does not not work, because the "Server public IP" (or wireguard Endpoint) is routed via br-lan and packets to the Endpoint will never reach the public Internet. I have solved this routing issue deleting this route. for this I create a new file /etc/hotplug.d/iface/99-ifup-wg with the following contents
#!/bin/sh
[ "$ACTION" = "ifup" -a "$INTERFACE" = "wgHome" ] && {
logger "iface wgHome up detected. Creating routes and initiating handshake with Server."
/sbin/route del server_public_ip.duckdns.org
/sbin/route add -net 192.168.2.0/24 gw 10.10.10.1
/usr/sbin/iptables -A FORWARD -i wgHome -j ACCEPT;
/bin/ping -c1 10.10.10.1
}
[ "$ACTION" = "ifdown" -a "$INTERFACE" = "wgHome" ] && {
logger "iface wgHome down detected. Deleting routes and forwarding rules."
/sbin/route delete -net 129.168.2.0/24 gw 10.10.10.1
/usr/sbin/iptables -D FORWARD -i wgHome -j ACCEPT
}
exit 0
Basically it deletes the automatic route (via br-lan) and adds a route to the home office LAN via the wireguard interface. The gateway to the Inter-LAN is the Server IP address onf the wireguard interface. It also enables forwarding rules on the firewall and inites the handshake with the server using a small ping packet.
This ping turns out to be very important, because wireguard is a silent protocol and will not initiate a handshake unless one of the peers initiates the connection. After I rebooted my router I lost connectivity to the home router, because the server could not reach the client peer and all ports on the client are closed unless a packet is sent to the server. The ping solves this use case. In my design the Client is initiating the connection and the ping ensures that as soon as the interface is up, a handshake is forced and both peers communicate and the client endpoint is updated on the server. I also enabled the keepalive to monitor the interface periodically.
I am using vpn-policy-routing to enable the selective routing and it is working fine to connect to the internet (https://github.com/stangri/openwrt_packages/tree/master/vpn-policy-routing/files). I am browsing the internet from all home computers using the wgHome interface (VPN) and those hosts selected to bypass the VPN are routed accordingly.
Other fixed issues (for reference in case that anybody is facing them):
-
VPN speed was underperforming. I was getting 2Mbps over the VPN link with a 20Mbps DSL link. The major issue was bufferbloat on the downlink. I installed SQM Qos with standard settings and the speed increased inmediately to 12Mbps over the VPN link and is stable with no bufferbloat.
-
MTU negotiation was failing: after initial wireguard connection, ping was OK, but could not browse internet. The issue was the MTU settings. AWS server started the wireguard interface with an mTU of 9000 and the openwrt client started the interface with an MTU of 1420. I fixed the issue forcing the MTU to 1420 on all peers.
Now the unsolved issue: I can ping from the Server to the Home router using the LAN IP
root@ubuntu-1804-image:~# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=220 ms
But I can NOT ping to any other host on the LAN!. I do not know if I need to enable routing or firewall rules on the Openwrt client to forward incoming traffic via the wireguard interface to other hosts on the LAN. This is preventing me to access my Home and office networks from my road warriors. any help/hint here would be much appreciated.
Thanks.