Wireguard issues with Openwrt [Solved]

I am triying to setup a VPN using Wireguard and I am having some issues with Openwrt.
Some of them have been solved, but for other I need some guidance. I will describe my setup and solved issues in case that this helps sombedy. And I am asking for help to solve the remaining issue.

The following is my Layer 3 network setup:

  • The main router is a VPS server (Amazon AWS running Ubuntu). It is configured as a wireguard peer. It performs inter-LAN routing (interconnects Home, Office and road warriors and provides default route to internet).
  • The home router is Openwrt 18.06.2 and requires selective routing. It is configured as wireguard peer and uses vpn-selective-routing to selectively route the traffic based on source IP on the home LAN. Most of the traffic is routed via the VPN, except for the smart TVs, that are routed via WAN interface (or Netflix will be blocked).
  • The office router is Openwrt 18.06.2 and routes all private traffic via VPN interface (traffic to home LAN), but all toher traffic is directly connected to the Internet also using vpn-selective-routing. It is configured as wireguard peer.
  • Road warriors route all the traffic via the wireguard interface to provide security when browsing internet in public spaces (cafes, hotels, etc) and to allow remote access to Home and Office Lans.

From a wireguard perspective, Client/Server roles are irrelevant, but I diferentiate both roles from a configuration perspective. In this configuration with road warriors, the Clients are the peers that will initiate the connection (handshake) with the Server peer and therefore the server needs to have an open ports to listen to incoming wireguard traffic. For enhacend security clients do not need to have any opened port as long as they initiate the handshake with the server.

The following is the wireguard config file on the Ubuntu server (/etc/wireguard/wg0.conf)

[Interface]
Address = 10.10.10.1/24
SaveConfig = true
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
ListenPort = 5555
PrivateKey = <my server private key>

[Peer]
PublicKey = <openwrt home router public key>
AllowedIPs = 10.10.10.2/32, 192.168.1.0/24

[Peer]
PublicKey = <openwrt office router public key>
AllowedIPs = 10.10.10.3/32, 192.168.2.0/24

[Peer]
PublicKey = <road warrior 1 public key>>
AllowedIPs = 10.10.10.4/32

Road warriors (android phones) connected straight forward and were quickly browsing the Internet via VPN. However Openwrt clients took longer to setup.

The first issue that I have noticed is Openwrt creates automatic routes, but they are incorrect routes. When a wireguard interface is added the following routes were automatically established in my home (and offce) router:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         <home public IP>    0.0.0.0         UG    0      0        0 eth1
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 wgHome
<Server public IP>   192.168.1.1     255.255.255.255 UGH   0      0        0 br-lan
<home public IP>    0.0.0.0         255.255.254.0   U     0      0        0 eth1
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan

This automatic routing does not not work, because the "Server public IP" (or wireguard Endpoint) is routed via br-lan and packets to the Endpoint will never reach the public Internet. I have solved this routing issue deleting this route. for this I create a new file /etc/hotplug.d/iface/99-ifup-wg with the following contents

#!/bin/sh
[ "$ACTION" = "ifup" -a "$INTERFACE" = "wgHome" ] && {
    logger "iface wgHome up detected. Creating routes and initiating handshake with Server."
    /sbin/route del server_public_ip.duckdns.org
    /sbin/route add -net 192.168.2.0/24 gw 10.10.10.1
    /usr/sbin/iptables -A FORWARD -i wgHome -j ACCEPT;
    /bin/ping -c1 10.10.10.1
}

[ "$ACTION" = "ifdown" -a "$INTERFACE" = "wgHome" ] && {
    logger "iface  wgHome down detected. Deleting routes and forwarding rules."
    /sbin/route delete -net 129.168.2.0/24 gw 10.10.10.1
    /usr/sbin/iptables -D FORWARD -i wgHome -j ACCEPT
}
exit 0

Basically it deletes the automatic route (via br-lan) and adds a route to the home office LAN via the wireguard interface. The gateway to the Inter-LAN is the Server IP address onf the wireguard interface. It also enables forwarding rules on the firewall and inites the handshake with the server using a small ping packet.
This ping turns out to be very important, because wireguard is a silent protocol and will not initiate a handshake unless one of the peers initiates the connection. After I rebooted my router I lost connectivity to the home router, because the server could not reach the client peer and all ports on the client are closed unless a packet is sent to the server. The ping solves this use case. In my design the Client is initiating the connection and the ping ensures that as soon as the interface is up, a handshake is forced and both peers communicate and the client endpoint is updated on the server. I also enabled the keepalive to monitor the interface periodically.

I am using vpn-policy-routing to enable the selective routing and it is working fine to connect to the internet (https://github.com/stangri/openwrt_packages/tree/master/vpn-policy-routing/files). I am browsing the internet from all home computers using the wgHome interface (VPN) and those hosts selected to bypass the VPN are routed accordingly.

Other fixed issues (for reference in case that anybody is facing them):

  • VPN speed was underperforming. I was getting 2Mbps over the VPN link with a 20Mbps DSL link. The major issue was bufferbloat on the downlink. I installed SQM Qos with standard settings and the speed increased inmediately to 12Mbps over the VPN link and is stable with no bufferbloat.

  • MTU negotiation was failing: after initial wireguard connection, ping was OK, but could not browse internet. The issue was the MTU settings. AWS server started the wireguard interface with an mTU of 9000 and the openwrt client started the interface with an MTU of 1420. I fixed the issue forcing the MTU to 1420 on all peers.

Now the unsolved issue: I can ping from the Server to the Home router using the LAN IP

root@ubuntu-1804-image:~# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=220 ms

But I can NOT ping to any other host on the LAN!. I do not know if I need to enable routing or firewall rules on the Openwrt client to forward incoming traffic via the wireguard interface to other hosts on the LAN. This is preventing me to access my Home and office networks from my road warriors. any help/hint here would be much appreciated.

Thanks.

Port forward should give you more control, unless you want to allow everybody and everything.

In the diagram Home and Office seem to have the same address space, I suppose it is a typo.
What is the output of the following commands in the Openwrt at home?
cat /etc/config/network; cat /etc/config/firewall; ip -4 addr ; ip -4 ro ; ip -4 ru

1 Like

Yes, it is a typo. already fixed.

cat /etc/config/network

	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd0d:297d:be4e::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth0'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option ip6hint '10'
	option ipaddr '192.168.1.1'
	option gateway '192.168.1.1'
	option broadcast '192.168.1.255'
	option stp '1'
	option ip6ifaceid '::1'

config interface 'wan'
	option ifname 'eth1'
	option proto 'dhcp'
	option peerdns '0'
	option dns '1.1.1.1'

config interface 'wan6'
	option ifname 'eth1'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'
	option peerdns '0'
	option dns '2606:4700:4700::1111,2606:4700:4700::1001'

config interface 'wgHome'
	option proto 'wireguard'
	option private_key 'xxxxxxxxxxxxxxxx'
	option delegate '0'
	option listen_port '51290'
	list addresses '10.10.10.2/24'
	option mtu '1420'

config wireguard_wgHome
	option description 'VPN'
	option public_key 'xxxxxxxxxxxxxxxxx'
	list allowed_ips '10.10.10.1/32'
	list allowed_ips '192.168.2.0/24'
	option route_allowed_ips '0'
	option persistent_keepalive '20'

cat /etc/config/firewall

.........

config zone
	option name 'wg'
	option output 'ACCEPT'
	option masq '1'
	option input 'REJECT'
	option forward 'REJECT'
	option mtu_fix '1'
	option network 'wgHome'

config forwarding
	option dest 'wg'
	option src 'lan'

config rule
	option enabled '1'
	option target 'ACCEPT'
	option src 'wg'
	option proto 'tcp'
	option dest_port '22'
	option name 'Allow-SSH-WG'

config rule
	option proto 'icmp'
	option icmp_type 'echo-request'
	option target 'ACCEPT'
	option src 'wg'
	option name 'Allow-Ping-WG'

ip -4 addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc cake state UP group default qlen 1000
    inet xx.xx.xx.xx/23 brd xx.xx.xx.xx scope global eth1
       valid_lft forever preferred_lft forever
66: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 192.168.1.1/24 brd 192.168.1.255 scope global br-lan
       valid_lft forever preferred_lft forever
68: wgHome: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.10.10.2/24 brd 10.10.10.255 scope global wgHome
       valid_lft forever preferred_lft forever

ip -4 ro

default via xx.xx.xx.xx dev eth1 proto static src xx.xx.xx.xx
10.10.10.0/24 dev wgHome proto kernel scope link src 10.10.10.2 
yy.yy.yy.yy/23 dev eth1 proto kernel scope link src yy.yy.yy.yy 
129.168.2.0/24 via 10.10.10.1 dev wgHome
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1 

ip -4 ru

0:	from all lookup local  
32686:	from all fwmark 0x20000 lookup 301 
32687:	from all fwmark 0x10000 lookup 300 
32766:	from all lookup main 
32767:	from all lookup default

Well thank you very much trendy.
Your commands helped me realize that I was missing a firewall rule:

config forwarding
        option dest 'lan'
        option src 'wg'

This allows forwarding from the wireguard interface to other hosts. This simple rule did the trick and everything working as intended.

thanks for the help.

2 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.