Wireguard traffic routing

toolcap · April 30, 2021, 11:57am

I am sorry I am asking this question which I guess has an easy answer but after a lot of unsuccessful attempts and head banging I decided it is best to seek help from professionals.
I am trying to setup a WireGuard server on my home LAN network.
The purpose is to allow me to access my LAN network and also allow me to surf the internet through my ISP router at home.
The network topology looks like in the picture.

In this example the mobile phone acts as the WireGuard client.

WireGuard server
The WG server hardware is a not used anymore DSL modem having one telephone line and a single Ethernet port. The telephone line port is not connected. It runs OpenWrt 19.07
The WireGuard server sits on my home LAN network and listens on port 51820. The ISP router has a port forwarding rule for the port 51820 and it sends the UDP traffic towards the WireGuard server.
I managed to setup the WG server on it and I can successfully connect to it from the mobile phone over internet.

Now the issue
When the mobile phone is connected to the WG server I can ping successfully the WG server address (10.14.0.1), however other places (Raspberry Pi or the workstation on home LAN or Internet through the home ISP router) aren't reachable.
I tried unsuccesfully to play with firewall rules but I wasn't able to access my LAN or Internet (through the home ISP router).
Any help/hint is highly appreciated.
Thank you

krazeh · April 30, 2021, 12:29pm

Have you added a static route for 10.14.0.0/24 on the ISP router? It needs to send traffic to the wireguard server.

toolcap · April 30, 2021, 12:50pm

I just tried it but it seems it didn't work.
This is how the routing table on the ISP router looks like

When I look at the wg0 interface on the WG server, I can see no packets being sent back to the wg client

wg0 interface is now part of the LAN zone of the WG server. Do I need to put wg0 in a different zone?

krazeh · April 30, 2021, 12:54pm

You need a rule which tells the router that traffic destined for 10.14.0.0/24 needs to be sent to the WG server (192.168.10.130). At the moment it looks like it's just sending it out to the ISP.

eduperez · April 30, 2021, 1:03pm

Once you can stablish the WG connection, the ISP router plays no role, as traffic just flows through the WG tunnel.

You first need to configure the client to send traffic for the 192.168.10.x network using the WG server as a gateway, or all traffic if that is not possible.

Then, you need to configure the WG server to allow traffic from the WG interface to the LAN interface.

Finally, your devices in the LAN network must be configured to accept traffic from a 10.14.0.x address, or use NAT on the router.

toolcap · April 30, 2021, 1:15pm

The ISP router is pretty dumb and it only allows me to add static rules for the ISP connection(PPPoE)

krazeh · April 30, 2021, 1:17pm

Ok, put the wg0 interface into it's own firewall zone and enable masquerading.

toolcap · April 30, 2021, 1:22pm

I ran tcpdump on the wg0 interface of the WG server when the WG client was connected and I could see packets coming from the client being received but no packets going back towards the client.

Is the traffic from the wg0 sent to the eth0 interfaces if they're both part of the LAN zone? If not, what else I need to do on the WG server (running Openwrt)?

Would it be possible to run the NAT on the WG server hardware instead of the dumb ISP router? Like putting the wg0 interface in its own zone and then NAT between the LAN and the WG zones.

toolcap · April 30, 2021, 1:59pm

Ok. I placed the wg0 in its own zone and I enabled masquerading between the "lan" and "wg_zone". I tried to ping the WG server from the client and the client from the WG server but I got no reply back

krazeh · April 30, 2021, 2:06pm

You only need masquerading on VPN > LAN.

If it still isn't working after changing that then we'll need to see the WG configs for both the server and client (redact any keys or public IPs).

toolcap · April 30, 2021, 2:27pm

I've changed the masquerading only for VPN > LAN

The WG server configuration is:

network.loopback=interface
network.loopback.ifname='lo'
network.loopback.proto='static'
network.loopback.ipaddr='127.0.0.1'
network.loopback.netmask='255.0.0.0'
network.globals=globals
network.globals.ula_prefix='fd14:2010:ab01::/48'
network.atm=atm-bridge
network.atm.vpi='1'
network.atm.vci='32'
network.atm.encaps='llc'
network.atm.payload='bridged'
network.atm.nameprefix='dsl'
network.atm.unit='root'
network.dsl=dsl
network.dsl.annex='b'
network.dsl.firmware='/lib/firmware/adsl.bin'
network.lan=interface
network.lan.type='bridge'
network.lan.proto='dhcp'
network.lan.ifname='eth0'
network.wan=interface
network.wan.ifname='dsl0'
network.wan.proto='pppoe'
network.wan.username='username'
network.wan.password='password'
network.wan.ipv6='1'
network.wan_dsl0_dev=device
network.wan_dsl0_dev.name='dsl0'
network.wan_dsl0_dev.macaddr='xx:xx:xx:xx:xx:xx'
network.wan6=interface
network.wan6.ifname='@wan'
network.wan6.proto='dhcpv6'
network.wg0=interface
network.wg0.proto='wireguard'
network.wg0.private_key='...'
network.wg0.listen_port='51820'
network.wg0.addresses='10.14.0.1/32'
network.@wireguard_wg0[0]=wireguard_wg0
network.@wireguard_wg0[0].public_key='...'
network.@wireguard_wg0[0].description='Phone'
network.@wireguard_wg0[0].allowed_ips='0.0.0.0/0'
network.@wireguard_wg0[0].route_allowed_ips='1'
network.@wireguard_wg0[0].persistent_keepalive='25'
firewall.@defaults[0]=defaults
firewall.@defaults[0].syn_flood='1'
firewall.@defaults[0].input='ACCEPT'
firewall.@defaults[0].output='ACCEPT'
firewall.@defaults[0].forward='ACCEPT'
firewall.@zone[0]=zone
firewall.@zone[0].name='lan'
firewall.@zone[0].input='ACCEPT'
firewall.@zone[0].output='ACCEPT'
firewall.@zone[0].device='br-lan' 'eth0' 'radio0.network1' 'radio0.network2'
firewall.@zone[0].forward='ACCEPT'
firewall.@zone[0].network='lan'
firewall.@include[0]=include
firewall.@include[0].path='/etc/firewall.user'
firewall.@zone[1]=zone
firewall.@zone[1].name='wg_zone'
firewall.@zone[1].input='ACCEPT'
firewall.@zone[1].forward='ACCEPT'
firewall.@zone[1].output='ACCEPT'
firewall.@zone[1].masq='1'
firewall.@zone[1].network='wg_zone wg0 lan'
firewall.@zone[1].device='wg0'
firewall.@forwarding[0]=forwarding
firewall.@forwarding[0].dest='lan'
firewall.@forwarding[0].src='wg_zone'
firewall.@forwarding[1]=forwarding
firewall.@forwarding[1].dest='wg_zone'
firewall.@forwarding[1].src='lan'
dhcp.@dnsmasq[0]=dnsmasq
dhcp.@dnsmasq[0].domainneeded='1'
dhcp.@dnsmasq[0].boguspriv='1'
dhcp.@dnsmasq[0].filterwin2k='0'
dhcp.@dnsmasq[0].localise_queries='1'
dhcp.@dnsmasq[0].rebind_protection='1'
dhcp.@dnsmasq[0].rebind_localhost='1'
dhcp.@dnsmasq[0].local='/lan/'
dhcp.@dnsmasq[0].domain='lan'
dhcp.@dnsmasq[0].expandhosts='1'
dhcp.@dnsmasq[0].nonegcache='0'
dhcp.@dnsmasq[0].authoritative='1'
dhcp.@dnsmasq[0].readethers='1'
dhcp.@dnsmasq[0].leasefile='/tmp/dhcp.leases'
dhcp.@dnsmasq[0].resolvfile='/tmp/resolv.conf.auto'
dhcp.@dnsmasq[0].nonwildcard='1'
dhcp.@dnsmasq[0].localservice='1'
dhcp.lan=dhcp
dhcp.lan.interface='lan'
dhcp.lan.start='100'
dhcp.lan.limit='150'
dhcp.lan.leasetime='12h'
dhcp.lan.dhcpv6='server'
dhcp.lan.ra='server'
dhcp.lan.ra_management='1'
dhcp.wan=dhcp
dhcp.wan.interface='wan'
dhcp.wan.ignore='1'
dhcp.odhcpd=odhcpd
dhcp.odhcpd.maindhcp='0'
dhcp.odhcpd.leasefile='/tmp/hosts/odhcpd'
dhcp.odhcpd.leasetrigger='/usr/sbin/odhcpd-update'
dhcp.odhcpd.loglevel='4'
==> /etc/resolv.conf <==
search lan
nameserver 127.0.0.1

==> /tmp/resolv.conf <==
search lan
nameserver 127.0.0.1

==> /tmp/resolv.conf.auto <==
# Interface lan
nameserver 192.168.10.1
head: /tmp/resolv.*/*: No such file or directory
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
4: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.130/24 brd 192.168.10.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fe80::3631:c4ff:fe8b:1b3b/64 scope link
       valid_lft forever preferred_lft forever
5: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 10.14.0.1/32 brd 255.255.255.255 scope global wg0
       valid_lft forever preferred_lft forever
default via 192.168.10.1 dev br-lan proto static src 192.168.10.130
192.168.10.0/24 dev br-lan proto kernel scope link src 192.168.10.130
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

The WG client configuration is something like this:

Client side
Client address: 10.14.0.10/32
DNS servers: 192.168.10.1
==============================
Peer side
Allowed IPs: 0.0.0.0/0, ::/128
Endpoint: wg-server: 51820
Keepalive: 25sec

When I run "tcpdump -ni wg0" on WG server I can see a lot of packets being sent from the WG client side(phone) but very few packets being sent back by the WG server towards the WG client

krazeh · April 30, 2021, 2:51pm

The allowed IPs should be 10.14.0.10/32

Just double check the zone you've created for the VPN. Make sure it's only got the VPN in it.

Restart interfaces (or your router) once you've made changes.

toolcap · April 30, 2021, 3:17pm

I changed the settings as suggested and rebooted the WG server just to be sure.
Now, it seems we're getting close
I connected the WG client and then I tried to ping the WG server (from the phone's shell) and it works. Below are some tcpdump logs from wg0 interface of the WG server

16:08:31.624383 IP 10.14.0.10 > 10.14.0.1: ICMP echo request, id 51, seq 25, length 64
16:08:31.624537 IP 10.14.0.1 > 10.14.0.10: ICMP echo reply, id 51, seq 25, length 64
16:08:32.648435 IP 10.14.0.10 > 10.14.0.1: ICMP echo request, id 51, seq 26, length 64
16:08:32.648591 IP 10.14.0.1 > 10.14.0.10: ICMP echo reply, id 51, seq 26, length 64

Pinging the WG server LAN address (192.168.10.130) works too

16:11:58.792291 IP 10.14.0.10 > 192.168.10.130: ICMP echo request, id 52, seq 15, length 64
16:11:58.792447 IP 192.168.10.130 > 10.14.0.10: ICMP echo reply, id 52, seq 15, length 64
16:11:59.812270 IP 10.14.0.10 > 192.168.10.130: ICMP echo request, id 52, seq 16, length 64
16:11:59.812425 IP 192.168.10.130 > 10.14.0.10: ICMP echo reply, id 52, seq 16, length 64

Pinging the other LAN (192.168.10.x) devices doesn't work but I guess this is the part where some other settings are required.

toolcap · May 1, 2021, 4:47am

Is there a simple way to make the phone connected to the WG server able to access the other LAN devices or the internet through the ISP router?
Since the ISP router is pretty dumb, any solution would have to involve settings on the WG server running Openwrt

eduperez · May 1, 2021, 7:27am

Wait! Is the WG connection working? This is the first step, you must ensure that traffic flows through the tunnel, before trying anything else.

eduperez · May 1, 2021, 7:29am

If the WG tunnel works, forget about the ISP router, forever.

eduperez · May 1, 2021, 7:30am

Please, post your current firewall config file.

toolcap · May 1, 2021, 7:44am

Yes, the WG connection works now. From the phone I can ping the WG server peer address (10.14.0.1) as well as the WG server's LAN address (192.168.10.130)
Pinging any other LAN devices from the phone still doesn't work

This is the current firewall configuration

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        list device 'br-lan'
        list device 'eth0'
        list device 'radio0.network1'
        list device 'radio0.network2'
        option forward 'ACCEPT'
        option network 'lan'

config include
        option path '/etc/firewall.user'

config zone
        option name 'wg_zone'
        option input 'ACCEPT'
        option forward 'ACCEPT'
        option output 'ACCEPT'
        option masq '1'
        option network 'wg_zone wg0'
        list device 'wg0'

config forwarding
        option dest 'lan'
        option src 'wg_zone'

config forwarding
        option dest 'wg_zone'
        option src 'lan'

toolcap · May 1, 2021, 7:54am

Going deeper into the current issue.

I am establishing the WG connection between the phone and the WG server. WG connection is established
From the phone's shell I am pinging some device on the LAN network, say the ISP router (192.168.10.1)
There's no reply back, however when I do a tcpdump on the WG server's br-lan interface I can see the ARP requests for WG client

# tcpdump dst 10.14.0.10 -ni br-lan
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-lan, link-type EN10MB (Ethernet), capture size 262144 bytes
08:52:20.583377 ARP, Request who-has 10.14.0.10 tell 192.168.10.1, length 46
08:52:21.583334 ARP, Request who-has 10.14.0.10 tell 192.168.10.1, length 46
08:52:22.583366 ARP, Request who-has 10.14.0.10 tell 192.168.10.1, length 46

So, it looks like the ISP router doesn't know where to send the replies.
On the WG server the routing table looks like this:

# ip route show
default via 192.168.10.1 dev br-lan proto static src 192.168.10.130
10.14.0.10 dev wg0 proto static scope link
192.168.10.0/24 dev br-lan proto kernel scope link src 192.168.10.130

pavelgl · May 1, 2021, 3:59pm

Enable masquerading on the lan zone.

uci set firewall.@zone[0].masq='1'
uci commit firewall
fw3 restart