Ping over Wireguard interface fails after wan interface changed

Installing and Using OpenWrt
1

Hello everyone,

I have two remote routers connected to a central router through wireguard which has been working flawlessly for the past few years. Last week, one of the remote routers moved from an internet connection that was using pppoe to a provider that uses DHCP - the wan interface was updated accordingly.

All seemed well until I noticed that I had stopped receiving syslog messages from the router with the updated wan configuration. It seems that something that I can't see in the configuration has broken the wireguard route to the central router (where the syslog server lives).

Pings to the central router fail, unless the interface is specified;

root@router1:~# ping 192.168.11.11
PING 192.168.11.11 (192.168.11.11): 56 data bytes
^C
--- 192.168.11.11 ping statistics ---
7 packets transmitted, 0 packets received, 100% packet loss

root@router1:~# ping -I 192.168.21.1 192.168.11.11
PING 192.168.11.11 (192.168.11.11) from 192.168.21.1: 56 data bytes
64 bytes from 192.168.11.11: seq=0 ttl=63 time=5.066 ms
64 bytes from 192.168.11.11: seq=1 ttl=63 time=4.829 ms
64 bytes from 192.168.11.11: seq=2 ttl=63 time=4.734 ms
64 bytes from 192.168.11.11: seq=3 ttl=63 time=5.237 ms
64 bytes from 192.168.11.11: seq=4 ttl=63 time=5.879 ms
64 bytes from 192.168.11.11: seq=5 ttl=63 time=5.239 ms
^C
--- 192.168.11.11 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 4.734/5.164/5.879 ms

Versus;

root@router2:~# ping 192.168.11.11
PING 192.168.11.11 (192.168.11.11): 56 data bytes
64 bytes from 192.168.11.11: seq=0 ttl=63 time=3.001 ms
64 bytes from 192.168.11.11: seq=1 ttl=63 time=2.782 ms
64 bytes from 192.168.11.11: seq=2 ttl=63 time=3.210 ms
64 bytes from 192.168.11.11: seq=3 ttl=63 time=3.203 ms
64 bytes from 192.168.11.11: seq=4 ttl=63 time=3.012 ms
64 bytes from 192.168.11.11: seq=5 ttl=63 time=2.941 ms
^C
--- 192.168.11.11 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 2.782/3.024/3.210 ms

Here is the routing table from the first router (changed from pppoe to DHCP wan);

root@router1:~# ip route show
default via <ISP gateway> dev wan proto static src <ISP network>.255 
<vpn endpoint> via <public IP> dev wan proto static 
<public IP>/21 dev wan proto kernel scope link src <ISP network>.255 
192.168.11.0/24 dev wg0 proto static scope link 
192.168.21.0/24 dev br-lan proto kernel scope link src 192.168.21.1 
192.168.99.0/24 dev wg0 proto static scope link

And the routing table from the second router;

root@router2:~# ip route show
default via 100.127.224.1 dev pppoe-wan proto static 
100.127.224.1 dev pppoe-wan proto kernel scope link src 100.127.238.247 
<vpn endpoint> via 100.127.224.1 dev pppoe-wan proto static 
192.168.11.0/24 dev wg0 proto static scope link 
192.168.12.0/24 dev br-lan proto kernel scope link src 192.168.12.1 
192.168.99.0/24 dev wg0 proto static scope link

/etc/config/network from the first router;

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd8e:89f2:ebf7::/48'
	option packet_steering '2'
	option dhcp_default_duid '00042d9ca702d1e44b439c750f2ec9911d0a'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.12.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'wan'
	option proto 'pppoe'
	option username <username>
	option password '<password>
	option ipv6 'auto'
	option peerdns '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'

config interface 'wg0'
	option proto 'wireguard'
	option private_key <privatekey>
	option multipath 'off'

config wireguard_wg0
	option description 'Central'
	option public_key <publickey>
	option preshared_key '<psk>
	option route_allowed_ips '1'
	option endpoint_host <vpn endpoint>
	option endpoint_port '51815'
	option persistent_keepalive '25'
	list allowed_ips '192.168.11.0/24'
	list allowed_ips '192.168.99.0/24'

/etc/config/network from the second router;

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd03:3416:2d10::/48'
	option packet_steering '2'
	option dhcp_default_duid '0004c444135a8b30437aa65600712118cb25'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.21.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config device
	option name 'wan'
	option macaddr <macaddress>

config interface 'wan'
	option proto 'dhcp'
	option device 'wan'
	option peerdns '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'
	option multipath 'off'

config interface 'wg0'
	option proto 'wireguard'
	option private_key <privatekey>
	option multipath 'off'

config wireguard_wg0
	option description 'Central'
	option public_key <publickey>
	option preshared_key <psk>
	list allowed_ips '192.168.11.0/24'
	list allowed_ips '192.168.99.0/24'
	option endpoint_host <vpn endpoint>
	option endpoint_port '51815'
	option persistent_keepalive '25'
	option route_allowed_ips '1'

Any guidance provided will be gratefully received !

2

I noticed that your WireGuard interfaces do not have an address attached.
Theoretically that is not necessary but I have seen it to lead to strange things, this could be one of those strange things

Consider to do a setup with WireGuard interfaces on their own subnet.

See:
WireGuard Server Setup Guide
WireGuard Client Setup Guide

3

Weird results;

Adding interface addresses from a separate network to the Wireguard interface on the central router and its peers didn't solve the issue. In fact, it broke the routing from the other peers which were working without interface addresses.

That led me to believe that the issue was with the single router, so I reset the configuration and re-created the Wireguard interface, using the same keys and network address space. Same issue.

I think the things that are bothering me are;

  1. Specifying the source address (of the lan address) in a ping works without issue. Any ideas why that would be the case ?
  2. This issue only started when the wan interface was changed from PPPoE to DHCP. This changed the net mask of the interface, but I can't figure why that could affect the routing table (as it appears correct).

Thanks again for the suggestion !

4

The WireGuard interfaces on both routers should have a (different) address in the same subnet this subnet has to be different from all other subnets involved.
On the peers you have to add this subnet as Allowed otherwise it will not work indeed

See: WireGuard Server Setup Guide
There is a paragraph about site-to-site setup

No guarantee that this will solve your problem but at least that is the proper way to setup

5

Yes - the interfaces were from 192.168.101.xx/24 with the network allowed on both sides.

Maybe the next step is to recreate the Wireguard interface on the central router from scratch and see if that makes any difference ?!

6

I would try that but be sure to uses addresses on the interface
If 192.168.101.0/24 is not used anywhere in your Network you can use that for the address of the Wireguard interface
One router should have 192.168.101.1/24 as WireGuard address and add Allowed IPs on peer 192.168.101.0/24
The other router 192.168.101.2/24 again add Allowed IPs on the peer 192.168.101.0/24

Furthermore you should also set a listen port on the WireGuard interface this is also missing!

7

Yes, that's exactly what I did..

Even if the router is behind CGNAT ?

I'll add in a listen port, but my understanding of CGNAT was that the central router wouldn't be able to initiate the tunnel.

8

One of the sides should be reachable with a public IP address that one needs to have a Listen port defined, that side is working as a "server" e.g. listening for connections on its defined port for which a firewall rule exists to open up that port. In your case this server side should not have an endpoint and port defined as it can try to make contact but that will not work as the other site is behind CGNAT.

The CGNAT side is the "client" which will start the connection that side does not need to have a listen port although it can not hurt (when it connects it sends its listen port to the server so that the server knows which port to use). As this side starts the connection it of course needs an endpoint and port to start the connection to the server side

9

Perhaps I didn't explain the initial setup very well - both of these routers were originally behind a CGNAT and authenticated to the ISPs using PPPoE.

Both of these routers have tunnels to a central router which has the 192.168.11.0/24 network and some resources which a syslog server.

These two routers have had Wireguard tunnels configured for the past 2 years (without interface addresses) and have been kept on current versions of OpenWrt over that time.

Recently, one of the routers was physically relocated to a new location. The new ISP provides an IP address from the fibre gateway via DHCP. No authentication is required, so the wan interface was changed from PPPoE to DHCP and the router is no longer behind a CGNAT.

No other changes were made - the Wireguard tunnel still establishes a connection to the central router and routes traffic from the lan interface (192.168.21.0/24) to the lan interface on the central router (192.168.11.0/24).

The only clue I had that something was up was that the router with the updated wan interface stopped sending messages to the syslog server on the 192.168.11.0 network.

On investigation, I found that the router with the updated wan interface could no longer ping the 192.168.11.0 network from the console, unless the ping specifies a source interface using the -I flag.

There is no impact on traffic originating on the routers lan interface - a device connected to the 192.168.21.0 network can access the 192.168.11.0 network and resources.

It is only applications on the router (specifically pings and syslog traffic) which is not being routed over the Wireguard interface to 192.168.11.0

I hope that makes the issue clearer !

10

The wan interface type (PPPoE or DHCP) shouldn't have anything to do with this issue, at least not directly.

I do think it is worth adding addresses to the WG interfaces on all of the routers, though. I suspect that this can resolve the problem.

11

My bad I did not catch that.

So both the routers you show are "clients" to a central server.
In that case they do not need a listen port and of course they do need an endpoint and port defined to connect to the server

But all WG interface should have an address on the WG interface
Server: address: 192.168.101.1/24
add to Peer_client 1 as allowed IP: 192.168.101.2/32
add to Peer_client 2 as Alowed IP: 192.168.101.3/32

Client 1: Address: 192.168.101.2/24, add as Allowed IP on the peer: 192.168.101.0/24
Client 2: Address: 192.168.101.3/24, add as Allowed IP on the peer: 192.168.101.0/24

Note that the clients have only one peer so they can allow the whole WG subnet

12

Thanks for the suggestion - I'll try that again next week.

13

I've created another interface on the central router (so as not to bust the connectivity to the other peers) and implemented the additional interfaces as suggested.

Success. Syslog messages are being passed, and the central router is pingable again from the console of the peer.

Thanks for the guidance, though I am still super curious as to why it was only this router that flipped out after the wan interface was changed from PPPoE to DHCP (after 2 years of reliable service) ??

It might have to remain a mystery, but I'll report in again when the next peer router migrates to DHCP (which might be some time).

Thanks again !

14

The mystery remains !