Wireguard unidirectional connections

eduperez · December 6, 2023, 8:20pm

I am having a weird issue with a site-to-site wireward tunnel, where traffic flows in both directions, but only for traffic initiated from one of the sites.

Let's say I have site called SERVER and site called CLIENT.

Site SERVER has several LAN networks, all within the 192.168.0.0/17 range, a public IP address, and a DDNS domain name; site CLIENT has several LAN networks, all within the 192.168.128.0/17 range, and sits behind a CGNAT.

This is the configuration on SERVER:

config interface 'wg_s2s_a'
	option proto 'wireguard'
	option private_key '[REDACTED]'
	option listen_port '51820'

config wireguard_wg_s2s_a 's2s_vpn_site_b'
	option public_key '[REDACTED]'
	option preshared_key '[REDACTED]'
	list allowed_ips '192.168.128.0/17'
	list allowed_ips 'fdb1:f846:87e0::/48'
	option route_allowed_ips '1'
	option persistent_keepalive '25'

And this is the configuration con CLIENT:

config interface 'wg_s2s_b'
	option proto 'wireguard'
	option private_key '[REDACTED]'
	option listen_port '51820'

config wireguard_wg_s2s_b 's2s_vpn_site_a'
	option public_key '[REDACTED]'
	option preshared_key '[REDACTED]'
	option endpoint_host '[REDACTED]'
	option endpoint_port '51820'
	list allowed_ips '192.168.0.0/17'
	list allowed_ips 'fdaf:5160:5d0f::/48'
	option route_allowed_ips '1'
	option persistent_keepalive '25'

The tunnel seems to be working OK on SERVER:

interface: wg_s2s_a
  public key: [REDACTED]
  private key: (hidden)
  listening port: 51820

peer: [REDACTED]
  preshared key: (hidden)
  endpoint: [REDACTED]:56245
  allowed ips: 192.168.128.0/17, fdb1:f846:87e0::/48
  latest handshake: 1 minute ago
  transfer: 71.21 KiB received, 73.57 KiB sent
  persistent keepalive: every 25 seconds

The tunnel seems to be working OK on CLIENT:

interface: wg_s2s_b
  public key: [REDACTED]
  private key: (hidden)
  listening port: 51820

peer: [REDACTED]
  preshared key: (hidden)
  endpoint: [REDACTED]:51820
  allowed ips: 192.168.0.0/17, fdaf:5160:5d0f::/48
  latest handshake: 3 seconds ago
  transfer: 125.69 KiB received, 284.06 KiB sent
  persistent keepalive: every 25 seconds

The routing on SERVER is also OK:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
[REDACTED]
192.168.1.0     *               255.255.255.0   U     0      0        0 br_lan
192.168.4.0     *               255.255.255.0   U     0      0        0 br_iot
192.168.5.0     *               255.255.255.0   U     0      0        0 br_kid
192.168.7.0     *               255.255.255.0   U     0      0        0 br_ext
192.168.128.0   *               255.255.128.0   U     0      0        0 wg_s2s_a

The routing on CLIENT is also OK:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
[REDACTED]
192.168.0.0     *               255.255.128.0   U     0      0        0 wg_s2s_b
192.168.129.0   *               255.255.255.0   U     0      0        0 br-lan
192.168.132.0   *               255.255.255.0   U     0      0        0 br-iot
192.168.135.0   *               255.255.255.0   U     0      0        0 br-ext

And now comes the fun part...

I can connect from any device (including the router) on SERVER's LAN to any device (including the router) on CLIENT's LAN. But I cannot connect from the router at CLIENT to the router at SERVER. I have tried to use tcpdump on SERVER, and I see the packets reaching the WAN interface, but I cannot see them on the wireguard interface, it looks like wireguard is rejecting or dropping those packets.

Any idea of what can be happening here, please?

egc · December 6, 2023, 8:25pm

Are both wireguard interfaces in the firewall lan zone?

NPeca75 · December 6, 2023, 8:27pm

Hi

you have kind of not obvious error & have a complex network,
maybe best thing is to stop firewall on both side for a moment to rule out firewall errors ?

eduperez · December 6, 2023, 8:47pm

I have several LAN zones on each end, and the wireguard interfaces are on their own zone, at both ends. I tried moving it to one of the LAN zones (on both ends), but the situation has not changed.

Thanks!

eduperez · December 6, 2023, 8:49pm

I just tried this, but the situation has not improved.

Thanks!

egc · December 6, 2023, 8:56pm

Just a thougt, both client and server specify a listening port of 51820, but the server shows connected to port 56245.

Try removing the listening port on the client.

Is port 51820 occupied on the client?

andyboeh · December 6, 2023, 9:10pm

I have a similar setup with several subnets on each side, but my configuration explicitly lists every /24 subnet. I do not use route_allowed_ips and create the routes myself.

I had a similar problem though: Traffic sent from one of the routers would have a wrong source IP set - but since you did tcpdump traces you would have probably spotted that.

In any case, my fix is a new route definition in /etc/config/network:

config route
	option interface 'WireGuard'
	option target '192.168.19.0/24'
	option source '192.168.17.1'

eduperez · December 7, 2023, 7:19am

I tried to change the port (ensuring to use a free one), and I tried to remove it from the client; but no changes here...

Thanks!

psherman · December 7, 2023, 7:23am

Can we see the complete (redacted where necessary) network and firewall files from each side?

_bernd · December 7, 2023, 7:33am

Only one peer needs to have static IP or a ddns record and a listen port set to be listening on. In your case the server.
The client however needs only to know where to connect.
If one site sits behind a chat you have no control over the outgoing port number so just do not set it.
I assume the issue is when the server wants to connect to the client.
I used this for site to site vpns in this situation. Just set the keepalive timers on the connection to ensure that the tunnel stays "up".

eduperez · December 7, 2023, 2:57pm

Yep, sure, of course! I just wanted to spare you all from the boring task of reading these looong files. I might have made some changes since my last post, but this configuration still works as I explained at the beginning of this thread:

Network config at SERVER:

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdaf:5160:5d0f::/48'
	option packet_steering '1'

config device
	option type '8021q'
	option ifname 'wan'
	option vid '6'
	option name 'wan_data'

config device
	option type '8021q'
	option ifname 'wan'
	option vid '3'
	option name 'wan_voip'

config device
	option type '8021q'
	option ifname 'wan'
	option vid '2'
	option name 'wan_iptv'

config interface 'wan'
	option proto 'pppoe'
	option device 'wan_data'
	option username '[REDACTED]'
	option password '[REDACTED]'
	option keepalive '5 5'
	option peerdns '0'
	option ipv6 '0'

config interface 'wan6'
	option proto '6in4'
	option tunlink 'wan'
	option peeraddr '[REDACTED]'
	option tunnelid '[REDACTED]'
	option username '[REDACTED]'
	option password '[REDACTED]'
	option ip6addr '[REDACTED]/64'
	list ip6prefix '[REDACTED]/48'
	option peerdns '0'
	option mtu '1472'

config device
	option name 'br-eth'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-eth'
	option vlan '1'
	list ports 'lan1'
	list ports 'lan4:t'

config device
	option type '8021q'
	option ifname 'br-eth'
	option vid '1'
	option name 'br_lan'

config interface 'lan'
	option proto 'static'
	option device 'br_lan'
	option ipaddr '192.168.1.254'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6hint 'fff1'

config bridge-vlan
	option device 'br-eth'
	option vlan '4'
	list ports 'lan3'
	list ports 'lan4:t'

config device
	option type '8021q'
	option ifname 'br-eth'
	option vid '4'
	option name 'br_iot'

config interface 'iot'
	option proto 'static'
	option device 'br_iot'
	option ipaddr '192.168.4.254'
	option netmask '255.255.255.0'

config device
	option type '8021q'
	option ifname 'br-eth'
	option vid '5'
	option name 'br_kid'

config bridge-vlan
	option device 'br-eth'
	option vlan '5'
	list ports 'lan4:t'

config interface 'kid'
	option proto 'static'
	option device 'br_kid'
	option ipaddr '192.168.5.254'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6hint 'fff5'

config device
	option type '8021q'
	option ifname 'br-eth'
	option vid '7'
	option name 'br_ext'

config bridge-vlan
	option device 'br-eth'
	option vlan '7'
	list ports 'lan2'
	list ports 'lan4:t'

config interface 'ext'
	option proto 'static'
	option device 'br_ext'
	option ipaddr '192.168.7.254'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6hint 'fff7'

config interface 'wg0'
	option proto 'wireguard'
	option private_key '[REDACTED]'
	option listen_port '51820'

config wireguard_wg0 'wg0_sfg'
	option public_key '[REDACTED]'
	option preshared_key '[REDACTED]'
	list allowed_ips '192.168.128.0/17'
	list allowed_ips 'fdb1:f846:87e0::/48'
	option route_allowed_ips '1'
	option persistent_keepalive '25'

Firewall config at SERVER:

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'lan'
	option dest 'wan'

config forwarding
	option src 'lan'
	option dest 'iot'

config forwarding
	option src 'lan'
	option dest 'vpn'

config zone
	option name 'iot'
	list network 'iot'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'iot'
	option dest 'wan'

config forwarding
	option src 'iot'
	option dest 'vpn'

config zone
	option name 'kid'
	list network 'kid'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'kid'
	option dest 'wan'

config zone
	option name 'ext'
	list network 'ext'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'ext'
	option dest 'wan'

config zone
	option name 'vpn'
	option device 'wg0'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'vpn'
	option dest 'iot'

config ipset
	option name 'bogons'
	option family 'ipv4'
	option match 'src_net'
	option loadfile '/etc/bogons'

config rule
	option name 'bogons'
	option src 'wan'
	option proto 'all'
	option family 'ipv4'
	option ipset 'bogons'
	option target 'DROP'
	option log 'FW_BOGON '

config ipset
	option name 'banned'
	option family 'ipv4'
	option match 'src_ip'
	option timeout '262144'

config rule
	option name 'banned'
	option src 'wan'
	option proto 'all'
	option family 'ipv4'
	option ipset 'banned'
	option target 'DROP'
	option log 'FW_BANNED '

Network config at CLIENT:

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdb1:f846:87e0::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'

config interface 'lan'
	option proto 'static'
	option device 'br-lan'
	option ipaddr '192.168.129.254'
	option netmask '255.255.255.0'

config device
	option name 'br-iot'
	option type 'bridge'

config interface 'iot'
	option device 'br-iot'
	option proto 'static'
	option ipaddr '192.168.132.254'
	option netmask '255.255.255.0'

config device
	option name 'br-ext'
	option type 'bridge'

config interface 'ext'
	option device 'br-ext'
	option proto 'static'
	option ipaddr '192.168.135.254'
	option netmask '255.255.255.0'

config interface 'wan'
	option device '/dev/cdc-wdm0'
	option proto 'qmi'
	option pincode '[REDACTED]'
	option pdptype 'ipv4v6'
	option apn '[REDACTED]'
	option username '[REDACTED]'
	option password '[REDACTED]'
	option peerdns '0'

config interface 'wg0'
	option proto 'wireguard'
	option private_key '[REDACTED]'

config wireguard_wg0 'wg0_srv'
	option public_key '[REDACTED]'
	option preshared_key '[REDACTED]'
	option endpoint_host '[REDACTED]'
	option endpoint_port '51820'
	list allowed_ips '192.168.0.0/17'
	list allowed_ips 'fdaf:5160:5d0f::/48'
	option route_allowed_ips '1'
	option persistent_keepalive '25'

Firewall config at CLIENT:

config defaults
	option syn_flood '1'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'lan'
	option dest 'wan'

config forwarding
	option src 'lan'
	option dest 'iot'

config forwarding
	option src 'lan'
	option dest 'vpn'

config zone
	option name 'iot'
	list network 'iot'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'iot'
	option dest 'wan'

config forwarding
	option src 'iot'
	option dest 'vpn'

config zone
	option name 'ext'
	list network 'ext'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'ext'
	option dest 'wan'

config zone
	option name 'vpn'
	list network 'wg0'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'vpn'
	option dest 'iot'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config rule                          
	option name 'SSH'            
	option src 'wan'             
	option proto 'tcp'           
	option dest_port '22'        
	option target 'ACCEPT'

eduperez · December 7, 2023, 3:01pm

Yes, thas is also how I understand that this works. Also, notice how the devices on the SERVER's network (the one with a public IP address) can reach the devices on the CLIENT's network (the one behind the CGNAT); I could get to understand the opposite case, but this situation is confusing.

eduperez · December 7, 2023, 3:14pm

I tried to restrict the allowed_ips to just the main LAN on each side (192.168.129.0/24 - 192.168.1.0/24), and something weird happened...

Now the router at SERVER cannot reach the router at CLIENT, using its address on the main LAN (and viceversa), but the devices on the main LAN from SERVER can reach the CLIENT router. And this matches the situation you explained:

How can I know which source address is using the router at CLIENT when it tries to reach the SERVER?

egc · December 7, 2023, 3:18pm

On the server:

Do you not need the other way around also?

Maybe it is there but indeed it is a loooong list

eduperez · December 7, 2023, 3:28pm

And we have a winner here!!!

I repeated the tests again, but this time I used tcpdump on the wireguard interface from the CLIENT, while it was doing a PING to the SERVER and... it was using CLIENT's WAN address!

I repeated the PING, this time specifying a correct source address (one from CLIENT's LAN), and it worked. Now I just need to figure out how to fix it for good, so it works for all the connections without having to specify a source address.

Many thanks!!!

eduperez · December 7, 2023, 3:44pm

Ok, I finally managed to fix this!

Why does CLIENT send packets using the wrong source address? Because it's end of the tunnel does not have any address, and it uses the first one it finds! I have been using the wireguard connection as a mere tunnel, and never thought I needed to give it an address.

This is my network config on the SERVER now:

config interface 'wg0'
	option proto 'wireguard'
	option private_key '[REDACTED]'
	option listen_port '51820'
	list addresses '172.16.0.1/32'

config wireguard_wg0 'wg0_sfg'
	option public_key '[REDACTED]'
	option preshared_key '[REDACTED]'
	list allowed_ips '172.16.0.0/24'
	list allowed_ips '192.168.128.0/17'
	list allowed_ips 'fdb1:f846:87e0::/48'
	option route_allowed_ips '1'
	option persistent_keepalive '25'

And on the CLIENT:

config interface 'wg0'
	option proto 'wireguard'
	option private_key '[REDACTED]'
	list addresses '172.16.0.2/32'

config wireguard_wg0 'wg0_srv'
	option public_key '[REDACTED]'
	option preshared_key '[REDACTED]'
	option endpoint_host '[REDACTED]'
	option endpoint_port '51820'
	list allowed_ips '172.16.0.0/24'
	list allowed_ips '192.168.0.0/17'
	list allowed_ips 'fdaf:5160:5d0f::/48'
	option route_allowed_ips '1'
	option persistent_keepalive '25'

(notice the new 172.16.0.0 network).

I have marked @andyboeh's post as a solution, but I have to thank all of you who helped me with this issue.

egc · December 7, 2023, 3:46pm

So the lesson learned: Always give the WG interface an address?

krazeh · December 7, 2023, 3:47pm

I do similar and, on testing just now, have the same issue you were seeing. But I've never needed to connect from either of the routers to anything on the other side of the tunnel, so have never experienced the issue in practice. Good to know it can be solved by assigning addresses to the interfaces on each end though.

_bernd · December 7, 2023, 4:19pm

Now that you have said it it strikes me that on your initial post the client has no address on the interface. And of course an wg interface should have an address because it's layer 3.

In addition: yes with Linux and some other platforms you can have so called unnumbered interfaces and then if you for instance have a non loopback IP assigned on loopback this is used.

This is for instance used on ospf networks to avoid the need to assign addresses on each and every interface.

andyboeh · December 7, 2023, 5:51pm

Great, I'm glad it helped Although I always had IP addresses on the WG interfaces, I needed the explicitly specify the source address with the additional route. I think I did this because I ended up with the wireguard interface addresses on the remote machines and I wanted the LAN IP addresses in all cases.