I am configuring an OpenWRT based client/server pair using two nanopi devices.
The server has a static IP and will be connected via an LTE router.. The static IP is provided by the cellular carrier that provides the LTE router, and that router is in bridge mode so that the external IP shows up as the WAN address for OpenWRT.
The client will have whatever IP address is assigned to it and can expect to be usually NAT'ed behind some router. Details are unknown and need to be irrelevant or this won't work.
At this point I have the server set up on the LTE router and the client is connected on both LAN and WAN through my workstation (different subnets ...). My workstation has 3 NICs so handling multiple connections is not an issue.
Using tcpdump on my workstation so that I can see the client's WAN traffic shows that the server and the client see each other:
sometimes:
18:43:59.117898 IP 192.168.12.191.28360 > 162.191.189.110.30210: UDP, length 148
18:43:59.312563 IP 162.191.189.110.30210 > 192.168.12.191.28360: UDP, length 92
18:44:04.157478 IP 192.168.12.191.28360 > 162.191.189.110.30210: UDP, length 148
18:44:04.413048 IP 162.191.189.110.30210 > 192.168.12.191.28360: UDP, length 92
and sometimes:
18:41:22.872760 IP 162.191.189.110.30210 > 192.168.12.191.28360: UDP, length 148
18:41:22.880576 IP 192.168.12.191.28360 > 162.191.189.110.30210: UDP, length 92
18:41:27.987307 IP 162.191.189.110.30210 > 192.168.12.191.28360: UDP, length 148
18:41:27.994143 IP 192.168.12.191.28360 > 162.191.189.110.30210: UDP, length 92
depending on which side initiates the handshake.
I have both the client and the server configured for debugging; dmesg on the client and on the server looks the same depending on who initiates the handshake.
If the server initiates the handshake, I get this on the client:
[ 7718.785976] wireguard: wg0: Receiving handshake initiation from peer 7 (162.191.189.110:30210)
[ 7718.786771] wireguard: wg0: Sending handshake response to peer 7 (162.191.189.110:30210)
[ 7718.790547] wireguard: wg0: Keypair 317 destroyed for peer 7
[ 7718.791063] wireguard: wg0: Keypair 318 created for peer 7
and on the server:
[ 5356.645012] wireguard: wg0: Sending handshake initiation to peer 3 (184.190.169.121:28360)
[ 5356.929295] wireguard: wg0: Invalid handshake response from 184.190.169.121:28360
[ 5362.393842] wireguard: wg0: Handshake for peer 3 (184.190.169.121:28360) did not complete after 5 seconds, retrying (try 19)
[ 5362.395003] wireguard: wg0: Sending handshake initiation to peer 3 (184.190.169.121:28360)
[ 5362.689348] wireguard: wg0: Invalid handshake response from 184.190.169.121:28360
[ 5367.514011] wireguard: wg0: Handshake for peer 3 (184.190.169.121:28360) did not complete after 5 seconds, retrying (try 20)
and vice-versa when the client initiates the handshake.
The wireguard configuration on the client is:
root@Client:~# wg showconf wg0
[Interface]
ListenPort = 28360
PrivateKey =blahblah
[Peer]
PublicKey = uElTU2ZA6s38hZm7svdb2Cgg2dOtICA6AcpmSwLmQzo=
AllowedIPs = 10.14.7.1/32
Endpoint = 162.191.189.110:30210
PersistentKeepalive = 25
and on the server:
root@Server:~# wg showconf wg0
[Interface]
ListenPort = 30210
PrivateKey =blahblah
[Peer]
PublicKey = f9Nz1Me+l60pAH3dusLBQ4U/DG/70fg8KdcQxmdDm1I=
PresharedKey = blahblah
AllowedIPs = 10.14.7.2/32
Endpoint = 184.190.169.121:28360
PersistentKeepalive = 25
The keys are correct, the server is properly getting the PSK from the client, and the endpoint addresses are right. The devices are talking to each other.
Seems to me this has to be a firewall issue, but I am missing something.
The client firewall is:
root@Client:/etc/config# cat firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option flow_offloading '0'
option synflood_protect '1'
config zone 'lan'
option name 'lan'
list network 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option masq '1'
config zone 'wan'
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option fullcone4 '1'
option fullcone6 '1'
option masq '1'
option mtu_fix '1'
list network 'wan'
list network 'wan6'
list network 'wg0'
config forwarding
option src 'lan'
option dest 'VPN'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'Reject-IPv6'
option family 'ipv6'
option src 'wan'
option dest '*'
option target 'REJECT'
option enabled '0'
config redirect
option dest 'lan'
option target 'DNAT'
option src 'VPN'
option name 'wireguard'
list proto 'all'
config zone
option name 'VPN'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'wg0'
option masq '1'
config forwarding
option src 'VPN'
option dest 'lan'
config forwarding
option src 'lan'
option dest 'VPN'
config rule
option src 'VPN'
option target 'ACCEPT'
option name 'allow ping over vpn'
list proto 'tcp'
list proto 'udp'
list proto 'icmp'
config rule
option name 'allow wireguard'
option src 'VPN'
option target 'ACCEPT'
list proto 'all'
option dest 'VPN'
list src_ip '10.14.7.0/8'
config rule
option name 'allow outbound wg'
option dest 'VPN'
option target 'ACCEPT'
list proto 'all'
list dest_ip '10.14.7.0/8'
config rule
and the server firewall:
root@Server:/etc/config# cat firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option flow_offloading '0'
option synflood_protect '1'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
option masq '1'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option fullcone4 '1'
option fullcone6 '1'
option masq '1'
option mtu_fix '1'
list network 'wan'
list network 'wan6'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'Reject-IPv6'
option family 'ipv6'
option src 'wan'
option dest '*'
option target 'REJECT'
option enabled '0'
config rule
option name 'Allow wireguard'
list proto 'udp'
option src 'VPN'
option target 'ACCEPT'
list src_ip '10.14.7.0/8'
option dest '*'
config zone
option name 'VPN'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option mtu_fix '1'
list network 'wg0'
config forwarding
option src 'VPN'
option dest 'lan'
config forwarding
option src 'lan'
option dest 'VPN'
config rule
option src 'VPN'
option target 'ACCEPT'
list proto 'tcp'
list proto 'udp'
list proto 'icmp'
list src_ip '10.14.7.0/8'
config rule
option name 'incoming wireguard connection'
option dest_port '30210'
option target 'ACCEPT'
list proto 'tcp'
list proto 'udp'
list proto 'icmp'
option src 'wan'
config rule
option name 'allow outbound wireguard'
option dest 'VPN'
option target 'ACCEPT'
list src_ip '10.14.7.0/8'
list proto 'all'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'wireguard'
list proto 'udp'
option src 'VPN'
option src_dport '30210'
option dest_ip '192.168.1.1'
config rule
I just know I'm gonna feel dumb when this problem is found. But can anyone tell me what is wrong?