config interface 'wg0'
option proto 'wireguard'
list addresses '10.0.0.4/32'
option private_key 'PRIVATE_KEY'
option listen_port '55656'
option metric '4'
config wireguard_wg0
option public_key 'PUBLIC_KEY'
option route_allowed_ips '1'
option endpoint_host 'master.server.com'
option persistent_keepalive '0'
list allowed_ips '10.0.0.3/32'
list allowed_ips '0.0.0.0/0'
option endpoint_port '55655'
config interface 'wg1'
option proto 'wireguard'
list addresses '10.0.0.1/32'
option private_key 'PRIVATE_KEY2'
option listen_port '55655'
option metric '1'
config wireguard_wg1
option public_key 'PUBLIC_KEY2'
option route_allowed_ips '1'
list allowed_ips '10.0.0.2/32'
list allowed_ips '172.17.1.0/24'
option persistent_keepalive '0'
So wg0 is WAN VPN connection over the physical WAN interface, it works as wireguard client.
And wg1 is a bridge VPN to other location, it works as wireguard server.
The problem is that only one connection can work at the same time. After any of these is up the second one can't make a handshake. Connection tracking for physical and wireguard WAN's is enabled. Other local services like FTP or HTTP are working fine on physical WAN with any wireguard interfaces connected, so this is not a routing problem.
Yeah, looks like it's the source of the problem. At least entropy is really too low and I've not even thought about this. Now haveged fixed this for me:
root@ROUTER:~# wg
interface: wg0
public key: PUBLIC_KEY3
private key: (hidden)
listening port: 55656
peer: PUBLIC_KEY
endpoint: ENDPOIT_IP:55655
allowed ips: 0.0.0.0/0, 10.0.0.3/32
latest handshake: 5 seconds ago
transfer: 3.35 GiB received, 596.86 MiB sent
interface: wg1
public key: PUBLIC_KEY4
private key: (hidden)
listening port: 55655
peer: PUBLIC_KEY2
endpoint: ENDPOIT_IP2:41538
allowed ips: 172.17.1.0/24, 10.0.0.2/32
transfer: 888 B received, 552 B sent
Because endpoint: ENDPOIT_IP2:41538 shows up by itself while it's not present in config, and some data is transmitted and received, it's still looks like a handshake problem, but entropy is high enough for it to work.
Router runs LEDE Reboot 17.01.5, maybe it's a good idea to try OpenWrt 18.06.1, but I'm not rushing with new releases usually.
It sinks to 1000 and then immediately raises over 2000 and so on. I've never seen embedded device with entropy over 3000, so basically wireguard is incompatible with all embedded devices without hardware accelerated encryption?
I have also 2 SSH connections at the same time, 2 WiFi networks with 8 clients connected total, two wireguard interfaces with one of them working fine, nothing else uses encryption I think.
Is there a way to make wireguard more verbose to debug this?
Looks like it is actually some kind of routing problem. Interesting why it affects only wireguard and none of other router services and forwarded ports, but without list allowed_ips '0.0.0.0/0' router now handles both wireguard connections. I've tested this before of course, but entropy was also a problem.
One of those situations when several problems combined made everything look mystical.
Will try to add packet marking to solve this.
And connection tracking doesn't fix this because it doesn't help for local services? This is why other services through forwarded ports were working fine with default route through VPN, but local wireguard wasn't? Or there is something else I didn't get right?
Connection tracking is a firewall feature, what does it have to do with an incorrect route?
If you are port forwarding through the working WG interface, of course it worked. WG also has roaming (if you don't set an endpoint IP on the remote side), so you should have had a route loop. Better to properly setup routes.