This is driving me a little nuts - I'm pretty sure this is a firewall problem (or a route problem) but I'm stumped, and unfortunately it's difficult to get myself into the right network topology to debug.
While wireguard is peer-to-peer - I still tend to think of it in hub-spoke model. I have mydomain.com which has a wireguard 'server' running. I host clients like my phone, but also remote OpenWRT clients.
One of the remote OpenWRT clients - the JFFS2 filesystem is full - ok, so I can fix this because I have newer (bigger) hardware I'm going to replace it with. So I'm trying to replicate the client side setup. I even documented the openwrt client setup on my blog.
My blog post is nearly an identical copy of the OpenWRT documentation - with a few personal details.
One trick I wanted to add to my new setup is instead of binding the wg0 interface to the wan - I want to link it to the lan. Why? Well, this let's me then connect from my "server" side over the wireguard connection to get to the ssh server on the openwrt client. Think of this like a cheater version of site-to-site.
So effectively I have exactly the client setup - but where the firewall adds it to the wan I've changed this to lan.
The wireguard connection works. If I ssh into the openwrt client side - I can ping my main network (the one over the tunnel). I can even reverse ssh from the server side (main network) back over the wireguard connection - and hit the ssh server on the openwrt client.
FWIW - I've set up the connection as a split tunnel - so only traffic for my main network is going over the wg0 link - stuff that goes to the internet is left alone.
What doesn't work is wifi clients on the openwrt client side -- they can't seem to get to the main network over the tunnel. This just keeps stumping me.
Of course - getting setup to debug this from both sides is tricky - thus slowing down my discovery process.
Maybe a quick fix if that does not help we need to see the configs.
On the server side on the peer for your openwrt client you need to add the subnet of the openwrt client router as allowed ips.
Note this only works if all involved subnets are different!
Oh - yeah - I for sure have different sub-nets for different 'locations' - I actually ran into this problem with my travel router overlapping with the rental wifi.. very annoying.
My subnet strategy is 10.<random value>.<another random value>.x - which mostly guarantee's that I won't have overlap with existing networks.
On the server side on the peer for your openwrt client you need to add the subnet of the openwrt client router as allowed ips.
This seems like a good thing to try - but where in the configs to I configure this?
Dumping the routes - things seem pretty much right?
# ip route show table all
default via 10.199.44.1 dev eth0.2 src 10.199.44.52
10.23.45.0/24 dev wg0 scope link
10.122.33.0/24 dev br-lan scope link src 10.122.33.1
10.199.44.0/22 dev eth0.2 scope link src 10.199.44.52
24.246.7.168 via 10.199.44.1 dev eth0.2
First let's talk about the "physical" layout
There is a home network - and that home network has an external IP
There is the remote network
The OpenWRT is plugged into the remote network, and has a wireguard connection to the home network
Let's break down some of those addresses above
10.199.44.1 - remote network gateway
10.199.44.52 - address of the OpenWRT client - the installation I'm trying to get right
10.122.33.0/24 - CIDR block of the remote network that the OpenWRT client is plugged into
10.23.45.0/24 - CIDR block for my home (lan) network
24.246.7.168 - the public (external) IP of my home network
Again - important to note that if I'm ssh'd into the OpenWRT client - the wireguard connection is working fine - and I can see my home network from that remote site.
Wireless devices connected to that OpenWRT client can't. Which is what I want to fix.
Bonus - ascii diagram of the situation
[ home network ] <-> [external IP] <-- internet <-- [remote network] <- [openwrt client] <- [wifi client of openwrt]
So while the [openwrt client] can see my home network via wireguard, the [wifi client of openwrt] cannot..
It can help if we see the whole picture, please connect to your OpenWRT device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button
Remember to redact keys, passwords, MAC addresses and any public IP addresses you may have:
ubus call system board
cat /etc/config/network
cat /etc/config/firewall
ip route show
wg show
I keep wishing for a nice 'client side' web tool that would let me easily paste in and sanitize a data dump like this. It would help users not disclose sensitive data. Maybe I'll eventually write one.
# ubus call system board
{
"kernel": "5.15.167",
"hostname": "clientOpenWRT",
"system": "Qualcomm Atheros QCA956X ver 1 rev 0",
"model": "TP-Link Archer C7 v5",
"board_name": "tplink,archer-c7-v5",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "23.05.5",
"revision": "r24106-10cc5fcd00",
"target": "ath79/generic",
"description": "OpenWrt 23.05.5 r24106-10cc5fcd00"
}
}
# cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fdc3:6c2f:2e5b::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0.1'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '10.133.32.1'
option netmask '255.255.255.0'
option ip6assign '60'
config device
option name 'eth0.2'
option macaddr '34:cc:a5:98:4b:7e'
config interface 'wan'
option device 'eth0.2'
option proto 'dhcp'
config interface 'wan6'
option device 'eth0.2'
option proto 'dhcpv6'
config switch
option name 'switch0'
option reset '1'
option enable_vlan '1'
config switch_vlan
option device 'switch0'
option vlan '1'
option ports '2 3 4 5 0t'
config switch_vlan
option device 'switch0'
option vlan '2'
option ports '1 0t'
config interface 'wg0'
option proto 'wireguard'
option private_key 'SECRETSECRET='
list addresses '10.13.13.10/32'
config wireguard_wg0 'wgserver'
option public_key 'SECRETSECRET='
option preshared_key 'SECRETSECRET='
option endpoint_host 'mydomain.com'
option endpoint_port '51820'
option route_allowed_ips '1'
option persistent_keepalive '25'
list allowed_ips '10.23.45.0/24'
# cat /etc/config/firewall
config defaults
option syn_flood '1'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
config zone 'lan'
option name 'lan'
list network 'lan'
list network 'wg0'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
config zone 'wan'
option name 'wan'
list network 'wan'
list network 'wan6'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
# ip route show
default via 10.118.32.1 dev eth0.2 src 10.118.32.56
10.23.45.0/24 dev wg0 scope link
10.133.32.0/24 dev br-lan scope link src 10.133.32.1
10.118.32.0/22 dev eth0.2 scope link src 10.118.32.56
24.246.7.168 via 10.118.32.1 dev eth0.2
# wg show
interface: wg0
public key: SECRETSECRET=
private key: (hidden)
listening port: 44944
peer: SECRETSECRET=
preshared key: (hidden)
endpoint: 24.246.7.168:51820
allowed ips: 10.23.45.0/24
latest handshake: 1 minute, 28 seconds ago
transfer: 131.15 KiB received, 271.88 KiB sent
persistent keepalive: every 25 seconds
ah.. I see the 'both sides' - and the other side is NOT OpenWRT -- well at least the wireguard is not. The destination wireguard is a linuxserver.io container running wireguard - and thus the 10.13.13.x addressing. This is working fine for multiple clients. I don't think we need to get into that side of things.
Use a /24 mask: list addresses '10.13.13.10/24' this will take care of a proper route for the WG subnet
Add the WG subnet
the above are not really show stoppers and otherwise things look fine
The other side is equally important as that should have a route to this side 10.133.32.0/24.
You can do that by adding 10.133.32.0/24 as allowed IPs to the Peer section, assuming route allowed IPs is enabled (which is standard in Linux)
This should take care of the routing between the two routers.
Note that your LAN clients have their own firewall so your clients of the OpenWRT router should allow traffic from 10.23.45.0/24 and if you have also other WireGuard clients connected to your linux server then also allow the WG subnet: 10.13.13.0/24
A quick and dirty work around is to NAT traffic coming out of the lan:
/etc/config/firewall:
config nat
option name 'SNAT-wgserver'
option src 'lan'
option src_ip '10.23.45.0/24'
option src_ip '10.13.13.0/24'
option target 'MASQUERADE'
list proto 'all'
option enabled '1'
If I make the one change above in my /etc/config/firewall I resolve my visibility issue.
config zone 'lan'
option name 'lan'
list network 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
config zone 'wan'
option name 'wan'
list network 'wan'
list network 'wan6'
list network 'wg0'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
Clients on the wifi can now see the home network.
Client on wifi IP: 10.133.32.221
Home network IP: 10.23.45.14
Ping from client..
% ping 10.23.45.14
PING 10.23.45.14 (10.23.45.14): 56 data bytes
64 bytes from 10.23.45.14: icmp_seq=0 ttl=62 time=24.021 ms
64 bytes from 10.23.45.14: icmp_seq=1 ttl=62 time=28.739 ms
So while this is good - I really want to connect my wireguard to the lan side of the firewall because that will let me play the 'trick' of tunneling back over the wireguard connection from the 'home' side to the router ssh interface (which isn't listening to the WAN side at all, but is on the LAN side)
config nat
option name 'SNAT-wgserver'
option src 'lan'
option src_ip '10.23.45.0/24'
option src_ip '10.13.13.0/24'
option target 'MASQUERADE'
list proto 'all'
option enabled '1'
As well as the additional line to my /etc/config/network
list allowed_ips '10.13.13.0/24' <<<< add
The result was the same - WiFi clients cannot see my home network, but if I ssh into the router it can see it just fine.
I may just give up - and leave the wireguard connected to the 'wan' side in the firewall, and allow ssh access from the WAN.
While this sounds dangerous, the OpenWRT in question - is behind a secondary router which is providing LTE access - and that connection is CGNAT as well. It is an unlikely attack vector.
Ok - so that works - I'm not entirely satisfied because it feels like there should be a more elegant solution.
I had to give up on putting the wireguard on the lan side of the firewall. This means that my wireguard configuration is just like the openwrt doc
This means that from the wireguard server side (my home network) - when I tried to connect back over the wireguard connection - I am blocked because the OpenWRT firewall doesn't have port 22 exposed. Unlike when I'm connected to the OpenWRT device via wireless and I can ssh to the router itself.
This is easy to fix with a firewall rule DANGER DANGER
config redirect
option dest 'lan'
option target 'DNAT'
option name 'OpenWRT ssh'
list proto 'tcp'
option src 'wan'
option src_dport '22'
option dest_ip '10.133.32.1'
option dest_port '22'
Where 10.133.32.1 is the IP of my OpenWRT router.
DO NOT DO THIS if your OpenWRT router is your direct connection to the internet. This allows traffic from the WAN side to see ssh.
In my case - the router is already behind another router (because that is my LTE connection).
In the end, while this is ugly - it is a simple solution. Simple is good.
You are correct that the required routes are needed - I had previous to this post already successfully set that up.
Thus, not a problem I had - but great for others reading this trying to get 'bi-directional' wireguard traffic to flow.
In summary - while I wasn't able to fix the routing issues on the openwrt wireguard client side (one day I'll wrap my head around all of this better) - the main problem I had was that with the recommended openwrt wireguard client setup, was that I wanted to access it via ssh from the "other side" of that link - and of course that was blocked by the firewall.