Wireguard Tunnel in Tunnel

Try

config route
        option interface 'wan'
        option target '0.0.0.0'
        option table '2'

config rule
        option src '10.0.102.2/32'
        option priority '2'
        option lookup '2'

Assuming wan is the correct name for the wan interface. Change it to the correct one if not.

I entered the route/rule as you suggested on Site 3 and traffic still exits via Site 1.

The Site 3 Inner Tunnel WG Peer setup has the following settings:

Allowed IPs: 10.0.102.2/32
Route Allowed IPs: Checked

Note: I've tried adding 0.0.0.0/0 and 0.0.0.0/32 to the Allowed IPs as well as checking and unchecking the Route Allowed IPs option in every combination possible and traffic still flows via Site 1.

My Laptop WG config looks like this:

[Interface]
PrivateKey = 
ListenPort = 51821
Address = 10.0.102.2/32
DNS = 1.1.1.1

[Peer]
PublicKey = 
PresharedKey = 
AllowedIPs = 0.0.0.0/0
Endpoint = 10.0.100.4:51821

For what it's worth I do not have masquerading turned on for the inner tunnel firewall zone I created nor is there any NAT happening at Site 3. Not sure if this has implications or not.

I'm bumping this to keep the topic open in hopes that I'll be able to mark it Solved.

After more researching, I'm thinking that Wireguard Allowed IPs and subsequent routing rules are taking priority over the OpenWrt route / rules. Right now the outer tunnel setup pushes all traffic back to Site 1 for Internet. I believe this is grabbing the traffic from the Inner tunnel at Site 3 and sending it back thru the outer tunnel.

Any thoughts on this would be welcome. I'm going to play some more when I have time.

What's the output of ip rule show, ip route show, ip route show table 2 on the router at site 3?

SITE 3

ip rule show

0: from all lookup local
2: from 10.0.102.2 lookup 2
32766: from all lookup main
32767: from all lookup default

ip route show

default dev WG51820 proto static scope link
10.0.100.0/24 dev WG51820 proto static scope link
10.0.100.4 via SITE3.x.x.1 dev eth0 proto static
10.7.7.0/24 dev br-lan proto kernel scope link src 10.7.7.1
SITE3.X.X.0/22 dev eth0 proto kernel scope link src SITE3.X.X.151
SITE1.x.x.x via SITE3.x.x.1 dev eth0 proto static

ip route show table 2

0.0.0.0 dev eth0 proto static scope link

And the contents of the current /etc/config/network and /etc/config/firewall files?

Interesting setup :slight_smile:

Just my thoughts take them for what it is worth :wink:

You have three sites and want to be able to connect from every site to the other sites.

Basically you can use a hub and spoke setup where site 1 is the hub and site 2 and 3 are the spokes, connection form 2 to 3 is routed via site 1.

This looks a bit like you have set up now.

Site 1 is a classic server setup with two peers, site 2 and site 3 as peers with their subnets as allowed ips.

Sites 2 and 3 are setup as a client with respect to that they have one peer (site 1) and endpoint set to site 1 but they are servers in the sense that they should allow incoming traffic basically as a site-to-site setup.
Furthermore site 2 has the subnet of site 1 and site 3 as Allowed IPs and site 3 has as Allowed IPs site 1 and 2.

Now you have a classic three site hub and spoke setup.

But you want extra routing, from site two you want to route all clients to site 1 save one, that client has to be routed to site 3, but as everything must go to site 1 you can simply route everything to site 1 by setting allowed IPs to 0.0.0.0/0 and enable route Allowed IPs.
Next on site 1 you make a routing table to route the client of site 2 to route to site 3.

On site 3 things are more complicated as you want all clients to be routed via site 1 but there is also a client from site 2 which needs to go out via the WAN.
So on this site 3 you should set 0.0.0.0/0 as Allowed IPs but disable Route Allowed IPs.
Use PBR on site 3 to route all your local LAN clients to site 1 via the WG tunnel.
Traffic from the client of site 2 which arrives will not use that routing table and will go out via the WAN.

Note this can all be done with just the one tunnel, the concept of inner and outer tunnel you are now using is "confusing" and not necessary.

Alternative is a mesh setup where all sites connect to all other sites.
Basically all sites are setup as a server with peers to all other sites, but these peers have an endpoint and make a connection, you use PBR on each site to do the routing.
Again each site has just one tunnel.
In your case this mesh setup might be the easier solution provided that each site is reachable via the internet.

Note both for this mesh setup and hub and spoke use as WG address a unique address in the same subnet and make sure all subnets are different.

As always have fun getting this working, I know I have :wink:

SITE 3 - /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdac:9252:9892::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1'
        list ports 'dummy0'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '10.7.7.1'
        option netmask '255.255.255.0'

config interface 'wan'
        option proto 'dhcp'
        option peerdns '0'
        option device 'eth0'
        list dns '1.1.1.1'
        list dns '8.8.8.8'

config interface 'wwan'
        option proto 'dhcp'
        option peerdns '0'
        list dns '1.1.1.1'
        list dns '8.8.8.8'

config interface 'WG51820'
        option proto 'wireguard'
        option private_key ''
        option listen_port '51820'
        list addresses '10.0.100.4/32'
        list dns '1.1.1.1'

config wireguard_WG51820
        option description 'SVR-WG51820'
        option public_key ''
        option preshared_key ''
        option endpoint_host ''
        option endpoint_port '51820'
        option route_allowed_ips '1'
        list allowed_ips '10.0.100.0/24'
        list allowed_ips '0.0.0.0/0'
        option persistent_keepalive '25'

config interface 'WG51821'
        option proto 'wireguard'
        option private_key ''
        option auto '0'
        list addresses '10.0.102.1/24'
        option listen_port '51821'
        option ip4table '2'

config wireguard_WG51821
        option description 'WG-SC-51821'
        option public_key ''
        option private_key ''
        option preshared_key ''
        option endpoint_port '51821'
        option endpoint_host '10.0.100.4'
        list allowed_ips '10.0.102.2/32'
        option route_allowed_ips '1'

config route
        option interface 'wan'
        option target '0.0.0.0/32'
        option table '2'

config rule
        option src '10.0.102.2/32'
        option priority '2'
        option lookup '2'
SITE 3 - /etc/config/firewall

config defaults
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'lan'
        option log '1'

config zone
        option name 'rvwireguard'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'WG51821'
        option log '1'

config zone
        option name 'wan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        list network 'wan'
        list network 'wwan'
        option log '1'

----STANDARD RULES REMOVED FOR POSTING----

config zone
        option name 'wireguard'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        list network 'WG51820'
        option log '1'

config forwarding
        option src 'lan'
        option dest 'wireguard'

config rule
        option name 'Allow-WG-Peer-2-Peer'
        list proto 'tcp'
        option src 'wireguard'
        option dest_port '80 22'
        option target 'ACCEPT'
        list src_ip '10.0.100.3'
        list src_ip '10.0.100.2'
        list src_ip '10.0.100.5'

config rule
        option name 'Allow-RV-WIreguard'
        list proto 'udp'
        option src 'wireguard'
        option dest_port '51821'
        option target 'ACCEPT'

config rule
        option name 'Allow-RVWG-DNS'
        option src 'rvwireguard'
        option dest_port '53 80 443'
        option target 'ACCEPT'
        list proto 'tcp'
        list proto 'udp'
        list proto 'icmp'
        option enabled '0'

config forwarding
        option src 'rvwireguard'
        option dest 'wan'

config rule
        option name 'Allow-WGR'
        option src 'rvwireguard'
        option dest 'wireguard'
        option target 'ACCEPT'
        list proto 'tcp'
        list proto 'udp'

config forwarding
        option src 'rvwireguard'
        option dest 'wireguard'

I had some time to experiment based on your recommendations but wasn't successful. I abandoned the outer/inner tunnel experiment as suggested. I'm pretty sure my issue is with how I've setup the routing tables and rules at Site 1 and 3 because t.

Your hub and spoke description is spot on to what I'm trying to accomplish. Here's a diagram of my setup

My configuration for each site is below based on what I think you were suggesting I do. The result I got was:

--Site 2 still worked with all Internet traffic egressing via Site 1's WAN connection

--Site 3 still connected to Site 1 via the WireGuard interface (I could see Rx/Tx for Site 3 on Site 1), but I could no longer reach Site 3 from Site 2...I use Site 2 to access and administer Site 3.

Since Site 3 is in a location where I don't have local access and operating on the assumption I would not get the configurations correct, I setup a cron job to overwrite Site 3's network config file with the last known good every hour as a failsafe. Site 3 access came back up as expected.

**Site 1**

config interface 'WG51820'
        option proto 'wireguard'
        option private_key ''
        option listen_port '51820'
        list dns '1.1.1.1'
        list dns '8.8.8.8'
        list addresses '10.0.100.1/24'

config wireguard_WG51820
        option description 'SITE2'
        option public_key ''
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_port '51820'
        list allowed_ips '10.0.100.2/32'
        option persistent_keepalive '0'

config wireguard_WG51820
        option public_key ''
        option description 'SITE3'
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_port '51820'
        list allowed_ips '10.0.100.4/32'
        option persistent_keepalive '25'

config route
        option interface 'WG51820'
        option target '10.0.100.4/32'      SITE 3 IP ADDRESS
        option table '2'
        option disabled '1'

config rule
        option src '10.0.100.2/32'         SITE 2 IP ADDRESS
        option priority '2'
        option lookup '2'
        option disabled '1'

**Site 2**

config interface 'WG51820'
        option proto 'wireguard'
        option private_key ''
        option listen_port '51820'
        list addresses '10.0.100.2/32'
        list dns '1.1.1.1'

config wireguard_WG51820
        option description 'SVR-WG51820'
        option public_key ''
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_host 'SITE 1 ENDPOINT'
        option endpoint_port '51820'
        option persistent_keepalive '25'
        list allowed_ips '0.0.0.0/0'
        list allowed_ips '10.0.100.1/32'        SITE 1 IP ADDRESS
        list allowed_ips '10.0.100.4/32'	    SITE 3 IP ADDRESS

**Site 3**

config interface 'WG51820'
        option proto 'wireguard'
        option private_key ''
        option listen_port '51820'
        list addresses '10.0.100.4/32'
        list dns '1.1.1.1'

config wireguard_WG51820
        option description 'SITE1'
        option public_key ''
        option preshared_key ''
        option endpoint_host 'SITE 1 ENDPOINT'
        option endpoint_port '51820'
        option route_allowed_ips '0'
        option persistent_keepalive '25'
        list allowed_ips '0.0.0.0/0'
        list allowed_ips '10.0.100.1/32'       SITE 1 IP ADDRESS
        list allowed_ips '10.0.100.2/32'       SITE 2 IP ADDRESS

config route
        option interface 'WG51820'
        option target '10.0.100.1/32'          SITE 1 IP ADDRESS
        option table '2'

config rule
        option src '10.7.7.0/24'               SITE 3 
        option priority '2'
        option lookup '2'

OK start with site 1:

In the Peer section, Allowed IPs, you have the address of the WG interface that is OK but you also have to add the address of the LAN subnet of that site e.g.
10.7.7.0/24 or whatever the LAN subnet is note it is the whole subnet so /24
(The allowed IPs do not only allowing traffic coming from that ip address but also make a route back)

For now disable all the routing rules of all sites

After a reboot you should be able to go from each site to the other two sites

This assumes for firewall settings the WG interface is placed in the lan zone, (so allowing everything and no masquerading)

First get that going then start with the specific routing

Site 1 now has 10.7.7.0/24 in the allowed ips for the Site 2 and Site 3 peers and includes the WireGuard interface in the Covered Networks of the Lan firewall zone.

From Site 2 I can traceroute to 10.7.7.1 (Site 3) and I see:

traceroute to 10.7.7.1 (10.7.7.1), 30 hops max, 46 byte packets
 1  10.0.100.1 (10.0.100.1)  25.262 ms  17.867 ms  18.716 ms
 2  10.7.7.1 (10.7.7.1)  72.147 ms  67.901 ms  69.592 ms

I can also login to Site 3's LuCI interface by going to 10.7.7.1.

Traffic seems to be successfully traversing Site 1 via the Wireguard interface and landing at Site 3 on the Lan side.

Can you show the wg setup of site 1 so that I can check it?

Site 1

config interface 'WG51820'
        option proto 'wireguard'
        option private_key ''
        option listen_port '51820'
        list dns '1.1.1.1'
        list dns '8.8.8.8'
        list addresses '10.0.100.1/24'

config wireguard_WG51820
        option description 'SITE2'
        option public_key ''
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_port '51820'
        list allowed_ips '10.0.100.2/32'
        list allowed_ips '10.7.7.0/24'
        option persistent_keepalive '0'

config wireguard_WG51820
        option public_key ''
        option description 'SITE3'
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_port '51820'
        list allowed_ips '10.0.100.4/32'
        list allowed_ips '10.7.7.0/24'
        option persistent_keepalive '25'

The peer of site 2 has to have the lan subnet of site 2.
The peer of site 3 has to have the lan subnet of site 3

Correct that reboot and check if all sites can be reached

Just to clarify. We are still talking Site 1 config changes only and I need to add:

list allowed_ips '10.6.6.0/24' to the Site 2 AND Site 3 peer configs?

My Site 3 peer already has the subnet of Site 3. Here's my updated config:

config wireguard_WG51820
        option description 'SITE2'
        option public_key ''
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_port '51820'
        list allowed_ips '10.0.100.2/32'
        list allowed_ips '10.7.7.0/24'        SITE 3 Subnet
        list allowed_ips '10.6.6.0/24'        SITE 2 Subnet
        option persistent_keepalive '0'

config wireguard_WG51820
        option public_key ''
        option description 'SITE3'
        option preshared_key ''
        option route_allowed_ips '1'
        option endpoint_port '51820'
        list allowed_ips '10.0.100.4/32'
        list allowed_ips '10.7.7.0/24'        SITE 3 Subnet
	list allowed_ips '10.6.6.0/24'        SITE 2 Subnet
        option persistent_keepalive '25'

Something weird is happening (I think) when I run traceroute from a console on Site 1 to Site 2

traceroute to 10.6.6.1 (10.6.6.1), 30 hops max, 46 byte packets
 1  10.0.100.4 (10.0.100.4)  33.706 ms  36.638 ms  32.183 ms
 2  10.0.100.4 (10.0.100.4)  51.467 ms  51.390 ms  48.531 ms

10.0.100.4 is the WireGuard IP address assigned to Site 3.

I also am not able to traceroute from a console on Site 3 to Site 2

traceroute to 10.6.6.1 (10.6.6.1), 30 hops max, 46 byte packets
 1  10.0.100.1 (10.0.100.1)  37.799 ms  33.921 ms  37.363 ms
 2  *  *  *
 3  *  *  *
 4  *  *  *

No wonder because you are making a mess of it :frowning:

All fixed and everything is happy now. Sorry for the mess and appreciate you bearing with me/teaching me.

1 Like

Make a backup so that you can go back to a working solution :slightly_smiling_face:

Will do. If you've had enough of me for today, let me know. I assume the next step is the routing rules and tables for Site 1 and Site 3?