I would approach this a bit differently...
Thanks all!
For an English description, for now what we're trying to do is have all traffic in and out of the secondary router (from all of its interfaces) to pass through the WireGuard VPN to have that additional layer of encryption, given that it's remotely located via the Ubiquiti dishes. The secondary router should only be able to access the internet (through the primary router's WAN interface) outbound. And the primary router's LAN interface (which correspond to ethernet jacks 1 and 2 on the router) should be able to access the secondary router (this lets us access the secondary router via ssh/Luci). The current firewall zones permit that traffic setup.
So, having said all that, we can now get the WireGuard interface to work! The final key was enabling masquerading on the secondary router's firewall zone that was attached to the WireGuard interface.
So, we're basically there thanks to all your help! What an education and a valuable thread!
It's also clear that we didn't understand how the firewall would work with the new WireGuard interface. To explain, in order to get the WireGuard interface to work on the secondary router with its own separate firewall zone (let's call that zone WG), we had to allow it to forward to the WAN and allow forwarding from LAN_PORT_1. And then I had to allow LAN_PORT_1 to forward to the WG zone. In other words, see below. In retrospect, this makes sense. But is another piece to the puzzle we were missing since we didn't have a solid understanding of how the tunnel worked.
config forwarding
option src 'WG'
option dest 'WAN'
config forwarding
option src 'LAN_PORT_1'
option dest 'WG'
So, I think it this point we're in a position to configure the WireGuard firewall to function the same as our original configuration (as it did without the WireGuard interface working).
It is interesting, that when I stop the WireGuard interface through Luci on the secondary router, internet access doesn't return. The WAN interface on this secondary router has to be restarted to restore internet access.
I'll come back tomorrow to write a summary so the solution to my experience is available for others to read and I'll mark the thread as solved.
Glad everything is working as you want now!
this is not unexpected, actually. It is a quirk of the way that the WG interface comes up -- it effectively replaces the existing routing table and does not restore it when the interface goes down. In order to resolve this, you can do one of two things:
Meanwhile...
If your problem is solved, please consider marking this topic as [Solved]. See How to mark a topic as [Solved] for a short how-to.
Hi all.
psherman, what a great idea, thanks! We added the hotplug.d script to fix that little routing table quirk!
However, I was all ready to come back and summarize the problem/solution and mark the thread as solved so maybe it could help other folks, but...
We thought we had it but it isn't quite there yet. We're able to get one VLAN on the secondary router to be able to access the internet. However, the other VLANs can't. That's really confusing to us.
We decided to draw a simplified diagram of our network to help explain. Relative to our conversation thus far, the main router is the ISP Router and the secondary router is Router 3. The VLAN/Firewall names are in the color corresponding their associated routers. I hope it's readable.
Without the WireGuard interfaces enabled, everything works as intended. But with the WireGuard interfaces enabled and functioning only Router 3a can access the internet just fine through WireGuard. Router 3b and wireless devices on Router 3 can't. Which is really strange.
The firewall configuration for Router 3 is below. I can't see why internet access works for only the R3_LAN_PORT_1 interface. Do you think we're missing something here?
Also, without the WireGuard interface, we are able to ssh 192.168.3.215 into Router 3 from the ISP Router. When the WireGuard interface is up, we can only access Router 3 by ssh 10.11.14.2. That's not a big deal, but curious if we can route 192.168.3.215 into Router 3 when the WireGuard interface is up. We tried some static route variations to no avail.
Thanks everyone.
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
option synflood_protect '1'config zone
option output 'ACCEPT'
option mtu_fix '1'
option network 'wan wan6'
option masq '1'
option forward 'REJECT'
option input 'DROP'
option name 'WAN'config rule
option name 'Allow-DHCP-Renew'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
option src 'WAN'config rule
option name 'Allow-Ping'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
option src 'WAN'config rule
option name 'Allow-IGMP'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
option src 'WAN'config rule
option name 'Allow-DHCPv6'
option proto 'udp'
option src_ip 'fc00::/6'
option dest_ip 'fc00::/6'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
option src 'WAN'config rule
option name 'Allow-MLD'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
option src 'WAN'config rule
option name 'Allow-ICMPv6-Input'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
option src 'WAN'config rule
option name 'Allow-ICMPv6-Forward'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
option src 'WAN'config include
option path '/etc/firewall.user'config zone
option name 'R3_LAN_Port_4'
option output 'ACCEPT'
option network 'R3_LAN_Port_4'
option forward 'REJECT'
option input 'ACCEPT'config zone
option name 'R3_LAN_Port_3'
option output 'ACCEPT'
option network 'R3_LAN_Port_3'
option forward 'REJECT'
option input 'ACCEPT'config zone
option name 'Wireless'
option output 'ACCEPT'
option network 'Wireless'
option input 'ACCEPT'
option forward 'REJECT'config forwarding
option src 'R3_LAN_Port_4'
option dest 'WAN'config forwarding
option src 'R3_LAN_Port_3'
option dest 'WAN'config forwarding
option src 'Wireless'
option dest 'WAN'config rule
option target 'ACCEPT'
option src 'Wireless'
option name 'Wireless DHCP'
option dest_port '67-68'config rule
option target 'ACCEPT'
option src 'Wireless'
option name 'Wireless DNS'
option dest_port '53'config rule
option target 'ACCEPT'
option src 'R3_LAN_Port_4'
option name 'R3_LAN Port 4 DHCP'
option dest_port '67-68'config rule
option target 'ACCEPT'
option src 'R3_LAN_Port_4'
option name 'R3_LAN Port 4 DNS'
option dest_port '53'config rule
option target 'ACCEPT'
option name 'R3_LAN Port 3 DHCP'
option src 'R3_LAN_Port_3'
option dest_port '67-68'config rule
option target 'ACCEPT'
option src 'R3_LAN_Port_3'
option name 'R3_LAN Port 3 DNS'
option dest_port '53'config zone
option output 'ACCEPT'
option name 'R3_LAN_Port_1'
list network 'R3_LAN_Port_1'
option input 'ACCEPT'
option forward 'REJECT'config forwarding
option src 'R3_LAN_Port_1'
option dest 'WAN'config zone
option name 'R3_LAN_Port_2'
option network 'R3_LAN_Port_2'
option output 'ACCEPT'
option forward 'REJECT'
option input 'ACCEPT'config forwarding
option src 'R3_LAN_Port_2'
option dest 'WAN'config rule
option src 'R3_LAN_Port_2'
option name 'R3_LAN Port 2 DHCP'
option target 'ACCEPT'
option dest_port '67-68'config rule
option src 'R3_LAN_Port_2'
option name 'R3_LAN Port 2 DNS'
option target 'ACCEPT'
option dest_port '53'config rule
option dest_port '67-68'
option src 'R3_LAN_Port_1'
option name 'R3_LAN Port 1 DNS'
option target 'ACCEPT'config rule
option dest_port '53'
option src 'R3_LAN_Port_1'
option target 'ACCEPT'
option name 'R3_LAN Port 1 DNS'config rule
option name 'Luci from WAN'
list proto 'tcp'
option target 'ACCEPT'
option dest_port '80'
option src 'WAN'config rule
option name 'Luci from WAN SSL'
list proto 'tcp'
option target 'ACCEPT'
option dest_port '443'
option src 'WAN'config rule
option name 'SSH from WAN'
list proto 'tcp'
option target 'ACCEPT'
option dest_port '22'
option src 'WAN'config zone
option name 'WireGuard'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'WireGuard'config forwarding
option src 'WireGuard'
option dest 'WAN'config forwarding
option src 'R3_LAN_Port_1'
option dest 'WireGuard'config forwarding
option src 'WireGuard'
option dest 'R3_LAN_Port_1'config forwarding
option src 'WAN'
option dest 'WireGuard'config rule
option name 'Luci from WireGuard'
option src 'WireGuard'
option dest_port '80'
option target 'ACCEPT'config rule
option name 'Luci from WireGuard SSL'
option src 'WireGuard'
option dest_port '443'
option target 'ACCEPT'config forwarding
option src 'R3_LAN_Port_2'
option dest 'WireGuard'config forwarding
option src 'WireGuard'
option dest 'R3_LAN_Port_2'config forwarding
option src 'R3_LAN_Port_3'
option dest 'WireGuard'config forwarding
option src 'WireGuard'
option dest 'R3_LAN_Port_3'config forwarding
option src 'Wireless'
option dest 'WireGuard'config forwarding
option src 'WireGuard'
option dest 'Wireless'config forwarding
option src 'R3_LAN_Port_4'
option dest 'WireGuard'config forwarding
option src 'WireGuard'
option dest 'R3_LAN_Port_4'
Try enabling masquerading on the WireGuard firewall zone.
FWIW, that firewall has a bunch of unnecessary rules, but they probably don't cause any issues here.
Also, this topology is really odd... why not just have a single router create all of the networks (and firewall rules governing the inter-VLAN routing). The rest of the routers would then simply be dumb APs/switches. This would simplify your routing considerably and make management much easier.
Your diagram isn't fully complete, since the routers hold more than one IP address.
Relevant to this situation we have:
ISP router
WAN : an Internet IP. If CGNAT, not a public routable IP, you have to be sure it doesn't conflict with any of your networks.
LAN3 : 192.168.3.1/24
Other LANs : .4.1, .8.1 etc -- not relevant to this problem
WG0 : 10.11.14.1/24
Router 3:
WAN 102.168.3.215/24
WG0 : 10.11.14.2/24
LAN1 : 192.168.11.1/24
LAN2 : 192.168.12.1/24
Router 3A:
WAN : 192.168.11.101/24
LAN : 192.168.6.1
Router 3B is basically the same situation as 3A.
Now working backwards from the desktop computer which is an endpoint user at the Router 3 site, Router 3A should NAT its Internet request from 192.168.6.x to 192.168.11.101 and forward it to 192.168.11.1 which is router 3. This is the same as if Router 3A were connected to an ISP directly. Basically Router 3 is the ISP for everything at site 3*. Router 3 then NATs to 10.11.14.2 and forwards to 10.11.14.1, which is Router 1. Note that this packet actually travels Wireguard encrypted on the 192.168.3.0 network. Router 1 unwraps the packet back to discover it is from 10.11.14.2 then NATs that to its public IP and forwards to the ISP.
This can be observed by running a traceroute on the desktop. Except of course that traceroute doesn't see what is outside the tunnel of packets that go via VPN. First try this with a numeric IP known to exist on the Internet such as 8.8.8.8. If that works but a DNS name does not work, you need to look at DNS configurations through the network.
Thanks all. We have it working now! This thread will be marked [Solved].
psherman: Agree that the network design is clunky. It wasn't designed from the ground-up but rather came to be over time and certainly could use improvement.
Relative to the firewall setup on Router 3, we just had it wide open when setting up WireGuard to make sure it wasn't working against us. It's since been closed up. Though, as you point out, since it's not exposed to the outside world, we're not too worried about it.
So, here's our problem/solution summary to add to the other excellent commentary from psherman, mk24 and krazeh in case this thread is happened upon by others. We hope it's helpful.
Thanks so much everyone! We are grateful for your help and to the community for providing this forum.
Problem: Getting WireGuard to work between two routers on a separate VLAN. One router is ISP-facing, the other is remote.
Reason for this need: The remote router (Router 3) is located 3 miles away on a set of 5 GHz Ubiquiti dishes in an urban area. Though there is WPA2 encryption on this link, we wanted another layer of security. WireGuard seemed like a great fit rather than OpenVPN.
Our network architecture is sketched in our previous post above (note, mk24 pointed out that it isn't really complete).
Solution:
config interface 'WireGuard'
option proto 'wireguard'
option private_key X1
option listen_port '51820'
list addresses '10.11.14.1/24'
config wireguard_WireGuard
option public_key Y2
option route_allowed_ips '1'
option persistent_keepalive '25'
option description 'Router 3a'
list allowed_ips '10.11.14.2/32'
list allowed_ips '192.168.11.204'
config forwarding
option src 'LAN'
option dest 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'LAN_Port_3'
config forwarding
option src 'LAN_Port_3'
option dest 'WireGuard'
config zone
option name 'WireGuard'
option output 'ACCEPT'
option masq '1'
option input 'ACCEPT'
option forward 'ACCEPT'
list network 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'WAN'
config interface 'WireGuard'
option proto 'wireguard'
option private_key X2
list addresses '10.11.14.2/32'
config wireguard_WireGuard
option description 'Router ISP'
option public_key X2
option endpoint_port '51820'
option persistent_keepalive '25'
option route_allowed_ips '1'
list allowed_ips '0.0.0.0/0'
option endpoint_host '192.168.0.1'
config zone
option name 'WireGuard'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'WireGuard'
option masq '1'
config forwarding
option src 'WireGuard'
option dest 'WAN'
config forwarding
option src 'LAN_Port_1'
option dest 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'LAN_Port_1'
config forwarding
option src 'WAN'
option dest 'WireGuard'
config forwarding
option src 'LAN_Port_2'
option dest 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'LAN_Port_2'
config forwarding
option src 'LAN_Port_3'
option dest 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'LAN_Port_3'
config forwarding
option src 'Wireless'
option dest 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'Wireless'
config forwarding
option src 'LAN_Port_4'
option dest 'WireGuard'
config forwarding
option src 'WireGuard'
option dest 'LAN_Port_4'
Explanation (as we understand it):
The two interfaces have their own IP address: 10.11.14.1 for the ISP Router and 10.11.14.2 for the remote Router 3. We can ssh into the remote Router three from the ISP router on 10.11.14.2 (based on our firewall rules, not shown) when the WireGuard interface is up (ifup WireGuard). And if it's ever down (ifdown WireGuard) we can ssh in on 192.168.3.215.
Because we want all traffic on remote Router 3 to pass through the WireGuard interface, we set list allowed_ips '0.0.0.0/0'. Since that interface needs to talk to the ISP Router, option endpoint_host
is set to 192.168.0.1. The interface is attached to its own firewall zone (labeled WireGuard in our case). Masquerading is set to ensure no routing conflicts with our network architecture.
On the ISP-facing router, list allowed_ips is set to the interface IP of Router 3 (10.11.14.2/32) so that only traffic destined for Router's 3 WireGuard interface is carried over the WireGuard connection. option endpoint_host is not set since it will simply listen for a connection from anywhere and accept it if the keys are right. We also added 192.168.11.204 in list allowed_ips so we can access devices on Router 3a from the ISP Router (prior to WireGuard, this was done with a static route). The interface is attached to its own firewall zone (labeled WireGuard in our case). Masquerading is set to ensure no routing conflicts with our network architecture.
When the WireGuard interface is taken down (which shouldn't really happen, but just in case), the routing table is not rewritten to take that into account, so we added a rule for hotplug.d in /etc/hotplug.d/iface/99-wan-reup on Router 3 (really isn't necessary on the ISP-facing router since only IP was included in the routing table rather than all subnets). This rule brings the WAN interface up again any time any other interface went down so that the routing table is restored to a pre-WireGuard setup.
Router 3 /etc/hotplug.d/iface/99-wan-reup
[ "${ACTION}" = "ifdown" ] && {
ifup wan
logger -t hotplug "Device: ${DEVICE} / Action: ${ACTION} so WAN has been restarted"
}
The endpoint_host is the "outside" of the tunnel, it is the place on the insecure network where encrypted packets will be sent. Though it does technically work to use any IP held by Router 1, it makes the most sense to use 192.168.3.1 as that is the interface directly facing Router 3.
Since your main concern is someone breaking into the wireless link, that .3.0 network should be strictly firewalled so that the only thing allowed into or out of either end is Wireguard packets. Then as you said, someone on the .3.0 network but without the Wireguard keys still cannot make any access to anything useful on your networks. This means necessarily that site 3 cannot have Internet access without Wireguard-- the so called "kill switch" functionality that is inherent in only allowing a route through Wireguard.
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.