That's massively simplified things (and hasn't broken anything). I now have three zones: wan, lan and guest. The Mullvad wg interface is in the wan zone; the domestic wg interface in the lan zone.
I can connect to the domestic wg interface, and clients appear from my ISP's static IP address. However, I still can't connect to 192.168.1.1/24 clients when in the 10.0.0.1/24 subnet. I can ping the router at 192.168.1.1 (from, say, 10.0.0.2), but that's it. Perhaps I'm missing a firewall rule, still?
I tried removing the 0x10000 Firewall Mark from my domestic wg interface, but, when I did, I found that there was no internet access for connected clients.
Here's my (much simpler) firewall config
config defaults
option syn_flood '1'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option network 'lan streaming wgserver'
config zone
option name 'wan'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
option input 'REJECT'
option network 'wan wan6 wg'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option src_ip 'fc00::/6'
option dest_ip 'fc00::/6'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config include
option path '/etc/firewall.user'
config zone 'guest'
option name 'guest'
option output 'ACCEPT'
option network 'guest'
option input 'REJECT'
option forward 'REJECT'
config rule 'guest_dhcp'
option name 'guest_DHCP'
option src 'guest'
option target 'ACCEPT'
option proto 'udp'
option dest_port '67-68'
config rule 'guest_dns'
option name 'guest_DNS'
option src 'guest'
option target 'ACCEPT'
option proto 'tcpudp'
option dest_port '53'
config rule
option src '*'
option target 'ACCEPT'
option proto 'udp'
option dest_port '1234'
option name 'Allow-Wireguard-Inbound'
config forwarding
option dest 'wan'
option src 'lan'
config forwarding
option dest 'wan'
option src 'guest'
Thanks, again. I'm going nuts trying to figure this out, so I really appreciate your assistance.
No dice, I'm afraid: same result. I'm minded to point the finger at Policy-Based Routing: perhaps the response isn't going back over the wgserver interface, but over something else (maybe out via Mullvad?) so that the response is never received?
Can you explain how you "discovered" that you had to set a firewall mark?
I see you followed my advice from that post I made back in July. Try setting in VPR advanced configuration settings set "Append local IP Tables rules" to "! -d 10.0.0.1/24". also make sure that wg interface for the wireguard server is ignored as well.
Also the only reason fwmark was needed was because I was previously experimenting with a setup that had Azire VPN as a wireguard client alongside the WG server. As I no longer am using azire as a client I can't say for sure you will need the fwmark 0x10000 set with the above settings I mentioned. Anyways I would try with and without the fwmark set.
I, too, am running both a Wireguard client (Mullvad, in my case) and server. It looks like the fwmark is required, along with the route that @lleachii suggested:
I've done as you suggested. Under VPR advanced settings, I added wgserver to the list of Ignored Interfaces, and added the destination exclusion you posted. It's still not working, but I definitely think that PBR is the issue here. Here's what I'm trying to achieve:
My streaming device is in 192.168.1.1/24 and has a static IP assigned to it. I want this to go over my ISP; e.g. due to Netflix spotting my VPN connection
My HDHomerun Connect is also in 192.168.1.1/24 and has a static IP assigned to it. I don't mind whether this goes through Mullvad or my ISP, but I do need clients in 10.0.0.1/24 to be able to reach it
10.0.0.1/24 should also pick up my ISP IP (since the goal of this exercise is to be able to appear from my ISP IP when overseas)
Here's my PBR config (with comments). Interestingly, when I remove the last rule (10.0.0.1/24 over the WAN), I can reach clients in 192.168.1.1/24 just fine. But, when I do this, clients connected to 10.0.0.1/24 end up with Mullvad's IP rather than my ISP's:
/etc/config/vpn-policy-routing
# route TV over ISP IP to not bother Netflix blockers
config policy
option name 'TV'
option local_address '192.168.1.100'
option interface 'wan'
option proto 'tcp'
option chain 'PREROUTING'
# this device works better when going over my ISP
config policy
option chain 'PREROUTING'
option interface 'wan'
option name 'Hello'
option local_address '192.168.1.102'
option proto 'tcp'
# This entire subnet goes over my ISP because sometime's it's useful
config policy
option proto 'tcp'
option chain 'PREROUTING'
option interface 'wan'
option name 'Streaming'
option local_address '192.168.3.1/24'
config vpn-policy-routing 'config'
option verbosity '2'
option ipv6_enabled '0'
option dnsmasq_enabled '0'
option strict_enforcement '1'
option boot_timeout '30'
option ipset_enabled '1'
list ignored_interface 'wgserver'
option enabled '1'
option append_local_rules '! -d 10.0.0.1/24'
# clients connected to my wireguard server should pick up my ISP IP
config policy
option proto 'tcp'
option chain 'PREROUTING'
option interface 'wan'
option name 'wgserver'
option local_address '10.0.0.0/24'
This can't be impossible, but I know there's a gap in my knowledge here that's preventing me from resolving it.
...and, I think this is the problem. When I add 10.0.0.1/24 to the PBR: a client in 10.0.0.1/24 tries to connect to a LAN client in 192.168.1.1/24, but the response goes over the WAN rather than remaining local. So, do I need to add an exclusion; something like "! -d 192.168.1.1/24"?
I can't help; because for some reason, you aren't showing all your routes at once. And for some reason, you're putting routes in 2 places. That's quite confusing.
Sorry, @lleachii; you're absolutely correct. It is getting a bit confusing. During the week, I'll clear all my settings and start again from scratch and try to keep everything as simple as possible. I can then post back if things still aren't working.
Thanks for your assistance and sorry for the confusion.
I have pretty much the exact same setup as you. The only difference being I no longer have my a wireguard interface setup as a client. My provider is PIA over OpenvVPN interface.
I think what is happening is when you setup mullvad vpn as a wireguard interface it changes your routers default routing table. In OpenVPN you would make sure the option route_nopull '1' was set to keep WAN as the default route. In Wireguard the option route_allowed_ips '0' should do the same but I've found that the wireguard interface would still change the default route, this is why when you delete your VPR policy
# clients connected to my wireguard server should pick up my ISP IP
config policy
option proto 'tcp'
option chain 'PREROUTING'
option interface 'wan'
option name 'wgserver'
option local_address '10.0.0.0/24'
your clients end up going over the default route (mullvad) because you no longer have a VPR policy telling them to go over the WAN.
You can try setting this as a VPR policy
# clients connected to my wireguard server should pick up my ISP IP
config policy
option proto 'tcp'
option chain 'PREROUTING'
option interface 'wan'
option name 'LAN routing'
option local_address '192.168.1.0/24 10.0.0.0/24'
This makes sure both your LAN and wgserver clients are going over your WAN. Also this is a very useful command that will help in troubleshooting
Thanks, @jvquintero1021. I didn't know about route_allowed_ips, so that's handy to know.
I've started from scratch. So far, I've got the basics working: WireGuard server; no Mullvad WireGuard client; no PBR. So far, so good: I can connect to my WireGuard server, clients appear from my ISP's IP address, and I can access clients in my lan zone.
/etc/config/firewall
config defaults
option syn_flood '1'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option network 'Streaming lan wgserver'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
option network 'wan wan6'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option src_ip 'fc00::/6'
option dest_ip 'fc00::/6'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config include
option path '/etc/firewall.user'
config zone
option forward 'REJECT'
option output 'ACCEPT'
option name 'guest'
option input 'REJECT'
option network 'Guest'
config forwarding
option dest 'wan'
option src 'guest'
config rule
option target 'ACCEPT'
option proto 'udp'
option dest_port '67-68'
option name 'guest_dhcp'
option src 'guest'
config rule
option enabled '1'
option target 'ACCEPT'
option proto 'tcp udp'
option dest_port '53'
option name 'guest_dns'
option src 'guest'
config rule
option src '*'
option target 'ACCEPT'
option proto 'udp'
option dest_port '52000'
option name 'Allow-Wireguard-Inbound'
Sorry, @lleachii, you're right: I was aware of its existence as you had highlighted it to me earlier. I wasn't clear on its purpose, though.
I'm very grateful for the help you and @jvquintero1021 have provided. The good news is that I think
I've got it all working now with a client and server running side-by-side, and with the requisite lan access. I'll do some more testing tonight when I'm back home and then post my configs here in case they're useful to somebody else.
Yep, looks like this is all working now. It seems the important detail is, indeed, to ensure route_allowed_ips is '0' to keep the WAN as the default route.
I was scratching my head today trying to figure out why all-of-a-sudden, I couldn't reach clients in my LAN through the WireGuard tunnel. It was because I was sitting on a 192.168.1.1/24 subnet at the time, so there was a clash when trying to reach clients on my home 192.168.1.1/24 subnet. So, I've changed my LAN subnet at home to something different to reduce the chances of a clash.
Anyway, for posterity, here are my configs in case they're useful for somebody else: