Enabling PBR on NAS(wg1) breaks external device access to NAS using wg0. Stumped

Hi everyone.

Just looking for a few comments, can paste as requested.

-I setup Policy based routing to route a few devices over wg1 (PC and NAS, over Mullvad). -This works great.
-I setup Wireguard on OpenWRT to "host" for my mobile devices, wg0. -This works great.
-Set synology NAS firewall to accept the devices tunnled home (through wg0). -Fail, but disabling pbr suddenly allows this to work.
-wg0 and lan are the same zone. wg1 and wan are the same zone.

Putting the NAS on wg1 using policy based routing breaks access that my wg0 devices have to the NAS when tunneling home.

Any thoughts would be appreciated

Without seeing your whole setup just some ides which might be wrong:

Traffic from the wg server comes in via the WAN. But traffic from your NAS is going out via the VPN (wg1), so that will not work.

If this is the problem, you have to make sure there is a rule for PBR that traffic with a destination of wg0 is going out via the WAN and that rule has to come first e.g. begore the rule of the NAS.

Other idea:
Maybe the routing table for wg1 is missing the route for wg0?

As said just some ideas

3 Likes

I think you may be onto something. I'm a bit new to all of this but am eager to learn. What files would help you pinpoint the issue? I'm guessing firewall, pbr, wg settings and possibly network?

Thanks!

I am travelling so have to do things off the top of my head so no guarantee that i can solve this in an instant.
However there are plenty very knowledgeable and helpful forum members who will chime in.

Please copy the output of the following commands and post it here using the "Preformatted text </> " button:

Remember to redact passwords, MAC addresses and any public IP addresses you may have:

cat /etc/config/network
cat /etc/config/firewall
cat /etc/config/pbr
ip ro
ip route show table all
ip ru
1 Like

cat /etc/config/network


config interface 'loopback'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'
        option device 'lo'

config globals 'globals'
        option ula_prefix 'x'

config interface 'lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option delegate '0'
        option device 'br-lan'

config device 'lan_eth1_1_dev'
        option name 'eth1.1'
        option macaddr 'key'
        option ipv6 '0'

config interface 'wan'
        option proto 'dhcp'
        option ipv6 'off'
        option device 'eth0.35'
        option hostname '*'
        option peerdns '0'
        list dns '9.9.9.9'
        list dns '1.1.1.1'

config device 'wan_eth0_35_dev'
        option name 'eth0.35'
        option macaddr 'key'
        option ipv6 '0'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '1 2 3 4 6t'
        option vid '1'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '0t 5'
        option vid '35'

config interface 'wg0'
        option proto 'wireguard'
        option private_key 'key'
        option listen_port 'port'
        list addresses '10.0.1.1/24'

config wireguard_wg0
        list allowed_ips '10.0.1.2/32'
        option endpoint_port 'port'
        option description 'A'
        option persistent_keepalive '25'
        option public_key 'key'

config wireguard_wg0
        option endpoint_port 'port'
        option public_key 'key'
        option description 'B'
        list allowed_ips '10.0.1.3/32'
        option persistent_keepalive '25'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1.1'
        option ipv6 '0'

config interface 'wg1'
        option proto 'wireguard'
        option private_key 'key'
        list addresses '10.5.0.2'

config wireguard_wg1
        option description 'VPN'
        list allowed_ips '0.0.0.0/0'
        option persistent_keepalive '25'
        option public_key 'key'
        option endpoint_host 'x'
        option endpoint_port 'x'

config wireguard_wg0
        list allowed_ips '10.0.1.4/32'
        option endpoint_port 'port'
        option persistent_keepalive '25'
        option description 'T'
        option public_key 'key'

config device
        option name 'wlan0'
        option ipv6 '0'

config device
        option name 'wlan1'
        option ipv6 '0'

config device
        option name 'wg1'
        option ipv6 '0'
        option mtu '1384'

config device
        option name 'eth0'
        option ipv6 '0'

config device
        option name 'eth1'
        option ipv6 '0'

config device
        option name 'wg0'
        option ipv6 '0'

cat /etc/config/firewall

config defaults
        option input 'ACCEPT'
        option output 'ACCEPT'
        option synflood_protect '1'
        option forward 'DROP'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        list network 'lan'
        list network 'wg0'
        option forward 'ACCEPT'

config zone
        option name 'wan'
        option output 'ACCEPT'
        option mtu_fix '1'
        option input 'DROP'
        option forward 'DROP'
        option masq '1'
        list network 'wan'
        list network 'wan6'
        list network 'wg1'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option family 'ipv6'
        list proto 'all'
        option src '*'
        option dest '*'
        option target 'REJECT'
        option name 'IPV6 Block'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'
        option family 'ipv4'

config include
        option path '/etc/firewall.user'

config redirect
        option target 'DNAT'
        option name 'VPN'
        option src 'wan'
        option src_dport 'nah'
        option dest 'lan'
        option dest_ip '192.168.1.1'
        list proto 'udp'

config include 'miniupnpd'
        option type 'script'
        option path '/usr/share/miniupnpd/firewall.include'
        option family 'any'
        option reload '1'

config rule
        option name 'IME1'
        option src '*'
        option src_port '16992'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME2'
        option src '*'
        option src_port '16993'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME3'
        option src '*'
        option src_port '16994'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME4'
        option src '*'
        option src_port '16995'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME5'
        option src '*'
        option src_port '623'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME6'
        option src '*'
        option src_port '664'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME7'
        option src '*'
        option src_port '5900'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME8'
        option src '*'
        option src_port '7578'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME9'
        option src '*'
        option src_port '5120'
        option dest '*'
        option target 'REJECT'

config rule
        option name 'IME10'
        option src '*'
        option src_port '5123'
        option dest '*'
        option target 'REJECT'

config redirect
        option name 'Divert-DNS, port 53'
        option src 'lan'
        option proto 'tcp udp'
        option src_dport '53'
        option dest_port '53'
        option target 'DNAT'

config rule
        option name 'Reject-DoT, port 853'
        option src 'lan'
        option dest 'wan'
        option proto 'tcp udp'
        option dest_port '853'
        option target 'REJECT'

config redirect
        option name 'Divert-DNS, port 5353'
        option src 'lan'
        option proto 'tcp udp'
        option src_dport '5353'
        option dest_port '53'
        option target 'DNAT'

config redirect
        option name 'Divert-DNS, port 9953'
        option src 'lan'
        option proto 'tcp udp'
        option src_dport '9953'
        option dest_port '53'
        option target 'DNAT'

config redirect
        option name 'Divert-DNS, port 1512'
        option src 'lan'
        option proto 'tcp udp'
        option src_dport '1512'
        option dest_port '53'
        option target 'DNAT'

config redirect
        option name 'Divert-DNS, port 54'
        option src 'lan'
        option proto 'tcp udp'
        option src_dport '54'
        option dest_port '53'
        option target 'DNAT'

cat /etc/config/pbr

config policy
	option name 'NAS'
	option src_addr '192.168.1.100'
	option interface 'wg1'

config policy
        option name 'PC'
        option interface 'wg1'
        option src_addr '192.168.1.110'

config policy
        option name 'Phone'
        option src_addr '192.168.1.111'
        option interface 'wg1'

config policy
        option interface 'wan'
        option dest_addr 'epdg.epc.mnc220.mcc302.pub.3gppnetwork.org'
        option name 'Koodo'
        option src_addr '192.168.1.111'
        option proto 'udp'

config policy
        option interface 'wan'
        option name 'Virgin'
        option dest_addr 'epdg.epc.mnc610.mcc302.pub.3gppnetwork.org'
        option src_addr '192.168.1.111'
        option proto 'udp'

config pbr 'config'
        option strict_enforcement '1'
        option src_ipset '0'
        option dest_ipset '0'
        option resolver_set 'dnsmasq.ipset'
        list ignored_interface 'vpnserver wgserver'
        option boot_timeout '30'
        option rule_create_option 'append'
        option procd_reload_delay '1'
        option webui_chain_column '0'
        option webui_show_ignore_target '0'
        option webui_sorting '1'
        list webui_supported_protocol 'tcp'
        list webui_supported_protocol 'udp'
        list webui_supported_protocol 'tcp udp'
        list webui_supported_protocol 'icmp'
        list webui_supported_protocol 'all'
        option verbosity '1'
        option webui_enable_column '1'
        option webui_protocol_column '1'
        option ipv6_enabled '0'
        option enabled '1'

config include
        option path '/etc/pbr.netflix.user'
        option enabled '0'

config include
        option path '/etc/pbr.aws.user'
        option enabled '0'

io ro

default via IP1 dev eth0.35 proto static src IP2
10.0.1.0/24 dev wg0 proto kernel scope link src 10.0.1.1
IP1/22 dev eth0.35 proto kernel scope link src IP2
IP3 via IP1 dev eth0.35 proto static
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1

ip route show table all

default via IP3 dev eth0.35 table pbr_wan
192.168.1.0/24 dev br-lan table pbr_wan proto kernel scope link src 192.168.1.1
default via 10.0.1.1 dev wg0 table pbr_wg0
192.168.1.0/24 dev br-lan table pbr_wg0 proto kernel scope link src 192.168.1.1
default via 10.5.0.2 dev wg1 table pbr_wg1
192.168.1.0/24 dev br-lan table pbr_wg1 proto kernel scope link src 192.168.1.1
default via IP3 dev eth0.35 proto static src IP4
10.0.1.0/24 dev wg0 proto kernel scope link src 10.0.1.1
IP1/22 dev eth0.35 proto kernel scope link src IP4
IP2 via IP3 dev eth0.35 proto static
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1
broadcast 10.0.1.0 dev wg0 table local proto kernel scope link src 10.0.1.1
local 10.0.1.1 dev wg0 table local proto kernel scope host src 10.0.1.1
broadcast 10.0.1.255 dev wg0 table local proto kernel scope link src 10.0.1.1
local 10.5.0.2 dev wg1 table local proto kernel scope host src 10.5.0.2
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast IP1 dev eth0.35 table local proto kernel scope link src IP4
local IP4 dev eth0.35 table local proto kernel scope host src IP4
broadcast 142.161.147.255 dev eth0.35 table local proto kernel scope link src IP4
broadcast 192.168.1.0 dev br-lan table local proto kernel scope link src 192.168.1.1
local 192.168.1.1 dev br-lan table local proto kernel scope host src 192.168.1.1
broadcast 192.168.1.255 dev br-lan table local proto kernel scope link src 192.168.1.1
unreachable iphere::/48 dev lo proto static metric 2147483647 pref medium
local ::1 dev lo table local proto kernel metric 0 pref medium

ip ru

0:      from all lookup local
30000:  from all fwmark 0x10000/0xff0000 lookup pbr_wan
30001:  from all fwmark 0x20000/0xff0000 lookup pbr_wg0
30002:  from all fwmark 0x30000/0xff0000 lookup pbr_wg1
32766:  from all lookup main
32767:  from all lookup default

Thanks so much for looking into this, hopefully not too much of a pain!

I believe if you insert this at the very top of pbr config (and restart pbr) it'll help.

config policy
	option name 'Ignore Local Requests'
	option interface 'ignore'
	option dest_addr '10.0.1.0/24 192.168.1.0/24'

If it doesn't, do whatever @egc suggests.

1 Like

Thanks, I will give it a shot.
Do you mean '192.168.1.0/24' and not '192.168.21.0/24'?

Yes, I corrected my earlier post.

1 Like

Wow, that instantly fixed it! Thank you so much. Backing up my router config.... right now. Can you explain what this policy does (It appears to ignore on the wg0 and lan addresses) but how does that fix the issue?

Stan is the man :slight_smile:

My guess, it will not touch traffic to the wg server (wg0) so that it will go out of the WAN?

@stangri, adding wg0 as ignored interface will this add the route to the server to pbr_wg1, which maybe also might work?

1 Like

It makes sure that when your NAS/local service replies to a request from LAN/device connected to your router via WG server, the reply is not routed via VPN.

I believe it would. I should implement a logic in pbr to automatically exclude wg servers based on the listen_port option, don't know when I'll get to it tho.

2 Likes

Thanks for the clarification and all of the help!

Thanks @stangri, when I made a PBR implementation for another third party firmware I simply copied all local routes (i.e. everything except default and 0.0.0.0/1, 128.0.0.0/1) to all tables.
It can have its drawbacks but for a simple implementation it seems to work well in practice.

Your implementation is much more sophisticated though :slight_smile:

I wish I could claim credit for that, I believe it was recommended thru user feedback, which led to implementation of ignore target.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.