I got PBR (Version 1.1.8-r2) working, but can't get my wireguard client to get out to the internet.
Can't ping anything via SSH. (ping -I wgc1 8.8.8.8) my openvpn clients work with ping though.
Have unchecked "Use Default Gateway" (0)
Can see Data-Received/Transmitted count changing value.
Have tried most things but haven't managed to figure out what's causing it.
Thanks
I can't reach internet with my wireguard client.
Let's take a look at the config...
Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </>
" button:
Remember to redact passwords, MAC addresses and any public IP addresses you may have:
ubus call system board
cat /etc/config/network
cat /etc/config/firewall
wg show
And please also show the remote peer's config.
My setup is wireguard client to connect to my VPN-provider (ovpn.com).
I don't know theirs config.
Client seems to connected but I can't use it to connect to internet through PBR (Version 1.1.8-r2) however my two VPN connections work and can be used through PBR
root@DEFCON:~# ubus call system board
{
"kernel": "5.15.167",
"hostname": "DEFCON",
"system": "ARMv8 Processor rev 4",
"model": "GL.iNet GL-MT6000",
"board_name": "glinet,gl-mt6000",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "23.05.5",
"revision": "r24106-10cc5fcd00",
"target": "mediatek/filogic",
"description": "OpenWrt 23.05.5 r24106-10cc5fcd00"
}
}
root@DEFCON:~# cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fda1:cc83:50bb::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'lan1'
list ports 'lan2'
list ports 'lan3'
list ports 'lan4'
list ports 'lan5'
config device
option name 'lan1'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan2'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan3'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan4'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan5'
option macaddr '94:83:c4:a5:ff:7f'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'
config device
option name 'eth1'
option macaddr '94:83:c4:a5:ff:7d'
config interface 'wan'
option device 'eth1'
option proto 'dhcp'
option dns_metric '20'
config interface 'wan6'
option device 'eth1'
option proto 'dhcpv6'
config device
option type 'bridge'
option name 'br-guest'
option bridge_empty '1'
config interface 'guest'
option proto 'static'
option device 'br-guest'
option ipaddr '192.168.2.1'
option netmask '255.255.255.0'
option gateway '192.168.1.1'
config interface 'OpenVPN'
option proto 'none'
option device 'tun0'
config interface 'OpenVPN2'
option proto 'none'
option device 'tun1'
option delegate '0'
config interface 'wgc1'
option proto 'wireguard'
option private_key '<private key>'
option mtu '1420'
list addresses '172.19.65.19/32'
list addresses 'fd00:0000:1337:cafe:1111:1111:c9d8:674a/128'
option dns_metric '10'
list dns '46.227.67.134'
list dns '192.165.9.158'
list dns '2a07:a880:4601:10f0:cd45::1'
list dns '2001:67c:750:1:cafe:cd45::1'
option defaultroute '0'
config wireguard_wgc1
option description 'vpn62-sthlm'
option public_key '<public key>'
option endpoint_host 'vpn62.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
**option disabled '1'**
config wireguard_wgc1
option description 'vpn23-sthlm'
option public_key '<public key>'
option endpoint_host 'vpn23.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
**option disabled '1'**
config wireguard_wgc1
option description 'vpn09-sthlm'
option public_key '<public key>'
option endpoint_host 'vpn09.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
**option disabled '1'**
config wireguard_wgc1
option description 'vpn15-sthlm'
option public_key '<public key>'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
option endpoint_host 'vpn15.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
root@DEFCON:~# cat /etc/config/firewall
config defaults
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option synflood_protect '1'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
option mtu_fix '1'
list network 'lan'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
list device 'tun0'
list device 'tun1'
list network 'OpenVPN'
list network 'OpenVPN2'
list network 'wan'
list network 'wan6'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config zone
option name 'guest'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
list network 'guest'
config forwarding
option src 'guest'
option dest 'wan'
config rule
option name 'Allow-DNS-Guest'
option src 'guest'
option dest_port '53'
option target 'ACCEPT'
config rule
option name 'Allow-DHCP-Guest'
list proto 'udp'
option src 'guest'
option dest_port '67'
option target 'ACCEPT'
config zone
option name 'wgc_fw'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
list network 'wgc1'
config forwarding
option src 'lan'
option dest 'wgc_fw'
config include 'pbr'
option fw4_compatible '1'
option type 'script'
option path '/usr/share/pbr/firewall.include'
config forwarding
option src 'guest'
option dest 'wgc_fw'
root@DEFCON:~# wg show
interface: wgc1
public key: <public key>
private key: (hidden)
listening port: 40524
peer: <peer key>
endpoint: 217.64.148.56:9929
allowed ips: 0.0.0.0/0, ::/0
latest handshake: 50 seconds ago
transfer: 47.21 KiB received, 529.23 KiB sent
persistent keepalive: every 25 seconds
First some sanitizing
Remove:
list device 'tun0'
list device 'tun1'
From
Add: list network 'wgc1'
Remove:
and
Remove the wrong gateway: option gateway '192.168.1.1'
from:
Remove option defaultroute '0'
you are not enabling Route Allowed IPs so it is redundant:
Remove option mtu_fix '1'
, it is not needed on the lan side:
Reboot and test again if it does not work then, Please connect to your OpenWRT device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
Remember to redact keys, passwords, MAC addresses and any public IP addresses you may have:
cat /etc/config/network
cat /etc/config/dhcp
cat /etc/config/firewall
ip route show
ip route show table all
ip rule show
wg show
cat /etc/config/pbr
service pbr status
cat /var/run/pbr.nft
nft -c -f /var/run/pbr.nft
I have tried making the changes and rebooting, still wireguard client doesn't work.
(seems to connect)
Found a setting under "DHCP & DNS" resolv & host files.
Should that point to /tmp/resolv.conf.wg ?
Will come later copy the output of the following commands:
No not really where does that come from?
Have you been using a hotplug script?
That is not recommended if you are using PBR already.
It seems you have been tinkering a lot you might consider starting fresh with resetting to defaults.
I think it comes from when I installed "wg-installer-server" and uninstalling don't removed all of it.
Should that be blank or point to ex: "/tmp/resolv.conf.d/resolv.conf.auto" or "/tmp/resolv.conf"
Just leave it blank, but I would recommended you reset to defaults and start fresh, who knows what other gremlins are present.
Well after after I removed it, it's come back again.
Must come from PBR.
I also using: " ovpn-update-resolv-9" and " 98-wg-update-resolv-4" to prevent DNS leak.
Those are the hotplug scripts I was talking about
They are not needed when using PBR as PBR's DNS policies should take care of that besides those scripts are mutually exclusive because you either use DNS via OpenVPN or via WireGuard but cannot have both.
P.S. regarding the long list of settings which could be improved and those hotplug scripts and who knows what else, a fresh start is not a bad idea.
Might consider sticking with only WireGuard, OpenVPN can have a mind of its own as it setups and pulls down its own interface
I have test without those hotplug scripts and I get DNS leaks.
Removed script for wireguard and tested with script for vpn.
Have you tested PBR alone and have dns working without leaks?
Using DNS through vpn-tunnel.
When I tested I not get it to work. Any suggestion about that?
I need both wireguard and vpn clients.
VPN has envionment varaibles i need for my control script.
@egc
What environments variables are we talking about and to what purpose?
For that you use PBR's DNS Policies or DNSMasq option 6
it does not matter what DNSMasq uses (a DNS server via the tunnel or via the WAN) you will always get some form of DNS leak, if you let DNSMasq use a DNS server via the tunnel then your clients which are using the WAN will have the wrong DNS, banks, amazon, netflix will not work any more, if you let DNSMasq use a DNS via the WAN then your lan Clients using the VPN will have the same problem.
What you need is split DNS, that is what o.a. PBR's DNS Policies or DNSMasq option 6 can provide:
Happy New year to all OpenWRT devs, users, other aficionados and everybody else
P.S. I am away for the remainder of the day
Thank you for your patience...
Happy New Year!
@egc
Here is output of the following commands.
root@DEFCON:~# ping -I eth1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=116 time=2.889 ms
64 bytes from 8.8.8.8: seq=1 ttl=116 time=2.828 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 2.828/2.858/2.889 ms
root@DEFCON:~# ping -I tun0 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=117 time=3.285 ms
64 bytes from 8.8.8.8: seq=1 ttl=117 time=3.093 ms
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 3.093/3.186/3.285 ms
root@DEFCON:~# ping -I tun1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=117 time=14.370 ms
64 bytes from 8.8.8.8: seq=1 ttl=117 time=14.292 ms
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 14.212/14.291/14.370 ms
**root@DEFCON:~# ping -I wgc1 8.8.8.8**
**PING 8.8.8.8 (8.8.8.8): 56 data bytes**
**--- 8.8.8.8 ping statistics ---**
**6 packets transmitted, 0 packets received, 100% packet loss**
root@DEFCON:~# cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fda1:cc83:50bb::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'lan1'
list ports 'lan2'
list ports 'lan3'
list ports 'lan4'
list ports 'lan5'
config device
option name 'lan1'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan2'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan3'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan4'
option macaddr '94:83:c4:a5:ff:7f'
config device
option name 'lan5'
option macaddr '94:83:c4:a5:ff:7f'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'
config device
option name 'eth1'
option macaddr '94:83:c4:a5:ff:7d'
config interface 'wan'
option device 'eth1'
option proto 'dhcp'
option dns_metric '20'
config interface 'wan6'
option device 'eth1'
option proto 'dhcpv6'
config device
option type 'bridge'
option name 'br-guest'
option bridge_empty '1'
config interface 'guest'
option proto 'static'
option device 'br-guest'
option ipaddr '192.168.2.1'
option netmask '255.255.255.0'
config interface 'OpenVPN'
option proto 'none'
option device 'tun0'
config interface 'OpenVPN2'
option proto 'none'
option device 'tun1'
option delegate '0'
config interface 'wgc1'
option proto 'wireguard'
option private_key '<private key>'
option mtu '1420'
list addresses '172.19.65.19/32'
list addresses 'fd00:0000:1337:cafe:1111:1111:c9d8:674a/128'
option dns_metric '10'
list dns '46.227.67.134'
list dns '192.165.9.158'
list dns '2a07:a880:4601:10f0:cd45::1'
list dns '2001:67c:750:1:cafe:cd45::1'
config wireguard_wgc1
option description 'vpn62-sthlm'
option public_key '<public key>'
option endpoint_host 'vpn62.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
option disabled '1'
config wireguard_wgc1
option description 'vpn23-sthlm'
option public_key '<public key>'
option endpoint_host 'vpn23.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
option disabled '1'
config wireguard_wgc1
option description 'vpn09-sthlm'
option public_key '<public key>'
option endpoint_host 'vpn09.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
option disabled '1'
config wireguard_wgc1
option description 'vpn15-sthlm'
option public_key '<public key>'
list allowed_ips '0.0.0.0/0'
list allowed_ips '::/0'
option endpoint_host 'vpn15.prd.kista.ovpn.com'
option endpoint_port '9929'
option persistent_keepalive '25'
root@DEFCON:~# cat /etc/config/dhcp
config dnsmasq
option domainneeded '1'
option localise_queries '1'
option rebind_protection '1'
option rebind_localhost '1'
option local '/lan/'
option domain 'home.lan'
option expandhosts '1'
option cachesize '1000'
option authoritative '1'
option readethers '1'
option leasefile '/tmp/dhcp.leases'
option localservice '1'
option ednspacket_max '1280'
config dhcp 'lan'
option interface 'lan'
option start '100'
option limit '150'
option leasetime '12h'
option dhcpv4 'server'
option dhcpv6 'server'
option ra 'server'
option ra_slaac '1'
list ra_flags 'managed-config'
list ra_flags 'other-config'
option force '1'
config dhcp 'wan'
option interface 'wan'
option ignore '1'
config odhcpd 'odhcpd'
option maindhcp '0'
option leasefile '/tmp/hosts/odhcpd'
option leasetrigger '/usr/sbin/odhcpd-update'
option loglevel '4'
config dhcp 'guest'
option interface 'guest'
option start '100'
option limit '150'
option leasetime '12h'
root@DEFCON:~# cat /etc/config/firewall
config defaults
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option synflood_protect '1'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
list network 'OpenVPN'
list network 'OpenVPN2'
list network 'wgc1'
list network 'wan'
list network 'wan6'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config zone
option name 'guest'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
list network 'guest'
config forwarding
option src 'guest'
option dest 'wan'
config rule
option name 'Allow-DNS-Guest'
option src 'guest'
option dest_port '53'
option target 'ACCEPT'
config rule
option name 'Allow-DHCP-Guest'
list proto 'udp'
option src 'guest'
option dest_port '67'
option target 'ACCEPT'
config include 'pbr'
option fw4_compatible '1'
option type 'script'
option path '/usr/share/pbr/firewall.include'
root@DEFCON:~# ip route show
0.0.0.0/1 via 10.128.0.1 dev tun0
default via 100.69.0.1 dev eth1 proto static src 100.69.14.38
10.128.0.0/22 dev tun0 proto kernel scope link src 10.128.0.85
10.129.0.0/22 dev tun1 proto kernel scope link src 10.129.3.92
46.227.67.134 dev tun0 scope link
100.69.0.0/17 dev eth1 proto kernel scope link src 100.69.14.38
128.0.0.0/1 via 10.128.0.1 dev tun0
192.165.9.158 dev tun0 scope link
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest proto kernel scope link src 192.168.2.1
217.64.148.50 via 100.69.0.1 dev eth1
217.64.148.56 via 100.69.0.1 dev eth1 proto static
root@DEFCON:~# ip route show
0.0.0.0/1 via 10.128.0.1 dev tun0
default via 100.69.0.1 dev eth1 proto static src 100.69.14.38
10.128.0.0/22 dev tun0 proto kernel scope link src 10.128.0.85
10.129.0.0/22 dev tun1 proto kernel scope link src 10.129.3.92
46.227.67.134 dev tun0 scope link
100.69.0.0/17 dev eth1 proto kernel scope link src 100.69.14.38
128.0.0.0/1 via 10.128.0.1 dev tun0
192.165.9.158 dev tun0 scope link
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest proto kernel scope link src 192.168.2.1
217.64.148.50 via 100.69.0.1 dev eth1
217.64.148.56 via 100.69.0.1 dev eth1 proto static
root@DEFCON:~# clear screen
root@DEFCON:~# ip route show table all
default via 100.69.0.1 dev eth1 table pbr_wan
192.168.1.0/24 dev br-lan table pbr_wan proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest table pbr_wan proto kernel scope link src 192.168.2.1
default via 10.128.0.85 dev tun0 table pbr_OpenVPN
192.168.1.0/24 dev br-lan table pbr_OpenVPN proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest table pbr_OpenVPN proto kernel scope link src 192.168.2.1
default via 10.129.3.92 dev tun1 table pbr_OpenVPN2
192.168.1.0/24 dev br-lan table pbr_OpenVPN2 proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest table pbr_OpenVPN2 proto kernel scope link src 192.168.2.1
default via 172.19.65.19 dev wgc1 table pbr_wgc1
192.168.1.0/24 dev br-lan table pbr_wgc1 proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest table pbr_wgc1 proto kernel scope link src 192.168.2.1
0.0.0.0/1 via 10.128.0.1 dev tun0
default via 100.69.0.1 dev eth1 proto static src 100.69.14.38
10.128.0.0/22 dev tun0 proto kernel scope link src 10.128.0.85
10.129.0.0/22 dev tun1 proto kernel scope link src 10.129.3.92
46.227.67.134 dev tun0 scope link
100.69.0.0/17 dev eth1 proto kernel scope link src 100.69.14.38
128.0.0.0/1 via 10.128.0.1 dev tun0
192.165.9.158 dev tun0 scope link
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev br-guest proto kernel scope link src 192.168.2.1
217.64.148.50 via 100.69.0.1 dev eth1
217.64.148.56 via 100.69.0.1 dev eth1 proto static
local 10.128.0.85 dev tun0 table local proto kernel scope host src 10.128.0.85
broadcast 10.128.3.255 dev tun0 table local proto kernel scope link src 10.128.0.85
local 10.129.3.92 dev tun1 table local proto kernel scope host src 10.129.3.92
broadcast 10.129.3.255 dev tun1 table local proto kernel scope link src 10.129.3.92
local 100.69.14.38 dev eth1 table local proto kernel scope host src 100.69.14.38
broadcast 100.69.127.255 dev eth1 table local proto kernel scope link src 100.69.14.38
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
local 172.19.65.19 dev wgc1 table local proto kernel scope host src 172.19.65.19
local 192.168.1.1 dev br-lan table local proto kernel scope host src 192.168.1.1
broadcast 192.168.1.255 dev br-lan table local proto kernel scope link src 192.168.1.1
local 192.168.2.1 dev br-guest table local proto kernel scope host src 192.168.2.1
broadcast 192.168.2.255 dev br-guest table local proto kernel scope link src 192.168.2.1
default from 2001:9b0:41::c934:936b via fe80::a67b:2cff:fe9e:ea01 dev eth1 proto static metric 512 pref medium
default from 2001:9b1:df0:e100::/56 via fe80::a67b:2cff:fe9e:ea01 dev eth1 proto static metric 512 pref medium
2001:67c:750:1:cafe:cd45:0:1 dev tun0 metric 1024 pref medium
2001:9b1:df0:e100::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2001:9b1:df0:e100::/56 dev lo proto static metric 2147483647 pref medium
2a07:a880:4601:1051::/64 dev tun0 proto kernel metric 256 pref medium
2a07:a880:4601:10f0:cd45::1 dev tun0 metric 1024 pref medium
2a07:a880:4604:1042::/64 dev tun1 proto kernel metric 256 pref medium
2000::/3 dev tun1 metric 1024 pref medium
fd00:0:1337:cafe:1111:1111:c9d8:674a dev wgc1 proto kernel metric 256 pref medium
fda1:cc83:50bb::/64 dev br-lan proto static metric 1024 pref medium
unreachable fda1:cc83:50bb::/48 dev lo proto static metric 2147483647 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev phy0-ap0 proto kernel metric 256 pref medium
fe80::/64 dev br-guest proto kernel metric 256 pref medium
fe80::/64 dev phy0-ap1 proto kernel metric 256 pref medium
fe80::/64 dev tun0 proto kernel metric 256 pref medium
fe80::/64 dev tun1 proto kernel metric 256 pref medium
fe80::/64 dev phy1-ap0 proto kernel metric 256 pref medium
local ::1 dev lo table local proto kernel metric 0 pref medium
local 2001:9b0:41::c934:936b dev eth1 table local proto kernel metric 0 pref medium
anycast 2001:9b1:df0:e100:: dev br-lan table local proto kernel metric 0 pref medium
local 2001:9b1:df0:e100::1 dev br-lan table local proto kernel metric 0 pref medium
anycast 2a07:a880:4601:1051:: dev tun0 table local proto kernel metric 0 pref medium
local 2a07:a880:4601:1051:20dc:133a:768f:ae0d dev tun0 table local proto kernel metric 0 pref medium
anycast 2a07:a880:4604:1042:: dev tun1 table local proto kernel metric 0 pref medium
local 2a07:a880:4604:1042:cd5e:8e82:d35e:2055 dev tun1 table local proto kernel metric 0 pref medium
local fd00:0:1337:cafe:1111:1111:c9d8:674a dev wgc1 table local proto kernel metric 0 pref medium
anycast fda1:cc83:50bb:: dev br-lan table local proto kernel metric 0 pref medium
local fda1:cc83:50bb::1 dev br-lan table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0 table local proto kernel metric 0 pref medium
anycast fe80:: dev br-lan table local proto kernel metric 0 pref medium
anycast fe80:: dev eth1 table local proto kernel metric 0 pref medium
anycast fe80:: dev phy0-ap0 table local proto kernel metric 0 pref medium
anycast fe80:: dev br-guest table local proto kernel metric 0 pref medium
anycast fe80:: dev phy0-ap1 table local proto kernel metric 0 pref medium
anycast fe80:: dev tun0 table local proto kernel metric 0 pref medium
anycast fe80:: dev tun1 table local proto kernel metric 0 pref medium
anycast fe80:: dev phy1-ap0 table local proto kernel metric 0 pref medium
local fe80::2444:3e7a:f0be:4fa6 dev tun0 table local proto kernel metric 0 pref medium
local fe80::483c:19ff:fecf:8dc0 dev eth0 table local proto kernel metric 0 pref medium
local fe80::9483:c4ff:fea5:ff80 dev br-guest table local proto kernel metric 0 pref medium
local fe80::9483:c4ff:fea5:ff80 dev phy0-ap1 table local proto kernel metric 0 pref medium
local fe80::9683:c4ff:fea5:ff7d dev eth1 table local proto kernel metric 0 pref medium
local fe80::9683:c4ff:fea5:ff7f dev br-lan table local proto kernel metric 0 pref medium
local fe80::9683:c4ff:fea5:ff80 dev phy0-ap0 table local proto kernel metric 0 pref medium
local fe80::9683:c4ff:fea5:ff81 dev phy1-ap0 table local proto kernel metric 0 pref medium
local fe80::d1a6:3dbd:178a:19b4 dev tun1 table local proto kernel metric 0 pref medium
multicast ff00::/8 dev eth0 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev br-lan table local proto kernel metric 256 pref medium
multicast ff00::/8 dev eth1 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev phy0-ap0 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev br-guest table local proto kernel metric 256 pref medium
multicast ff00::/8 dev phy0-ap1 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev tun0 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev tun1 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev phy1-ap0 table local proto kernel metric 256 pref medium
multicast ff00::/8 dev wgc1 table local proto kernel metric 256 pref medium
root@DEFCON:~# ip rule show
0: from all lookup local
29994: from all fwmark 0x40000/0xff0000 lookup pbr_wgc1
29996: from all fwmark 0x30000/0xff0000 lookup pbr_OpenVPN2
29998: from all fwmark 0x20000/0xff0000 lookup pbr_OpenVPN
30000: from all fwmark 0x10000/0xff0000 lookup pbr_wan
32766: from all lookup main
32767: from all lookup default
root@DEFCON:~# wg show
interface: wgc1
public key: <public key>
private key: (hidden)
listening port: 59342
peer: <peer key>
endpoint: 217.64.148.56:9929
allowed ips: 0.0.0.0/0, ::/0
latest handshake: 22 seconds ago
transfer: 35.01 KiB received, 130.82 KiB sent
persistent keepalive: every 25 seconds
root@DEFCON:~# cat /etc/config/pbr
config pbr 'config'
option enabled '1'
option verbosity '2'
option strict_enforcement '1'
option resolver_set 'none'
list resolver_instance '*'
option ipv6_enabled '0'
list ignored_interface 'vpnserver'
option boot_timeout '30'
option rule_create_option 'add'
option procd_boot_delay '15'
option procd_reload_delay '1'
option webui_show_ignore_target '0'
option nft_rule_counter '0'
option nft_set_auto_merge '1'
option nft_set_counter '0'
option nft_set_flags_interval '1'
option nft_set_flags_timeout '0'
option nft_set_policy 'performance'
list webui_supported_protocol 'all'
list webui_supported_protocol 'tcp'
list webui_supported_protocol 'udp'
list webui_supported_protocol 'tcp udp'
list webui_supported_protocol 'icmp'
config include
option path '/usr/share/pbr/pbr.user.aws'
option enabled '0'
config include
option path '/usr/share/pbr/pbr.user.netflix'
option enabled '0'
config dns_policy
option name 'Johan-PC-lan'
option src_addr '192.168.1.133'
option dest_dns '46.227.67.134'
option enabled '0'
config dns_policy
option name 'Johans-S24'
option src_addr 'A4:A4:90:89:52:18'
option dest_dns '46.227.67.134'
option enabled '0'
config dns_policy
option name 'Paddan'
option src_addr '192.168.1.151'
option dest_dns '213.80.98.2'
config dns_policy
option name 'Redirect Local IP DNS'
option src_addr '192.168.1.5'
option dest_dns '1.1.1.1'
option enabled '0'
config policy
option name 'Ignore Local Requests'
option interface 'ignore'
option dest_addr '10.0.0.0/24 10.0.1.0/24 192.168.100.0/24 192.168.1.0/24'
option enabled '0'
config policy
option name 'Johan-PC-lan'
option src_addr '192.168.1.133'
option interface 'OpenVPN'
config policy
option name 'Paddan'
option src_addr '192.168.1.151'
option interface 'wan'
config policy
option name 'Johans-S24'
option src_addr 'A4:A4:90:89:52:18'
option interface 'OpenVPN2'
config policy
option name 'iplocation'
option dest_addr 'iplocation.com'
option interface 'wan'
config policy
option name 'ipleak.net'
option interface 'wan'
option dest_addr 'ipleak.net'
option enabled '0'
config policy
option name 'Plex/Emby Local Server'
option interface 'wan'
option src_port '8096 8920 32400'
option enabled '0'
config policy
option name 'Plex/Emby Remote Servers'
option interface 'wan'
option dest_addr 'plex.tv my.plexapp.com emby.media app.emby.media tv.emby.media'
option enabled '0'
config dns_policy
option name 'iplocation dns'
option dest_dns '213.80.98.2'
option src_addr 'iplocation.com'
root@DEFCON:~# service pbr status
pbr - environment
pbr 1.1.8-r2 running on OpenWrt 23.05.5.
Dnsmasq version 2.90 Copyright (c) 2000-2024 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
pbr fw4 nft file: /usr/share/nftables.d/ruleset-post/30-pbr.nft
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000 mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000 mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add chain inet fw4 pbr_mark_0x030000
add rule inet fw4 pbr_mark_0x030000 mark set mark and 0xff00ffff xor 0x030000
add rule inet fw4 pbr_mark_0x030000 return
add chain inet fw4 pbr_mark_0x040000
add rule inet fw4 pbr_mark_0x040000 mark set mark and 0xff00ffff xor 0x040000
add rule inet fw4 pbr_mark_0x040000 return
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.133 } goto pbr_mark_0x020000 comment "Johan-PC-lan"
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.151 } goto pbr_mark_0x010000 comment "Paddan"
add rule inet fw4 pbr_prerouting ether saddr { A4:A4:90:89:52:18 } goto pbr_mark_0x030000 comment "Johans-S24"
add rule inet fw4 pbr_prerouting ip daddr { 23.88.65.58 } goto pbr_mark_0x010000 comment "iplocation"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.1.151 } meta nfproto ipv4 tcp dport 53 dnat ip to 213.80.98.2:53 comment "Paddan"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.1.151 } meta nfproto ipv4 udp dport 53 dnat ip to 213.80.98.2:53 comment "Paddan"
add rule inet fw4 pbr_dstnat ip saddr { 23.88.65.58 } meta nfproto ipv4 tcp dport 53 dnat ip to 213.80.98.2:53 comment "iplocation dns"
add rule inet fw4 pbr_dstnat ip saddr { 23.88.65.58 } meta nfproto ipv4 udp dport 53 dnat ip to 213.80.98.2:53 comment "iplocation dns"
pbr chains - policies
chain pbr_forward { # handle 37
}
chain pbr_input { # handle 38
}
chain pbr_output { # handle 39
}
chain pbr_postrouting { # handle 41
}
chain pbr_prerouting { # handle 40
ip saddr 192.168.1.133 goto pbr_mark_0x020000 comment "Johan-PC-lan" # handle 1115
ip saddr 192.168.1.151 goto pbr_mark_0x010000 comment "Paddan" # handle 1116
ether saddr a4:a4:90:89:52:18 goto pbr_mark_0x030000 comment "Johans-S24" # handle 1117
ip daddr 23.88.65.58 goto pbr_mark_0x010000 comment "iplocation" # handle 1118
}
chain pbr_dstnat { # handle 36
ip saddr 192.168.1.151 meta nfproto ipv4 tcp dport 53 dnat ip to 213.80.98.2:53 comment "Paddan" # handle 1119
ip saddr 192.168.1.151 meta nfproto ipv4 udp dport 53 dnat ip to 213.80.98.2:53 comment "Paddan" # handle 1120
ip saddr 23.88.65.58 meta nfproto ipv4 tcp dport 53 dnat ip to 213.80.98.2:53 comment "iplocation dns" # handle 1121
ip saddr 23.88.65.58 meta nfproto ipv4 udp dport 53 dnat ip to 213.80.98.2:53 comment "iplocation dns" # handle 1122
}
pbr chains - marking
chain pbr_mark_0x010000 { # handle 1103
meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 1104
return # handle 1105
}
chain pbr_mark_0x020000 { # handle 1106
meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 1107
return # handle 1108
}
chain pbr_mark_0x030000 { # handle 1109
meta mark set meta mark & 0xff03ffff | 0x00030000 # handle 1110
return # handle 1111
}
chain pbr_mark_0x040000 { # handle 1112
meta mark set meta mark & 0xff04ffff | 0x00040000 # handle 1113
return # handle 1114
}
pbr nft sets
pbr tables & routing
IPv4 table 256 pbr_wan route:
default via 100.69.0.1 dev eth1
IPv4 table 256 pbr_wan rule(s):
30000: from all fwmark 0x10000/0xff0000 lookup pbr_wan
IPv4 table 257 pbr_OpenVPN route:
default via 10.128.0.85 dev tun0
IPv4 table 257 pbr_OpenVPN rule(s):
29998: from all fwmark 0x20000/0xff0000 lookup pbr_OpenVPN
IPv4 table 258 pbr_OpenVPN2 route:
default via 10.129.3.92 dev tun1
IPv4 table 258 pbr_OpenVPN2 rule(s):
29996: from all fwmark 0x30000/0xff0000 lookup pbr_OpenVPN2
IPv4 table 259 pbr_wgc1 route:
default via 172.19.65.19 dev wgc1
IPv4 table 259 pbr_wgc1 rule(s):
29994: from all fwmark 0x40000/0xff0000 lookup pbr_wgc1
root@DEFCON:~# cat /var/run/pbr.nft
#!/usr/sbin/nft -f
add chain inet fw4 pbr_mark_0x010000
add rule inet fw4 pbr_mark_0x010000 mark set mark and 0xff00ffff xor 0x010000
add rule inet fw4 pbr_mark_0x010000 return
add chain inet fw4 pbr_mark_0x020000
add rule inet fw4 pbr_mark_0x020000 mark set mark and 0xff00ffff xor 0x020000
add rule inet fw4 pbr_mark_0x020000 return
add chain inet fw4 pbr_mark_0x030000
add rule inet fw4 pbr_mark_0x030000 mark set mark and 0xff00ffff xor 0x030000
add rule inet fw4 pbr_mark_0x030000 return
add chain inet fw4 pbr_mark_0x040000
add rule inet fw4 pbr_mark_0x040000 mark set mark and 0xff00ffff xor 0x040000
add rule inet fw4 pbr_mark_0x040000 return
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.133 } goto pbr_mark_0x020000 comment "Johan-PC-lan"
add rule inet fw4 pbr_prerouting ip saddr { 192.168.1.151 } goto pbr_mark_0x010000 comment "Paddan"
add rule inet fw4 pbr_prerouting ether saddr { A4:A4:90:89:52:18 } goto pbr_mark_0x030000 comment "Johans-S24"
add rule inet fw4 pbr_prerouting ip daddr { 23.88.65.58 } goto pbr_mark_0x010000 comment "iplocation"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.1.151 } meta nfproto ipv4 tcp dport 53 dnat ip to 213.80.98.2:53 comment "Paddan"
add rule inet fw4 pbr_dstnat ip saddr { 192.168.1.151 } meta nfproto ipv4 udp dport 53 dnat ip to 213.80.98.2:53 comment "Paddan"
add rule inet fw4 pbr_dstnat ip saddr { 23.88.65.58 } meta nfproto ipv4 tcp dport 53 dnat ip to 213.80.98.2:53 comment "iplocation dns"
add rule inet fw4 pbr_dstnat ip saddr { 23.88.65.58 } meta nfproto ipv4 udp dport 53 dnat ip to 213.80.98.2:53 comment "iplocation dns"
root@DEFCON:~# nft -c -f /var/run/pbr.nft
Spent the afternoon reconfiguring PBR and Wireguard.
Downloaded the latest (Version 1.1.8-r4)
Removed everything that wasn't needed and just used PBR.
It seems like I needed to get new settings for wireguard from the VPN provider.
Now I can ping (ping -I wgc1 8.8.8.8) and get a response. However, I can't ping with curl.
I don't have wan access from PBR, tried rule (wan-access 192.168.0.1/24 wan) but it didn't work.
Are separate rules needed for each IP number?
I am familiar with the OpenVPN man page, but that does not answer the question why you cannot use WireGuard and have to use OpenVPN.
BTW 2.4 is old if you want to reference the man page use the current one (for 24.10 and main): https://openvpn.net/community-resources/reference-manual-for-openvpn-2-6/
With your configuration there are just to many things sightly off, as already earlier recommended, I would reset and start fresh.
Some general recommendations for the most trouble-free VPN setup if you are using multiple tunnels with PBR.
First get your basic setup in order, regarding DNS set a publicly available DNS server on your WAN (e.g. 1.1.1.1 or 149.112.112.112) and if you have implemented IPv6 also set a publicly available IPv6 DNS server on the WAN (e.g. 2620:fe::fe or 2620:fe::9 )
Keep Ocams razor and KISS in mind.
After each step make a backup so that you can go back to a working configuration.
Stick to one kind of VPN, preferable WireGuard as it is faster and easier to setup and OpenVPN setups and tears down its own interface which makes it is less predictable so you can run into timing issues.
When using multiple tunnels it is recommended to let the default routing (table main) use the WAN for several reasons, VPNs are not always reliable, servers are sometimes overcrowded or down for maintenance etc and if you are not careful in your setup you might end up running a tunnel in a tunnel.
So for WireGuard do not enable Route Allowed IPs (or disable Default routing) for OpenVPN add to the config: pull-filter ignore "redirect-gateway" and if you are using IPv6: pull-filter ignore "redirect-gateway ipv6" (some providers e.g. Mullvad even push /4 IPv6 routes which need additional blocking)
When using multiple tunnels you can use PBR to direct your traffic to one tunnel or another, to stop DNS leaks use DNSMasq option 6 or use DNS policies of the PBR app(https://docs.openwrt.melmac.net/pbr/) (note those are not compatible with DNS hijacking). If you also use IPv6 make sure you also take that into account when setting up PBR DNS policies and make sure you redirect ipv6 DNS also. To do this either make two DNS policies, one with IPv4 source and destination address and one with IPv6 source and destination address or more easily use the MAC address or e.g. lan interface as source and the VPN interface as destination as that does setup both IPv4 and IPv6 (make sure you have set an IPv4 and IPv6 DNS address on the VPN interface which you want to use).
It is certainly possible to do things differently these are just my personal recommendation for the easiest and most robust and trouble-free setup.
I seem to have PBR that works. I have wan as the default gateway.
Had to use the script "ovpn-update-resolv-9" to get dns from vpn-provider in to tunnel.
What doesn't work is using DNS policies from the PBR program to get wan to get dns from internet provider. (Have dns from vpn-provider)
Tried option 6 ("6,213.80.98.2,213.80.101.3") in DHCP options,
I couldn't get it to work.
Haven't yet tested config tag 'tag1'
to get wan clients internet provider dns.
it feels like I'm not really doing everything right
I'm more familar with Murphys law.
After I have defeated Murphy's law it seems that PBR works.
However I wonder about some things that can be improved.
Wondering how to merge ipv4 and ipv6 DNS policy so I only need one policy rule.
See the picture.
LAN clients get DNS through WAN from PBR DNS policy however it seems that both VPN and WAN DNS are assigned to clients.
ipleak shows dns from both wan and vpn, it should only be from wan.
@egc