PBR not working properly with OpenVPN Clients

sTeve51 · June 15, 2024, 3:54pm

I have 2 OpenVPN clients and I am using policy based routing to manage the traffic, I set the wan interface as default gateway, the problem is that PBR doesn't seem to be working properly. When I create a policy based on domain, if add ipleak.net it works, the site shows me the IP of the VPN tunnel I choose and the DNS I have on AdGuard Home, but when I add all the YouTube domains to go through a VPN interface, I see ads of my country instead of ads of the country I am connecting to. I am using a GL.iNet device and with their stock firmware it works perfectly, I always see ads of the country I choose on the VPN client based on domain name, adding the exact same domains I add on vanilla OpenWrt of course. It's like the domains are being ignored and the traffic is going through the wan instead.

I am not sure if that is a leak or something, but it is weird, I tried without AdGuard Home and still the same, I hope someone can help because I want to stay on pure OpenWrt, I tried a lot of configs on the firewall and still no luck, this issue is driving me crazy. Thanks

egc · June 15, 2024, 4:30pm

Please connect to your OpenWRT device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:

Remember to redact keys, passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/dhcp
cat /etc/config/firewall
ip route show
ip route show table all
ip rule show
for ovpn in $(ls /etc/openvpn/*.ovpn);do echo $ovpn; cat $ovpn; echo;done
logread | grep openvpn
cat /etc/config/pbr
/etc/init.d/pbr status
uci set pbr.config.verbosity='2
uci commit pbr
/etc/init.d/pbr reload
/etc/init.d/pbr status

# for ipset/nftset reboot and after reboot contact the domains first before getting output of the following items:
nft list ruleset
cat /tmp/dnsmasq.d/pbr

sTeve51 · June 15, 2024, 9:26pm

Sure, here is it, I have recreated the issue from the scratch.
The packages I installed are these, in case it helps
OpenVPN
openvpn-openssl
ip-full
luci-app-openvp
PBR
pbr
luci-app-pbr
AdGuard
adguardhome
(I use more packages but they are not related to the issue and they were not installed during this test)
The log is kinda big I had to upload it to drive, I wasn't able to post it here unless I split it I guess
Thank you for helping me

test log

egc · June 16, 2024, 6:26am

That makes it more difficult to quote things but no problem

It looks like it is working:

       set pbr_nordvpntun2_4_dst_ip_cfg046ff5 { # handle 688
                type ipv4_addr
                flags interval
                counter
                auto-merge
                comment "Test"
                elements = { 95.85.16.212 counter packets 4025 bytes 567747, 142                                                                                                             .250.64.142 counter packets 0 bytes 0,
                             142.250.64.174 counter packets 36 bytes 6869, 142.2                                                                                                             50.64.202 counter packets 0 bytes 0,
                             142.250.64.206 counter packets 74 bytes 15478, 142.                                                                                                             250.64.234 counter packets 35 bytes 6833,
                             142.250.64.238 counter packets 31 bytes 6521, 142.2                                                                                                             50.189.138 counter packets 291 bytes 80792,
                             142.250.189.142 counter packets 71 bytes 17254, 142                                                                                                             .250.217.170 counter packets 70 bytes 8420,
                             142.250.217.174 counter packets 68 bytes 14825, 142                                                                                                             .250.217.202 counter packets 91 bytes 16101,
                             142.250.217.206 counter packets 80 bytes 15025, 142                                                                                                             .250.217.234 counter packets 25 bytes 4677,
                             142.250.217.238 counter packets 40 bytes 9792, 142.                                                                                                             251.35.234 counter packets 16 bytes 5648,
                             172.217.2.202 counter packets 31 bytes 4207, 172.21                                                                                                             7.2.206 counter packets 0 bytes 0,
                             172.217.3.78 counter packets 11 bytes 2984, 172.217                                                                                                             .15.206 counter packets 4 bytes 1396,
                             172.217.165.196 counter packets 0 bytes 0, 172.217.                                                                                                             165.202 counter packets 93 bytes 12343,
                             172.217.165.206 counter packets 68 bytes 16131, 192                                                                                                             .178.50.42 counter packets 27 bytes 4779,
                             192.178.50.46 counter packets 9 bytes 2880, 192.178                                                                                                             .50.74 counter packets 81 bytes 10194,
                             192.178.50.78 counter packets 78 bytes 18663 }

These are some of the ip addresses which are used as destinations and are send via the tunnel.
Perhaps you might not have added all relevant youtube addresses and/or youtube is tracking your DNS origin and the way you have setup is that your DNS origin is coming from your home.

Check if you have really all relevant youtube destinations and try with setting resolver set to nft to dynamically catch all domain names, use the GUI to change I do not know if the option is really nft:
You now have set the resolver set to none:

option resolver_set 'none'

Furthermore you might look into "DNS leak"

Here are my personal notes about DNS leak:

Some small remarks:

In DHCP you have:

        list dhcp_option '6,192.168.73.1'
        list dhcp_option '3,192.168.73.1'

That is redundant as 192.168.73.1 is your routers address so that is done by default

In VPN configs you can add:
redirect-private def1 besides pull-filter ignore "redirect-gateway" (no need to put the -- in front although it does not hurt)

sTeve51 · June 16, 2024, 3:44pm

I have added all domains, when I add those on their stock GL.iNet firmware it works and all the YouTube traffic goes through the VPN, I have been playing with the firewall all night and still no luck, so I think is not a firewall problem.

Another really important detail I forgot to mention, when I added that domain based policy, the traffic shown on the interface is not accurate, I could be watching a 4K video on YouTube and the RX packages of the VPN tunnel I chose barely increase (on the stock firmware I see a lot of bandwidth with the same policy and domains). However if I add a policy based on a device instead of domain, the traffic on the VPN interface is accurate and I see a lot of MBs being used. I don't think that's a bug, that's in fact the reason since it feels like the domains could be ignored and the traffic is going through the wan instead. Makes more sense now.

Based on my tests, I don't think YouTube tracks the DNS because I have had a similar setup on their stock firmware, the DNS leak showed the VPN IP and the DNS I had on AdGuard and the policy worked perfectly.

As you know, in the log I posted here I have AdGuard Home installed, I followed the official OpenWrt documentation, so AdGuard is listening on 192.168.73.1 and dnsmasq was moved to port 54, I always see the DNS I have on AdGuard, never my ISP DNS, so I think there is not a DNS leak.

So I could just add this at the bottom of the config?

redirect-private def1
pull-filter ignore "redirect-gateway"

Thanks for your time I really appreciate it

egc · June 16, 2024, 4:13pm

Correct, but that will not solve your problem

The set with IP addresses is populated it is using the routing table of pbr_nordvpntun2 and if you add ipleak.net it shows the correct location.

So the things I can think of is:
you did not add all necessary domains or did not set the resolver nft set or it tracks DNS.

A DNS leak is that your DNS goes out via the WAN and not via the VPN but if you say that is not the problem OK, but for things like Netflix, Amazon, BBC, banking apps that is a major problem.

You say stock firmware works, can it also do PBR?
If so does your DNS query go out via the WAN or is it using the tunnel?

sTeve51 · June 16, 2024, 5:13pm

Exactly, I even see RX packets on the 'nordvpntun2' interface while using YouTube but the bandwidth it shows is wrong, maybe they're partially going through the tunnel or are not being forced to, it's strange. While watching a 4K video on YouTube the traffic doesn't even increase 5MB

Yes, the stock GL firmware is based on OpenWrt, they have their own GUI, on their GUI it works, all traffic goes through the VPN and the bandwidth goes accordingly to what I do, it can do PBR but it's more limited, and I am not sure if they use the pbr packages internally. My DNS query go through AdGuard too but I can choose between the VPN DNS my custom AdGuard DNS.

How can I change it?

egc · June 16, 2024, 5:20pm

It is in my earlier post.

If the stock firmware works, why not use that?

Anyway this is all I can think off.

sTeve51 · June 16, 2024, 5:37pm

Just tried, still the same

I want to stay on pure OpenWrt because is more recent and better for all the things I do, I will keep trying

sTeve51 · June 16, 2024, 9:07pm

Maybe if the VPN traffic uses the OpenVPN client DNS the issue could be solved so that way ipleak.net would show the VPN IP and the VPN DNS instead of the VPN IP and my Custom DNS.
What would the best way to achieve that, based on my current configuration? I could add the DNS addresses of my VPN provider, but I am using AdGuard

egc · June 17, 2024, 7:04am

This latest PBR package has a setting for that with which you can set your local clients to use a specific tunnel for the VPN

Not sure if that is already available other wise you can use option 6 for that specific clients see:

Do not use the DNS server from your provider unless it is publicly available, in your case that is a bad idea regardless as the ip range of both tunnels are the same

10.100.0.0/24 dev tun1 proto kernel scope link src 10.100.0.2
10.100.0.0/24 dev tun0 proto kernel scope link src 10.100.0.2

sTeve51 · June 18, 2024, 8:35pm

I installed the new PBR package and I was able to use the VPN DNS while the tun0 was the default gateway, however my wan DNS was also being used so it was a leak. But I was able to use the VPN DNS without adding them manually, which is cool.
I changed the default gateway to be the wan interface, and it stopped working, ipleak shows the VPN IP and my wan DNS, not sure what I am doing wrong.
I added list dns '10.0.0.2' to my VPN interface

config interface 'tun1'
	option proto 'none'
	option device 'tun1'
	list dns '10.0.0.2'

I also tried 10.100.0.2 because that's the IP of my interface and nothing worked, only my wan DNS was being used, I tried more things but I will leave a picture below, I'm testing on a specific device

unannunciated · June 18, 2024, 9:30pm

couple pointers first

never install modules that are not of your os release.
when experimenting with pbr, reboot often
don't be redundant in your Policies table.
don't try to enforce default gw, PBR handles that

My threat model differs from yours, and I use WG, but the basics should be the same. I noticed your policies table is incomplete.

On the line where you're experimenting, add the verb "prerouting" in the chain column and "all" in the protocol column. Save and Reboot.

sTeve51 · June 18, 2024, 10:49pm

That seems to be a bug in LuCI because when I hit edit, "prerouting" is selected as default, same for the protocol, and sure, I was just testing

egc · June 19, 2024, 6:14am

About the DNS probably better to not use10.0.0.2 as you have two tunnels with the same subnet so it could end up in the wrong tunnel.

The problem this DNS setting should solve is that your DNS request goes trough the same tunnel as your LAN clients traffic.
It does not need to be the providers DNS server.

So just use e.g. 9.9.9.9 for DNS on your test Samsung and then check with ipleak.net and dnsleaktest.com on your test samsung if the DNS has the same country as your IP address.

Make sure that IPv6 is not interfering so either disable that or make sure that it is also routed via the tunnel.

Furthermore WebRTC can also reveal your actual location

sTeve51 · June 20, 2024, 3:08am

It didn't end up working, but I did a workaround for now.
I made 3 VLANS and I linked both VPN tunnels to 2 of those, also made one for IoT. I linked tun0 to guestwifi, tun1 to a new one. I also routed the IoT through tun0. That way I just connect to the one I want to use, probably not the smartest way to achieve what I want, but for now I think I'm ok with that and what I wanted is kinda complex considering my full settings. The future PBR package with DNS policies might help

I will leave some pictures below of my firewall settings, if you think something can be enhanced or done in a better way, any recommendation is appreciated
I am using AdGuard and all the DNS queries are going through it, even for my guest Wi-Fi, I could change it I think it's ok, I just don't want to use my ISP DNS.

I have 2 small questions, on each VLAN I had to add a DHCP option list in order for the DNS to work, example: 6,192.168.1.1
If I also add 3,192.168.1.1 what would the difference? I only added: 6,lan-ip

On the lan zone, I set foward as reject, is it supposed to act like a kill-swich if one of my VPN tunnel fails?

Firewall

VPN Tunnels

Same settings for both tunnels and IoT

PBR

Traffic rules