VPN Policy-Based Routing + Web UI - ARCHIVE #1

I'm not sure I fully understand, but "this" will not change your current tunnel config/metrics. If you only have one tunnel, openvpn-policy-routing should work without any additional changes to your current tunnel.

What's your goal?

Oh I want multiple tunnels but I can't get the metric set on the interface. So trying just one vpn first.

Oh, you got me, I thought that the solution found by @nidstigator would work for everyone.

I think I found a problem in my config. For some reason the vpn was ignoring my /etc/config/openvpn and was starting using a config file in the /etc/openvpn directory which was missing the route-nopull option. So problem (of setting metric on the interface is solved).

Great! Please let me know how it goes with openvpn-policy-routing, especially if you'll be using domain-based policies. You're the second user with multiple tunnels, your feedback would be invaluable!

Sorry just one more question. Do I need to add a custom distfeeed for

opkg install openvpn-policy-routing luci-app-openvpn-policy-routing

to work? I have RC2 install and it does not find these two packages.

Edit: Looking at your github page now. I guess I need to compile them myself, right?

Or you can grab the pre-built ipk files from https://github.com/stangri/Files

These packages will not be backported to 17.01, but they should eventually be available in snapshots. I've submitted them to be included in official repo/builds, but there's some work I need to do before they are accepted.

Also, with my vpnbypass update breaking the buildbots earlier today I'm guessing all my submissions will be reviewed extra carefully now. :slight_smile:

1 Like

Good point. What's the most reliable way to detect if dnsmasq is being used for name resolution? Presence of /var/run/dnsmasq/dnsmasq.cfg*? Or is parsing of either /var/etc/dnsmasq.conf* or /var/run/dnsmasq.cfg* required?

You might see IPv6 settings in luci app now. It's not supported in the main app yet, but if your OpenVPN provider(s) support IPv6 and you can help test IPv6 related functionality, please chime in.

[quote="stangri, post:49, topic:1422"]
What's the most reliable way to detect if dnsmasq is being used for name resolution?
[/quote]Looking at the pid file or the process are probably the easiest ways to see if dnsmasq is running, but that provides no info on what it is doing: DNS, DHCP or both.

root@LEDE:~# pgrep dnsmasq
2610
root@LEDE:~# ls  /var/run/dnsmasq/dnsmasq*.pid
/var/run/dnsmasq/dnsmasq.cfg02411c.pid

Between parsing /var/etc/dnsmasq.conf* and using netstat -- what's the better way in your opinion?

Can I ask in what order are the rules evaluated?
If I have domain based policy

/whatismyip.com/tun1route

but also say

config policy
        option comment 'FireTV'
        option local_addrs '192.168.8.247'
        option remote_addrs '0.0.0.0/0'
        option gateway 'tun0'

will the connection to whatismyip.com if done on my FireTV go though tun1 or tun0?
I would like to be able to set the order, in fact in this case what I want to achieve is to have whatismyip.com to go though tun1 and everything else though tun0.

It is my understanding that domain-based policies have higher priority than IP/port-based policies.
So if you hit whatismyip.com from FireTV, it (like every other device on your network) should use tun1route.

PS. No need to explicitly specify option remote_addrs '0.0.0.0/0', you can just omit it.

1 Like

There are some issues with the routing I don't understand. First I try

/whatismyipaddress.com/tun0route

with no IP/port based route. The result is good, at whatismyipaddress.com I am getting the IP of my tun0 VPN, while checking for example www.whatismyip.com I am getting my wan IP. Exactly what I expect.

Now I add one IP based route for my laptop saying to redirect all request to tun1 VPN.
The domain based policy no longer works. Going to whatismyipaddress.com I get IP of my wan interface. However, the IP based route works, so going to www.whatismyip.com I am getting IP of my tun1 VPN. Removing the IP based route makes the domain based policy work again.

The multiple tunnels support has been added recently, without much testing (hence the thread subject), but let's see if we can get to the root of the problem.
First some questions -- strict or lazy enforcement? Did you reload the service after adding a new policy? Are both tunnels up? What's the most recent output (logread -e "policy")? When you say "works/no longer works" do you mean only on the affected laptop? Does domain rule still work for other devices on your network after you add laptop->tun1 policy?

OK, I had time to do more testing to answer your questions.
I used strict enforcement and created 2 rules. 2 VPNs on tun0 and tun1 up and running.

/whatismyipaddress.com/tun0route
config policy
        option comment 'MBP'
        option local_addrs '192.168.8.144'
        option gateway 'tun1'

Then tested whatismyipaddress.com from my MBP and desktop. MBP gave me IP of my wan interface not tun0. But the desktop has honoured the rule and gave me IP on tun0 interface.

Rest worked as expected, going to different ip checker www.whatismyip.com gave me IP of tun1 on MBP and IP of wan interface on desktop.

Then I have switched to lazy enforcement and retested both. There was no difference (with both VPN up). Each time I have made the change I have reloaded the config via the web interface. Finally, logread -e "policy" gives:

Thu Feb 16 19:28:16 2017 user.notice openvpn-policy-routing [16907]: creating table wan/eth1.2/xx.xx.xx.xx TID=200 FW_MARK=0x010000 IP_SET=wanroute [✓]
Thu Feb 16 19:28:16 2017 user.notice openvpn-policy-routing [16907]: creating table ukvpn/tun0/10.8.0.2 TID=201 FW_MARK=0x20000 IP_SET=tun0route [✓]
Thu Feb 16 19:28:17 2017 user.notice openvpn-policy-routing [16907]: creating table svkvpn/tun1/10.0.100.25 TID=202 FW_MARK=0x40000 IP_SET=tun1route [✓]
Thu Feb 16 19:28:17 2017 user.notice openvpn-policy-routing [16907]: routing 'MBP' via tun1 (192.168.8.144 to *.*.*.*) [✓]
Thu Feb 16 19:28:17 2017 user.notice openvpn-policy-routing [16907]: service started with gateways: wan/eth1.2/xx.xx.xx.xx ukvpn/tun0/10.8.0.2 svkvpn/tun1/10.0.100.25 [✓]

Thank you for a thorough investigation. The results of it defy my understanding of how it should work. I might have to add another iptables rule to force-match/mark packets based on IPs found in ipsets to fix that. It might take me some time to research/implement.

PS. For the time being, can you try resolving the domain(s) you want to be routed via tun1 and add their IP address(es) to the IP-based policies?

@dziny
Care to share the contents of your mangle table at the point when you have multiple tunnels up and things work strangely?

iptables -L -v -t mangle

One possible reason for unexpected behaviour could be that this app does not apply a mask properly to the mark everywhere (also at evaluation of the mark) and some other app (like SQM or QOS) also marks packets, so that the total mark is different. Do you use any QoS tools or such apps?

@stangri
Like I said earlier, you might investigate using a suitable mask there in "ip rule add". E.g. mwan3 does that.
https://github.com/openwrt/packages/pull/3903#issuecomment-275620686

https://github.com/openwrt/packages/blob/master/net/mwan3/files/lib/mwan3/mwan3.sh#L67

http://man7.org/linux/man-pages/man8/ip-rule.8.html

hmmm.
Looking at your code more closely I notice that you actually have "iptables -t mangle -F" there several times. E.g. https://github.com/openwrt/packages/pull/4005/files#diff-5b00e32f5597ec2aa554c51217cf3682R134

If I understand that correctly, you throw away all settings & rules in the mangle table made by any other app, e.g. firewall itself, SQM or qos-scripts. So your app practically kills all other apps using mangle table???? (at least if they start before this app).

You said earlier that vpnbypass worked ok with SQM. You might test starting first vpnbypass (or this) and then restarting SQM (so that its rules are created after your app's purge action.)

My bad, I've replaced it with this:
iptables_reset(){ iptables-save | grep -Fv -- "openvpn-policy-routing" | iptables-restore; lsmod | grep -q ip6table_nat && ip6tables-save | grep -Fv -- "openvpn-policy-routing" | ip6tables-restore; }
I've struggled with creating PROCD-compatible rules within the instance and just couldn't make them work.

I'll test locally before pushing it.

I've tried using fwmark $mark/0xff0000 at both matching and marking, but it didn't help at all.

Moving from vpnbypass to this was a more challenging task than I expected. I'll need to take a little break to clear my head and I'll get back to it within a week.