VPN Policy-Based Routing + Web UI -- Discussion

Can you flush dns cache on client?

You are right. That's something wrong on client's side.

  1. Open ipinfo.io on first client -> ip according to policy
  2. restart vpn-policy-routing and clear cache on first client (command taken from https://help.dreamhost.com/hc/en-us/articles/214981288-Flushing-your-DNS-cache-in-Mac-OS-X-and-Linux)
  3. Open ipinfo.io on first client -> default ip, no entries in ipsets
  4. Open ipinfo.io on second client for 1st time ever -> ip according to policy, new entry in ipset
  5. Open ipinfo.io on first client -> still default ip

No dnsmasq restart required.

Hello,

I'm trying to set up Policy based routing on my 1900ACS.
Atm it's not working as it should but my guess is that this an issue with my VPN setup.
I'm connected to a L2TP Server (working if it's in the "wan zone" in firewall settings). Once I activate the Policy based routing it has simply no effect (one ruled added: my PC to WAN).

There has to be an obvious error on my part. I'm happy if anyone could help me here.

Installed this on 2 routers yesterday, following the instructions to the Tee.
Loading the Luci gui results in the following:

/usr/lib/lua/luci/dispatcher.lua:460: Failed to execute cbi dispatcher target for entry '/admin/services/vpn-policy-routing'.
The called action terminated with an exception:
/usr/lib/lua/luci/model/cbi/vpn-policy-routing.lua:32: attempt to call method 'sub' (a nil value)
stack traceback:
[C]: in function 'assert'
/usr/lib/lua/luci/dispatcher.lua:460: in function 'dispatch'
/usr/lib/lua/luci/dispatcher.lua:141: in function </usr/lib/lua/luci/dispatcher.lua:140>

Manual restart of the service after enabling in config produces this:

root@Tek-Node-2:/old_root/root# /etc/init.d/vpn-policy-routing restart
vpn-policy-routing 0.0.1-19 stopped [✓]
Creating table 'wan6//0.0.0.0/::/0' [✗]
BusyBox v1.25.1 () multi-call binary.

Usage: grep [-HhnlLoqvsriwFE] [-m N] [-A/B/C N] PATTERN/-e PATTERN.../-f FILE [FILE]...
...and the help info x3 or 4 (seems to do so for each of the VPN connections it finds)...
Creating table 'vpn1//0.0.0.0/::/0' [✗]
Creating table 'vpn2//0.0.0.0/::/0' [✗]
ERROR: vpn-policy-routing 0.0.1-19 failed to set up any interface!
vpn-policy-routing 0.0.1-19 monitoring interfaces: vpn1 vpn2 vpn3 wan [✓]

One router is an x86 arch, the other an ar71xx, both running LibreMesh (LEDE 17.01.4-based).

Uninstalled and reinstalled. Nada. Went into the config and enabled there, in case this were necessary. No change.

Is there something I'm doing wrong? Something I must edit, to make this work as intended?

Whoa, this isn't OpenWrt-specific, afaik the :sub is part of the basic Lua string library. I don't know if it's going to help, but try running this in the terminal after the luci package is installed:

sed -i 's/:sub/.sub/g' /usr/lib/lua/luci/model/cbi/vpn-policy-routing.lua
rm -rf /var/luci-modulecache/; rm -f /var/luci-indexcache;

Then reload the page in WebUI. Let me know if that helps. If it doesn't, ask the LibreMesh creators why the string:sub and string.sub do not work in Lua on their build.

For the actual service, I suspect the built-in grep is failing on -m1. Try the following:
opkg update; opkg install grep first and then restart/reload the service.

Well thank you, for such a prompt response!

After doing all of what you suggested (no answer on the inquiry with the LibreMesh folks yet, of course), very little changed. ONLY, in fact, the results of restarting the service (didn't bother editing):

root@Tek-Node-2:/old_root/root# /etc/init.d/vpn-policy-routing restart
vpn-policy-routing 0.0.1-19 stopped [✓]
Creating table 'wan6//0.0.0.0/::/0' [✗]
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
Creating table 'cjdns//0.0.0.0/::/0' [✗]
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
Creating table 'azire_sweden//0.0.0.0/::/0' [✗]
ERROR: vpn-policy-routing 0.0.1-19 failed to set up any interface!
vpn-policy-routing 0.0.1-19 monitoring interfaces: cjdns azire_sweden azire_us wan [✓]

I might mention that I am doing all changes here in a "temporary" zram-extroot environment on the wndr3700v2, to prevent any permanent changes that I cannot easily undo. Should I run the sed command on the actual root? I guess that could be corrected by a simple un/reinstall...

And I did. And it didn't help. Lemme determine for you the actual origin of the Lua code I'm working with, because I believe very little in the LEDE 17.01.4 base (if any) has been changed for LiMe's sake. They just add packages and scripting, from my limited knowledge.

I have no idea how does zram-extroot work. But maybe that's why the luci app is not working.

Can you post /etc/config/network and post the output of ifconfig?

Just pushed vpn-policy-routing 0.0.1-19b01 to the repo. I no longer think that grep -m1 was a problem but on the off chance it was, please try 19b01.

That's quite alright. The script I found (although I can't find it again to link here for you) is actually called ram-root. Given sufficient ram, it allows for an on-the-fly extroot /overlay created on a ram- or zram-drive, thus preventing any damage to real production stuff with test changes. Honestly, I use it even when NOT testing changes, simply because it's faster...lol

Anyhow, I know just as little about sed as you do about zram-extroot, and as a result, had not realized that we were just replacing :sub with .sub. Either way was simply no-go, zram-extroot or not.

Still have not determined the origin of the Lua issue, as I was editing when this reponse came. I'll let you know.

On top of what I've requested above, please try and post the output of:

lua
test='abcdefgh'
=test:sub(0,3)

Then press Ctrl-C to exit Lua session.

OK, so thanks to @NeoRaider I think I've fixed the luci app, please try luci-app-vpn-policy-routing 19 from my repo and let me know.

Could you show me your config files? I'm trying to get two different SSIDs set up where only one gets routed through VPN but I'm having massive trouble.

Still waiting for the feedback on the luci app. I might have to tweak the service as well to support LibreMesh.

How do I run these?

Has anyone managed to find a solution for the DNS leak issue? I'm running a local resolver (unbound) and no matter what policies I add, the DNS traffic is always routed over the WAN link if I set the route-nopull directive in the OpenVPN profile.

Is it feasible/possible to override the default routing table using policy-based routing? I know that isn't the current behavior in this package but it seems like it would be ideal for the VPN interface to be the implicit default gateway and then add rules to route selected IPs/ports over the WAN. Currently I have to add route-nopull to get the package to work which causes the WAN to be the default instead of the VPN, which is causing issues for traffic originating from the router (i.e., my unbound DNS resolver).

Hi,
first of all, thank you stangri for your great work :slight_smile:
I have a small problem with latest build. port based rules are not honored (log shows green ticks and support log looks also fine - rules are created). Thing is I want to split traffic between two gateways based on the forwarded port. So when you connect to IP1 and PORT1 you get reply through IP1, but if you connect to IP2 and PORT2 you get reply through IP2, but device behind firewall is the same only service (port) is different.
This worked with buid 18 or 19 but after update to 21 these rules don't work, only generic rules with IP address.
Is there any way to download older (working for me) version ?
Thanks.

Igor,
comment out line 220 and uncomment line 221 in /etc/init.d/vpn-policy-routing.

Please PM me your config and the output of /etc/init.d/vpn-policy-routing status tho, as build 21 should still work if you specify an IP and the port in the same policy.

Stangri,
thank you very much, your sugestion worked :slight_smile:
Config and status sent...

1 Like

Well, after banging my head against this for a long time I've managed to solve both the issues with DNS leaks as well as the inter-VLAN routing issues. To recap, I have two VLANs (DMZ and LAN), one of which I want to route over the VPN by default and one of which I want to route over the WAN, but I also want to be able to access the DMZ from the LAN. I also want to be able to use this package to allow port forwards to the LAN as well as selectively route traffic from the LAN to the WAN by IP, port, hostname, etc .

I don't fully understand why some of it works the way it does but here was my solution in case anyone else runs into similar problems. I'm also not sure if the order is significant or not:

  1. Remove the route-nopull directive from the OpenVPN config file so that by default, all traffic is routed over tun0. This will fix the DNS leaks that were caused by traffic being routed over the WAN by default when using route-nopull.
  2. Bring up the OpenVPN tunnel
  3. Create a routing table for the VLAN to be routed over the WAN (in my case, the DMZ). First, define it in /etc/iproute2/rt_tables:
#
# reserved values
#
128	prelocal
255	local
<...>
5       dmz
  1. Map the DMZ VLAN to the dmz routing table:
$ ip rule add from 10.10.10.0/24 table dmz
  1. Add a default route over the WAN interface as well as a route to the LAN to the dmz table:
$ ip route add default via <ISP WAN gateway IP> dev eth1.2 table dmz
$ ip route add 192.168.1.0/24 dev br-lan table dmz
  1. Start the vpn-policy-routing service and add rules, ignoring the DMZ subnet. I think one of the reasons I was running into trouble before was I was trying to set a rule to route the DMZ VLAN over the WAN, and there was no way to set a route to the LAN from the DMZ using vpn-policy-routing. So, traffic made it to the DMZ host from the LAN, but then the response was being sent over the WAN and the connection appeared to hang.

Update: I just realized the setup above breaks DNS for the DMZ because the router doesn't know how to route packets back to it. Adding another routing table for packets destined for the DMZ seems to have resolved it:

Add a new rule ("todmz") to /etc/iproute2/rt_tables:

[...]
128	prelocal
255	local
254	main
253	default
10	dmz
5	todmz
0	unspec

Create the following script to automatically set up the DMZ routing rules and configure it to start on boot:

#!/bin/sh

ip rule add from 10.10.10.0/24 table dmz
ip route add default via <ISP WAN Gateway IP> dev eth1.2 table dmz
ip route add 192.168.1.0/24 dev br-lan table dmz

ip rule add to 10.10.10.0/24 table todmz
ip route add 10.10.10.0/24 dev eth0.3 table todmz
1 Like

@algebro -- sorry I was of no help to you in resolving your problem. Now that you resolved the issue, you may want to consider making a wiki article about it.

I just pushed vpn-policy-routing 0.0.1-22 to my repo (build 21 push was premature).

If you set vpn-policy-routing.config.iprule_enabled='1', it will try to insert the ip rules for policies with only local device in them, which should reduce the number of iptables rules and improve overall performance (if you have many policies resulting in iptables rules).

By default this feature is not enabled, because having some policies resulting in ip rules, some in ipsets and some in iptables rules would make it impossible to establish priorities or make policies targeting same ip but with and without specific ports work (like @iGz demonstrated).

I'm seeking feedback from the userbase on this feature. Again, to activate it, run:

uci set vpn-policy-routing.config.iprule_enabled='1'
uci commit vpn-policy-routing
opkg update
opkg upgrade vpn-policy-routing

I have ipv6 related question. Right now I have ipv6 disabled as I didn't want to deal with an additional nightmare when using VPN policy based routing. But as this realistically that is not a long term solution so I would like to make progress on it.

Here is my scenario: My own ISP provides both ipv4 and ipv6 addresses - so far so good.
My openvpn providers are not so progressive. Few do both but rest of them only do ipv4.

My worry is as follows: With ipv6 enabled say I want to connect to www.google.com. I set up routing based on dns name that www.google.com should use "vpn1". Since google supports both ipv4 and ipv6 the router after a dns query receives list of ipv4 and ipv6 addresses to which www.google.com points to.

What happens next? Say provider "vpn1" does not support ipv6. How will my connection be handled? I assume when dns returns both ipv4 and ipv6 then ipv6 is prefered. But "vpn1" cannot handle ipv6! Will it failover to another connection (I don't have strict policy enforcement enabled)? Or if I enable it will the connection fail completely? Or will it go to ipv4? Can anyone provide answers about this?