Policy-based routing and Wireguard VPN failover if peer does not provide WAN access

Installing and Using OpenWrt
1

Hello everyone,

I am trying to solve the following problem and would appreciate any advice:

Problem Statement

I have a WAN interface and two wireguard interfaces WG1 and WG2 that are connected to a single peer each (a commercial VPN for privacy reasons) and are setup to route all traffic on that interface through the peer (via "allowed IPs" being set to 0.0.0.0\0).

Using the pbr service, I had set-up various rules to route traffic through WAN, WG1 and WG2 depending on sources (specified via IP or hostname) and destinations (specified via IP or domain name).

Recently, I realized that WG1 had no internet access, because of some problem with the peer, and some service was not accessible.

What I want to achieve now is to have a form of failover, that routes the traffic that is supposed to go over WG1 over WG2 in case that WG1 has no internet connection.

Solution attempts

  1. First I tried to use a pbr configuration without strict enforcement, adding a rule for WG2 after the ones for WG1. So if the interface WG1 is down, it should use the next matching rule.
    However, the problem here is that in this situation the WG1 is not down - it simply provides has no internet connection.

  2. Then I thought maybe I could set-up multiple peers in the WG1 interface. But this does not seem to work, since I cannot "magically" make the wireguard interface switch the routing options. Also, how would it know that one peer does not have internet access.

  3. After reading up a little I stumbled about the mwan3 package. After reading some tutorials, I thought the correct way would configure the whole policy based routing in mwan3 (even though I find it more complicated to set-up the rules), but as I understand it, domain-based routing for the destinations or hostname-based routing for the sources cannot be configured there.

Questions

  • Is there any way to still achieve my original goal with OpenWRT?
    In order to achieve that I am willing to
    • change my firmware version (at the moment I am on 23.05)
    • have mwan3 and pbr both running if necessary to achieve my goals
    • use the cli instead of luci, even though I am used to the latter.
    • using VLANs, even though I have no experience on that regard; but I am willing to learn
  • If this is not possible at all, what is the closest that I could achieve?
2

You need 2 wireguard interfaces to properly route the traffic.

3

I have already two wireguard interfaces setup, WG1 and WG2. I followed a tutorial for mwan3 to set up metrics such that I can configure a failover configuration using these devices. My problem ist, that only using mwan3 I could not configure rules that respect hostnames/domains.

4

You can use the tracking part of mwan3 (no members, policies or rules).
This will save you the headache of creating your own script to monitor the connection.

Then create a script in /etc/mwan3.user that does the following:

  • If WG1 goes into disconnected state, the main rules (via WG1) must be disabled (using uci commands) and the pbr service should be reloaded.
  • When the connection is restored, the changes must be reverted and the service should be reloaded again.
5

Hey pavelgl,
thank you so much for your answer.
I think this puts me on the right track, even though I have never written a script, but I guess there will be tutorials somewhere. Obviously, if you have some hints towards that, feel free to share them.

6

I use WireGuard with fail over but that uses standby tunnels so if one fails the next one is started.

It is not the same as what you want but perhaps it gives you some inspiration

See:

At the bottom a very simple PBR example

1 Like
7

Should be something like this:

#!/bin/sh

if [ "$ACTION" = "disconnected" -o "$ACTION" = "connected" ] && [ "$INTERFACE" = "WG1" ]; then
        
case $ACTION in

  disconnected)
    uci set pbr.@policy[5].enabled='0'
    uci set pbr.@policy[6].enabled='0'
    /etc/initd/pbr reload
    ;;

  connected)
    uci revert pbr
    /etc/initd/pbr reload
    ;;

  *)
    exit 0
    ;;
esac
fi
  • Run uci show pbr to see the numbers of the policies that need to be listed and disabled.
  • Make sure that WG1 is the correct logical interface name declared (in uppercase) in /etc/config/network and /etc/config/mwan3.
  • To check if it works, set the track ip option(s) in the mwan3 interface section to something unreachable (such as 1.2.3.4) and restart the mwan3 service.
1 Like
8

Thank you very much @pavelg and @egc for the pointers.
I will try to implement the solution of @pavelgl first and report back.

1 Like