Mwan3 with two routers

[darkislight]

Hey all,

I have a setup where I have a router connected to my Fibre ONT ("Main Router") which is the default gateway for the network. I also have a second router ("Backup Router") which is connected to my neighbours WiFi as a backup link.

I updated the Main Router with a static route pointing to the backup router with a metric of 20 for 0.0.0.0/0. The problem is that the wan link won't go down on the Main Router when there's issues with the ONT (or if it looses power) and therefore will not automatically forward packets to the Backup Router.

mwan3 should be able to test the wan for a stable connection, and when it fails, to remove the default routing for 0.0.0.0/0 to the wan interface so that the router will forward to the Backup Router which is already in the routing tables. Can anybody help me achieve this please?
I've tried adding just the wan link in mwan3 and also the wan and the lan link in a failover setup but it doesn't work regardless of the different settings I've tried. Thanks in advance!

Also if its possible when the WAN link goes down, to be able to run a script?

Here's a diagram to explain the setup:

The link will never go down because the LAN Port from the ONT and the WAN Port from the Main Router go through the switch. Both are on a separate, untagged two port VLAN. This is so that the switch can power the ONT via the Switch PoE. Main Router is also connected via its LAN Port to the switch (.69.254). Yes, I know this is an unusual way of connecting both devices. Open to suggestions for a better way to power my ONT via my switch. Although the backup link will be double natted, it will be rarely used since its slow (40 megabit down and up via the 5GHz Wifi 4 connection).

[d687r02j8g]

@darkislight

if its possible when the WAN link goes down, to be able to run a script?

Watchcat can do this. It works really well.

Watchcat - network watchdog utility

[darkislight]

Thanks @d687r02j8g based on what you have said it made me look into some non-mwan3 solutions. I figured out a solution... end of message. Jokes, here's what I did so others coming here might have some help.

I looked at watchcat and it seemed like a good candidate but in the end I choose pingcheck.

Firstly, I have set my wan interface gateway metric to 5 (advanced settings). Then, because my wan connection is DHCP client the udhcpc daemon automatically sets the default route with no metric (does not respect the gateway metric on interface). So /etc/udhcpc.user file executes when the dhcp client gets a new lease and I do the following:

# This script is sourced by udhcpc's dhcp.script at every DHCP event.
echo Running dhcp client script: $1
if [ $1 == "bound" ]; then
  echo Setting default route to metric 5
  ip route del default dev $interface
  ip route add default via $router dev $interface metric 5
fi

If the interface is bound, delete the default route with no metric, and then set the default route with a metric of 5.

Config file /etc/config/pingcheck:

config default
	option host 1.1.1.1
	option interval 10
	option timeout 30
	option protocol icmp

config interface
	option name wan
	option panic 5

config interface
	option name lan
	option host 192.168.69.251

• LAN will ping backup router and WAN connection will ping 1.1.1.1 (Cloudflare DNS) every 10 seconds, if pings fail over the course of 30 seconds, then the connection has gone offline.
• When a connection fails, all scripts in /etc/pingcheck/offline.d folder will execute.
• When a connection is back up, all scripts in /etc/pingcheck/online.d folder will execute.
• If wan connection is down for 5 minutes then run panic scripts in folder /etc/pingcheck/panic.d.

/etc/pingcheck/offline.d/redirect-router.sh:

#!/bin/sh

if [[ "${INTERFACE}" == "wan" ]]; then
echo "net $INTERFACE offline"
ip route add default via 192.168.69.251 dev br-lan metric 2
fi

Checks if the wan interface went offline, if so, adds the route for the backup router with metric of 2.

/etc/pingcheck/online.d/redirect-router.sh:

#!/bin/sh

if [[ "${INTERFACE}" == "wan" ]]; then
echo "net $INTERFACE online"
ip route del default via 192.168.69.251 dev br-lan metric 2
fi

Checks if the wan interface came back online, if so, deletes the route for the backup router.

/etc/pingcheck/panic.d/redirect-router.sh:

#!/bin/sh

if [[ "${INTERFACE}" == "wan" ]]; then
echo "net $INTERFACE needs a reset"
udhcpc -i eth1 -n
fi

My wan interface uses DHCP from ISP to get info, so this runs every 5 minutes when the connections been down to help bring back the Fibre connection (again because the link isn't directly connected to ONT). This will re-run the DHCP client from the wan connection and exit straight away if the lease could not be obtained (ie link is still down).

To explain what the above does, when the wan link goes down, it adds in the route to the backup router with a lower metric. Routes with lowest metric goes first, so setting the default wan metrics to 5 means that I can set the backup router metric lower and then packets will be routed to the backup router vs the wan connection.