Fallback static route to location with multiple paths

Installing and Using OpenWrt Network and Wireless Configuration
1

Hey there,

I'm currently managing 2 locations both using a MikroTik RB760iGS (hex S) running OpenWRT as their main router.

Those locations are linked through a P2P Wireless link, and also have a Wireguard VPN running between them.

Location 1

root@RTR_LOC1:~# ip a show dev sfp #Wifi Link
3: sfp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 2c:c8:1b:99:b7:1b brd ff:ff:ff:ff:ff:ff
    inet 172.31.0.1/24 brd 172.31.0.255 scope global sfp
       valid_lft forever preferred_lft forever
    inet6 fe80::2ec8:1bff:fe99:b71b/64 scope link 
       valid_lft forever preferred_lft forever
root@RTR_LOC1:~# ip a show dev wg0 # Wireguard Link
37: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none 
    inet 172.29.0.2/24 brd 172.29.0.255 scope global wg0
       valid_lft forever preferred_lft forever

Location 2:

root@RTR_LOC2:~# ip a show dev lan2 # Wifi Link
5: lan2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 2c:c8:1b:99:b6:49 brd ff:ff:ff:ff:ff:ff
    inet 172.31.0.2/24 brd 172.31.0.255 scope global lan2
       valid_lft forever preferred_lft forever
    inet6 fe80::2ec8:1bff:fe99:b649/64 scope link 
       valid_lft forever preferred_lft forever
root@RTR_LOC2:~# ip a show dev wg0 # Wireguard Link
19: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none 
    inet 172.29.0.1/24 brd 172.29.0.255 scope global wg0
       valid_lft forever preferred_lft forever

LOC1 uses nets within 10.64.0.0/11 and LOC 2 nets within 10.0.0.0/11. Static routes are as follows:

root@RTR_LOC1:~# ip route show 10.0.0.0/11
10.0.0.0/11 via 172.31.0.2 dev sfp proto static 
10.0.0.0/11 via 172.29.0.2 dev wg0 proto static metric 1 

root@RTR_LOC2:~# ip route show 10.64.0.0/11
10.64.0.0/11 via 172.31.0.1 dev lan2 proto static 
10.64.0.0/11 via 172.29.0.2 dev wg0 proto static metric 1

Now my Issue currently is that if I stop a MAN interface on either Router the routes are changed accordingly. But the Router on the other location doesn't know that the route just died. Which ends up with traffic only working from the location that knows of the failed Interface.

I saw that apparently you could add something like:

ip route add 10.0.0.0/11 via 172.31.0.2 dev sfp monitor on

But ip even in the full version isn't happy with that. Error: either "to" is duplicate, or "monitor" is a garbage. Nor is there a config option where I could use this through /etc/config/network .

Do I have to script something myself here?

2

Looks like this static route is wrong, as the gateway should be 172.29.0.1.
It is best enable the route_allowed_ips option and remove those static routes for both peers.

You probably want to explicitly specify the MAN interface for the tunlink option.

1 Like
3

The Static route was indeed wrong. :+1:

Though it didn't change anything in the behavior. :confused:

Setting route_allowed_ips for the peer doesn't create any additional routes. Probably because there already is a route present for that destination via the MAN WiFi link.

tunlink would only make sense to set to the WAN interface not MAN. The whole idea of the Wireguard tunnel is to have a fallback route in case the WiFi interconnect dies.

4

Make sure to specify a custom metric on the WireGuard interface and restart it.

Post the main routing table in the normal mode and when the MAN link is down.
We need to better understand what actually happens to make it work as you want.

1 Like
5

Before stopping MAN on RTR_LOC2:

root@RTR_LOC1:~# ip route show
default via 192.168.178.1 dev wan proto static 
10.0.0.0/11 via 172.31.0.2 dev sfp proto static 
10.0.0.0/11 dev wg0 proto static scope link metric 1 
10.64.1.0/24 dev DMZ proto kernel scope link src 10.64.1.1 
10.64.2.0/24 dev IOT proto kernel scope link src 10.64.2.1 
10.64.3.0/24 dev GUEST proto kernel scope link src 10.64.3.1 
10.64.4.0/24 dev ASBCH proto kernel scope link src 10.64.4.1 
10.64.6.0/24 dev MECHA proto kernel scope link src 10.64.6.1 
10.65.1.0/24 dev SMMK proto kernel scope link src 10.65.1.1 
10.65.2.0/24 dev DMZSMMK proto kernel scope link src 10.65.2.1 
10.66.0.0/24 dev SPACE proto kernel scope link src 10.66.0.1 
10.67.0.0/16 via 172.30.0.2 dev MAN1 proto static 
78.94.151.74 via 192.168.178.1 dev wan proto static 
172.16.0.0/24 dev BRADM proto kernel scope link src 172.16.0.1 
172.16.1.0/24 via 172.31.0.2 dev sfp proto static 
172.18.0.0/24 dev MAN2 proto kernel scope link src 172.18.0.1 
172.29.0.0/24 dev wg0 proto static scope link metric 1 
172.29.0.1 dev wg0 proto static scope link metric 1 
172.30.0.0/24 dev MAN1 proto kernel scope link src 172.30.0.1 
172.31.0.0/24 dev sfp proto kernel scope link src 172.31.0.1 
192.168.178.0/24 dev wan proto kernel scope link src 192.168.178.2 

root@RTR_LOC2:~# ip route show
default via xxx.xxx.xxx.xxx dev wan proto static 
10.29.3.0/24 dev lan proto kernel scope link src 10.29.3.1 
10.29.4.0/24 dev DMZ proto kernel scope link src 10.29.4.1 
10.29.6.0/24 dev GUEST proto kernel scope link src 10.29.6.1 
10.29.7.0/24 dev VOIP proto kernel scope link src 10.29.7.1 
10.29.8.0/24 dev CCTV proto kernel scope link src 10.29.8.1 
10.64.0.0/11 via 172.31.0.1 dev lan2 proto static 
10.64.0.0/11 dev wg0 proto static scope link metric 1 
xx.xx.xx.xx/30 dev wan proto kernel scope link src xx.xx.xx.xx 
172.16.0.0/24 via 172.31.0.1 dev lan2 proto static 
172.16.1.0/24 dev ADMBR proto kernel scope link src 172.16.1.1 
172.29.0.0/24 dev wg0 proto static scope link metric 1 
172.29.0.2 dev wg0 proto static scope link metric 1 
172.31.0.0/24 dev lan2 proto kernel scope link src 172.31.0.2

After Stopping MAN on RTR_LOC2:

root@RTR_LOC2:~# ip route show
default via xxx.xxx.xxx.xxx dev wan proto static 
10.29.3.0/24 dev lan proto kernel scope link src 10.29.3.1 
10.29.4.0/24 dev DMZ proto kernel scope link src 10.29.4.1 
10.29.6.0/24 dev GUEST proto kernel scope link src 10.29.6.1 
10.29.7.0/24 dev VOIP proto kernel scope link src 10.29.7.1 
10.29.8.0/24 dev CCTV proto kernel scope link src 10.29.8.1 
10.64.0.0/11 dev wg0 proto static scope link metric 1 
xxx.xxx.xxx.xxx/30 dev wan proto kernel scope link src 78.94.151.74 
172.16.1.0/24 dev ADMBR proto kernel scope link src 172.16.1.1 
172.29.0.0/24 dev wg0 proto static scope link metric 1 
172.29.0.2 dev wg0 proto static scope link metric 1

Routing table for RTR_LOC1 does not change. Though I'd want the route:

10.0.0.0/11 via 172.31.0.2 dev sfp proto static

To be removed. Since the nexthop 172.31.0.2 isn't available since the interface that holds this IP on RTR_LOC2 is Stopped.

lmk if I should remove local routes for the sake of clarity. I left them in for completeness sake for now.

1 Like
6

Since the MAN link is always up on RTR_LOC1, some sort of watchdog is necessary to enable/disable the route dynamically based on the connectivity status.

mwan3 can provide such functionality, but it depends on the legacy iptables and would be an overkill, since a simple scheduled script should be enough to solve the issue:

uci set network.loc2="route"
uci set network.loc2.interface="man"
uci set network.loc2.target="10.0.0.0/11"
uci set network.loc2.gateway="172.31.0.2"
uci commit network
/etc/init.d/network reload
cat << "EOF" > /root/man_watchdog.sh
RT_GW="$(uci -q get network.loc2.gateway)"
if ping -q -c 3 -w 5 "${RT_GW}" &> /dev/null
then RT_OFF="0"
else RT_OFF="1"
fi
uci set network.loc2.disabled="${RT_OFF}"
/etc/init.d/network reload
EOF
cat << "EOF" >> /etc/crontabs/root
* * * * * . /root/man_watchdog.sh
EOF
uci set system.@system[0].cronloglevel="9"
uci commit system
/etc/init.d/cron restart
7

OK, so script it is. Thanks a lot!

1 Like
8

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.