Two routers, two internet connections and high availability?

Hi there, (sorry for my bad english)
I have two Internet connections and was (actually... I AM) using mwan3

There are two apartments:

  • The first one has a cable modem, and a linksys wrt 1200 running openwrt
  • The second one has a fiber modem, and a linksys wrt 1900 running openwrt
  • The apartments are connected by ethernet cable.
  • Both routers see all 4 vlans (vlan 1 for apartment, vlan 2 for the other apartment, 3 for cameras and IoT, 4 for VMs)
  • I'm running keepalived in the routers, so one of them is the active gateway for both apartaments, while the other is stand by

What I want to do?

I wanted to use the router on each apartment to be the active (default) gateway for that apartment AND keep vlan 1 and 2 visible to each other

So, lets say:
Apartment 1 devices will be on 172.16.0.0, with gateway 172.16.0.1
Apartment 2 devices will be on 172.17.0.0, with gateway 172.17.0.1
If 172.16.0.2 ask to connect to internet, ok
If 172.16.0.2 ask to connect to 172.17.0.2, it's not ok... Because the gateways are different, but I want that to work.

So if I lost router from apartment 1, the internet keeps on and vlan comunications too...

It's possible to achieve that?

Do you also have some managed switches in each apartment? Otherwise I cannot imagine how would users in one apartment reach the other apartment when the router in their apartment goes offline.

3 Likes

What I have is something like this...

I suppose you are using VRRP in keepalived, which uses master and backups. You could try to assign R1 as master for apt1 and R2 as backup.
Then with mwan3 you can detect a failed uplink and redirect internet bound traffic from apt1 to R2 to be sent to its uplink. For that it would be easier to have a dedicated vlan among the routers.

I suppose you are using VRRP in keepalived, which uses master and backups. You could try to assign R1 as master for apt1 and R2 as backup.
Then with mwan3 you can detect a failed uplink and redirect internet bound traffic from apt1 to R2 to be sent to its uplink. For that it would be easier to have a dedicated vlan among the routers.

It's hard to explain this in english, lol

Both routers have IP addresses on VLANs 100 and 200, so they can both access the internet.
At the moment you're right, I'm using VRRP in keepalived, so the "VIPs" (one in vlan 1, one in vlan 2) are the default gateway that all devices receive through DHCP (VIP on vlan 1, to devices on vlan 1, and VIP on vlan 2, to devices on vlan 2).
I can make another vrrp instance and separate VIP 1 and VIP 2, so when both are running, VIP 1 should be on Router 1 and VIP 2 on Router 2. If I do this, both vlans (1 and 2) have internet access, but vlan 1 will not be able to talk to vlan 2. I think this is called asymmetric traffic, because PC1 will use it's default gateway (VIP2) to try to reach PC2, which will try to reply using it's default gateway (VIP1) ... no?

1 Like

If you have one VIP on each vlan, then that's fine. The clients will always be talking to the master router for each vlan.
A client will always use the master router VIP as gateway to talk to networks outside of the local net.
But if one router is master for both vlans 1 and 2, then packets will be traversing the Trunk 2 times to reach the internet in the scenario that R2 is master and PC1 is trying to get to internet.

But if one router is master for both vlans 1 and 2, then packets will be traversing the Trunk 2 times to reach the internet in the scenario that R2 is master and PC1 is trying to get to internet.

Yep. If both apartments was adjacent (don't know if this is the correct word), but one is on first floor, the other is on seventh floor, so I don't have the option to connect both routers directly on both modems, and my solution was two ethernet cables between SW01 and SW02 forming a 2 Gbps trunk (bond?)

Talking about a solution... any idea if it's possible to achieve what I asked on first post?

I think that since you have the managed switches in both apartments, you could make a link aggregation. But apart from the bandwidth it is adding some unnecessary latency.

Yes, I don't see the reason not to work.

Nope, just tested.

PC1 using GW1 (on router 1)
PC2 using GW2 (on router 2)
PC1 cannot ping PC2

Disable keepalived on Router 2, so both VIPs (GW 1 and 2) on Router 1
Ping works.

Did you setup the dedicated link between the routers?

What for?
I didnt understand

To pass the packets that should be traversing both routers. Since both are master on a different vlan, they need another link to pass the intervlan packets.

I think I was not able to explain the full cenario.
Both routers are on all 4 vlans, so they can be master.
If I change this, let's say... R1 will not be on vlan 2, and R2 will not be on vlan 1, so when PC1 pings PC2, it will ask R1 to deliver the packet, R1 will use "dedicated link with R2" and ask R2 to deliver packet to PC2, yeah... this will make PC1 ping PC2, BUT... if R1 is down, PC1 will not ping PC2 neither will see the internet, because there is no way for R2 to become master on vlan 1

My assumption was that R1 will be master in vlan1 and backup in vlan2, R2 vice versa. Can't you run different instances for different vlans?

The trunk cabling need to carry all VLANs to both apartments, all the way through to the routers. Then the physical location of the active router would not matter. If the downstairs router is offline, the switch will send all of the downstairs PC traffic upstairs (still on the downstairs VLAN) for routing to the Internet (on VLAN 100 or 200 depending on availability of the Internet connections).

In this sort of failover setup one box is doing all the work and the other one is inactive standing by. It gets a lot more complicated if you want two boxes each doing part of the work (routing their local apartment) then automatically reconfiguring into a different role to take over as the only router should one fail.

Yep... I'm trying now to "force" a static route on "clients", so everybody points to the same router (actually, same IP), for inter-vlan routing.

Yeah. Tested with one Windows PC. For future reference: (those are not my IP addresses)

  • Apartment 1, Router wrt1200, Vlan 1 IP 172.16.0.3, Vlan 2 IP 172.17.0.3
  • Apartment 2, Router wrt1900, Vlan 1 IP 172.16.0.4, Vlan 2 IP 172.17.0.4
  • Keepalived VRRP instance 1 virtual_ipaddress set to 172.16.0.1 and 172.17.0.1
  • Keepalived VRRP instance 2 virtual_ipaddress set to 172.16.0.2 and 172.17.0.2
  • On DHCP config, for vlan 1, pushed default gateway as 172.16.0.1
  • On DHCP config, for vlan 2, pushed default gateway as 172.17.0.2 and dhcp option 121,172.16.0.0/16,172.17.0.1 (this sets a static route to 172.16.0.0/16 via 172.17.0.1)

I think the downside of this is: not all clients gets dhcp option 121

https://techhub.hpe.com/eginfolib/networking/docs/switches/5500hi/5998-5335_hi-avail_cg/content/377975841.htm
The scenario I was trying to explain.
Each router is master in the local vlan. If one router dies, the other takes over.

1 Like

All the time, they talk about "external networks" and "access internet", and I don't see any connection or config between them, except VRRP. So... something is missing.
You can try and build 4 vms, 2 with openwrt and 2 with linux/windows, put both openwrt on vlan 1 and 2, put one linux/win on vlan 1 and the other on vlan 2. If one openwrt is the gateway for one linux/win, and the other is the gateway for the other linux/win, they wont ping each other. You need some other config to acomplish this.
And there is the fact that english is not my native language :sweat_smile:

@luizluca can help? =P

Nothing is missing, it's just not tailor made to your scenario.
In the event that the VRRP instance cannot send anything to an interface where it has backup role, did you try what I told you with the dedicated cross link between the routers?
You can also provide the configurations that you have applied. So far we are talking in theory.

It isn't mine neither, no need to keep repeating that.

1 Like