Mwan3 doesn't work with multiple WAN interfaces in the same subnet

I have 3 routers which connects to the Internets. They are placed into the same VLAN (and same subnet), with IP addresses:

  • Router 1: 192.168.1.201
  • Router 2: 192.168.1.202
  • Router 3: 192.168.1.203

I have another router in the same subnet, and I want to load balancing the connections for my clients.

config interface 'to_router1'
    option  device      'eth0.1'
    option  proto       static
    option  ipaddr      192.168.1.1
    optio   netmask     255.255.255.0
    option  gateway     192.168.1.201
    option  metric      128

config interface 'to_router2'
    option  device      '@to_router1'
    option  proto       static
    option  ipaddr      192.168.1.2
    option  netmask     255.255.255.0
    option  gateway     192.168.1.202
    option  metric      256

config interface 'to_router3'
    option  device      '@to_router1'
    option  proto       static
    option  ipaddr      192.168.1.3
    option  netmask     255.255.255.0
    option  gateway     192.168.1.203
    option  metric      512

I also have a 4th router which belongs to another VLAN/subnet. When doing load balancing between 4 routers, all traffic goes to the 4th router. I tried to force traffic to go to the each of the 3 routers above by editing /etc/config/mwan3, but none of them worked.

root@OpenWrt:~# wget -q -O - http://ipv4.icanhazip.com
Failed to send request: Operation not permitted

Removing Router 2 and Router 3 and it will loads balancing between Router 1 and Router 4 just fine.

It seems mwan3 doesn't like routers in the same subnet, or maybe my /etc/config/network configuration is incorrect? I also tried to replace @to_router1 with eth0.1, but it did not work either.

Ah, I am not the first one who encountered such problem. It had been reported as a bug/feature request: https://github.com/openwrt/packages/issues/15224

It was because mwan3 works per kernel network interface. By defining 2 interfaces in uci, it actually generates a single interface with 2 addresses in the kernel and mwan3 won't work with that. Currently there is no solution yet.

This is generally bad idea. You should not have 3 interfaces in the same subnet.

I know, as it messes up routing a little bit. In reality it doesn't matter through which interface local traffic flows. My production network is a big pile of rubble right now, and there is nothing I can do about it. For now I need to make it work first.

A router routes traffic between multiple networks and should be connected to all the different networks.

Having multiple networks with same network ranges or subnets is not advisable as it creates problems difficult to identify and resolve.

Please configure unique WAN subnets to avoid problems not just with mwan3 but other packages to avoid general routing issues.