Mwan3 equal cost loadbalancing not working

[Knogle]

Ahoy friends.
Currently i got a x86 system running with OpenWRT. Also 2 Dual Stack WAN connections with IPv4 static public address and IPv6.
They are running fine.
But now i'd like to load balance the traffic, because both WAN links have the same bandwidth.
I've performed all the steps mentioned in the OpenWRT mwan3 page, but unfortunately it didn't work well.
So i've reset everything to default settings, but sometimes it's not possible to resolve.
That's the settings i use, i'm curious about some things.
I've got the warning: WARNING: Interface wanb6 has no default route in the main routing table
WARNING: Interface wan6 has no default route in the main routing table

Do i have to use different metrics for both WAN interfaces on interfaces tab, or do i have to do any further changes?
Does someone have some working config for equal cost load balancing with 2 WAN interfaces?

Which routing table do i have to choose? Routing table 1, or 2? Or 220? What does it mean?
I hope someone can give me some advice in order to get it working!

Thanks in advance!

[trendy]

@Knogle

Do i have to use different metrics for both WAN interfaces on interfaces tab, or do i have to do any further changes?

It is a prerequisite for the interfaces to have different metrics before you start configuring mwan3.

@Knogle

Does someone have some working config for equal cost load balancing with 2 WAN interfaces?

It is in the mentioned in the wiki. As long as your rules use policy balanced, the load is distributed between two interfaces in a 3/2 ratio.

@Knogle

Which routing table do i have to choose? Routing table 1, or 2? Or 220? What does it mean?

That should not be used, if you are not using more routing tables, usually from other programs like vpn-policy-routing.

[Knogle]

Thanks for your help, is it recommended to use the LuCi interface? I've copied the config shown in the docs page, so i'll try it out. Btw. in the LuCi interface there are several policies, do i have to delete all of the necessary ones, and only keep "balanced"? Or do i have to move them up and down?

[trendy]

@Knogle

is it recommended to use the LuCi interface?

You can of course, I don't think there is any restriction.

@Knogle

I've copied the config shown in the docs page, so i'll try it out.

If I am not mistaken, it is already included as the default one.

@Knogle

Btw. in the LuCi interface there are several policies, do i have to delete all of the necessary ones, and only keep "balanced"? Or do i have to move them up and down?

No, just make a rule with policy balanced and drag it to the top.

[Knogle]

Thanks, i tried that, using the default config, and moving "balanced" to the top. But unfortunately the traffic is not being balanced.
Also what does the ipv6 routing table error mean?

[trendy]

@Knogle

moving "balanced" to the top

The rule and not the policy I hope.

@Knogle

But unfortunately the traffic is not being balanced.

Did you try on a multiple connection protocol like torrent or download manager? Or are you expecting that the youtube will utilize both connections?

@Knogle

Also what does the ipv6 routing table error mean?

I think it is self explaining. Is there the default route from both ipv6 interfaces in the main routing table? They must be both there with different metrics.

[leeandy]

You missed of this step:

Configure a different metric for each WAN interface

This is an important step and is compulsory. Time and time again fail to configure this and have a none working setup.

• You must configure each WAN interface with a different routing metric. This metric will only have an effect on the default routing table, not on the mwan3 routing tables.

• The default (primary) WAN interface should have the lowest metric (e.g. 10) and each additional WAN interface a higher metric (e.g. 20, 30, etc.). Values are not important, but should always be unique.

• Every WAN interface should have a default gateway configured.

[Knogle]

Thanks, it's working now!
Btw. how can i get the IPv6 interfaces work? Both are DHCPv6 clients, wan6 and wanb6, but according to mwan3 both don't have a default route configured in the main routing table. How can i do so for the wan6 and wanb6 interfaces?

[leeandy]

If your internet connection is dual stack type, you just need to configure interfaces / members / policies / rules same as ipv4 configuration, no need to worry about the default route / metric.

[Knogle]

Yeah the problem is, i cannot assign a metric to the wan6 and wanb6 interface, because they are DHCPv6 clients.

[trendy]

You can:

root@magiatiko:[/]#uci show network.wan6
network.wan6=interface
network.wan6.proto='dhcpv6'
network.wan6.peerdns='0'
network.wan6.ifname='@wan'
network.wan6.metric='20'
network.wan6.sourcerouting='0'
network.wan6.reqprefix='56'
network.wan6.reqaddress='try'

[Knogle]

Thanks for your advice, using this way i was able to assign a metric to these interfaces, in LuCi there was no option given.
Well i got the metrics assigned, and everything else in mwan3, unfortunately mwan3 says wan6 and wanb6 are diasbled.

IP has changed after my restarts, so i didn't remove it from the screenshots.

[trendy]

Do a mwan3 status; ip -6 ro li tab all

[Knogle]

@trendy

mwan3 status; ip -6 ro li tab all

root@OpenWrt:~# mwan3 status; ip -6 ro li tab all
Interface status:
 interface wan is online 01h:22m:44s, uptime 01h:22m:57s and tracking is active
 interface wan6 is offline and tracking is paused
 interface wanb is online 01h:22m:44s, uptime 01h:22m:56s and tracking is active
 interface wanb6 is offline and tracking is paused

Current ipv4 policies:
balanced:
 wanb (50%)
 wan (50%)
wan_only:
 wan (100%)
wan_wanb:
 wan (100%)
wanb_only:
 wanb (100%)
wanb_wan:
 wanb (100%)

Current ipv6 policies:
balanced:
 unreachable
wan_only:
 unreachable
wan_wanb:
 unreachable
wanb_only:
 unreachable
wanb_wan:
 unreachable

Directly connected ipv4 networks:
78.35.145.32
78.35.146.125
172.20.32.0/19
172.20.192.0/19
172.21.32.0/19
172.21.0.0/19
127.0.0.0/8
192.168.200.0/24
172.20.160.0/19
192.168.111.0/24
195.14.226.161
224.0.0.0/3
172.20.96.0/19
172.20.224.0/19
172.20.64.0/19

Directly connected ipv6 networks:
fdfb:9584:eb33:30::/64
2001:4dd0:4ed6:20::/64
2001:4dd0:4ed6:10::/64
fdfb:9584:eb33::/64
2001:4dd0:5033:20::/64
2001:4dd0:5033::/64
2001:4dd0:5033:10::/64
2001:4dd0:5033:30::/64
2001:4dd0:af0e:4ed6::/64
fe80::/64
fdfb:9584:eb33:20::/64
fe80::15ae:79b2:4399:7882
fdfb:9584:eb33:10::/64
2001:4dd0:af0e:5033::/64
2001:4dd0:4ed6::/64
fe80::200:ff:fe00:0
fe80::cc19:b4f:8ad:b78f
2001:4dd0:4ed6:30::/64

Active ipv4 user rules:
 2782  154K S https  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 443 
 4895  812K - balanced  all  --  *      *       0.0.0.0/0            0.0.0.0/0            

Active ipv6 user rules:
 1421  108K S https  tcp      *      *       ::/0                 ::/0                 multiport dports 443 
 7747  745K - balanced  all      *      *       ::/0                 ::/0                 

default from 2001:4dd0:4ed6::/48 via fe80::200:ff:fe00:0 dev pppoe-wan proto static metric 512 pref medium
default from 2001:4dd0:5033::/48 via fe80::200:ff:fe00:0 dev pppoe-wanb proto static metric 512 pref medium
default from 2001:4dd0:af0e:4ed6::/64 via fe80::200:ff:fe00:0 dev pppoe-wan proto static metric 512 pref medium
default from 2001:4dd0:af0e:5033::/64 via fe80::200:ff:fe00:0 dev pppoe-wanb proto static metric 512 pref medium
2001:4dd0:4ed6::/64 dev br-Lab proto static metric 1024 pref medium
2001:4dd0:4ed6:10::/64 dev br-Trusted proto static metric 1024 pref medium
2001:4dd0:4ed6:20::/64 dev br-VMs proto static metric 1024 pref medium
2001:4dd0:4ed6:30::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2001:4dd0:4ed6::/48 dev lo proto static metric 2147483647 pref medium
2001:4dd0:5033::/64 dev br-Lab proto static metric 1024 pref medium
2001:4dd0:5033:10::/64 dev br-Trusted proto static metric 1024 pref medium
2001:4dd0:5033:20::/64 dev br-VMs proto static metric 1024 pref medium
2001:4dd0:5033:30::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2001:4dd0:5033::/48 dev lo proto static metric 2147483647 pref medium
2001:4dd0:af0e:4ed6::/64 dev pppoe-wan proto static metric 256 pref medium
2001:4dd0:af0e:5033::/64 dev pppoe-wanb proto static metric 256 pref medium
fdfb:9584:eb33::/64 dev br-Lab proto static metric 1024 pref medium
fdfb:9584:eb33:10::/64 dev br-Trusted proto static metric 1024 pref medium
fdfb:9584:eb33:20::/64 dev br-VMs proto static metric 1024 pref medium
fdfb:9584:eb33:30::/64 dev br-lan proto static metric 1024 pref medium
unreachable fdfb:9584:eb33::/48 dev lo proto static metric 2147483647 pref medium
fe80::200:ff:fe00:0 dev pppoe-wan metric 1 pref medium
fe80::200:ff:fe00:0 dev pppoe-wanb metric 1 pref medium
fe80::15ae:79b2:4399:7882 dev pppoe-wan proto kernel metric 256 pref medium
fe80::cc19:b4f:8ad:b78f dev pppoe-wanb proto kernel metric 256 pref medium
fe80::/64 dev eth6 proto kernel metric 256 pref medium
fe80::/64 dev eth5 proto kernel metric 256 pref medium
fe80::/64 dev eth4 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev eth0.4 proto kernel metric 256 pref medium
fe80::/64 dev eth0.220 proto kernel metric 256 pref medium
fe80::/64 dev eth0.300 proto kernel metric 256 pref medium
fe80::/64 dev eth0.320 proto kernel metric 256 pref medium
fe80::/64 dev eth0.340 proto kernel metric 256 pref medium
fe80::/64 dev br-Lab proto kernel metric 256 pref medium
fe80::/64 dev br-Management proto kernel metric 256 pref medium
fe80::/64 dev br-Trusted proto kernel metric 256 pref medium
fe80::/64 dev br-VMs proto kernel metric 256 pref medium
local ::1 dev lo table local proto kernel metric 0 pref medium
anycast 2001:4dd0:4ed6:: dev br-Lab table local proto kernel metric 0 pref medium
local 2001:4dd0:4ed6::1 dev br-Lab table local proto kernel metric 0 pref medium
anycast 2001:4dd0:4ed6:10:: dev br-Trusted table local proto kernel metric 0 pref medium
local 2001:4dd0:4ed6:10::1 dev br-Trusted table local proto kernel metric 0 pref medium
anycast 2001:4dd0:4ed6:20:: dev br-VMs table local proto kernel metric 0 pref medium
local 2001:4dd0:4ed6:20::1 dev br-VMs table local proto kernel metric 0 pref medium
anycast 2001:4dd0:4ed6:30:: dev br-lan table local proto kernel metric 0 pref medium
local 2001:4dd0:4ed6:30::1 dev br-lan table local proto kernel metric 0 pref medium
anycast 2001:4dd0:5033:: dev br-Lab table local proto kernel metric 0 pref medium
local 2001:4dd0:5033::1 dev br-Lab table local proto kernel metric 0 pref medium
anycast 2001:4dd0:5033:10:: dev br-Trusted table local proto kernel metric 0 pref medium
local 2001:4dd0:5033:10::1 dev br-Trusted table local proto kernel metric 0 pref medium
anycast 2001:4dd0:5033:20:: dev br-VMs table local proto kernel metric 0 pref medium
local 2001:4dd0:5033:20::1 dev br-VMs table local proto kernel metric 0 pref medium
anycast 2001:4dd0:5033:30:: dev br-lan table local proto kernel metric 0 pref medium
local 2001:4dd0:5033:30::1 dev br-lan table local proto kernel metric 0 pref medium
anycast 2001:4dd0:af0e:4ed6:: dev pppoe-wan table local proto kernel metric 0 pref medium
local 2001:4dd0:af0e:4ed6:15ae:79b2:4399:7882 dev pppoe-wan table local proto kernel metric 0 pref medium
anycast 2001:4dd0:af0e:5033:: dev pppoe-wanb table local proto kernel metric 0 pref medium
local 2001:4dd0:af0e:5033:cc19:b4f:8ad:b78f dev pppoe-wanb table local proto kernel metric 0 pref medium
anycast fdfb:9584:eb33:: dev br-Lab table local proto kernel metric 0 pref medium
local fdfb:9584:eb33::1 dev br-Lab table local proto kernel metric 0 pref medium
anycast fdfb:9584:eb33:10:: dev br-Trusted table local proto kernel metric 0 pref medium
local fdfb:9584:eb33:10::1 dev br-Trusted table local proto kernel metric 0 pref medium
anycast fdfb:9584:eb33:20:: dev br-VMs table local proto kernel metric 0 pref medium
local fdfb:9584:eb33:20::1 dev br-VMs table local proto kernel metric 0 pref medium
anycast fdfb:9584:eb33:30:: dev br-lan table local proto kernel metric 0 pref medium
local fdfb:9584:eb33:30::1 dev br-lan table local proto kernel metric 0 pref medium
anycast fe80:: dev eth6 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth4 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth5 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0.320 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0.220 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0.4 table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0.300 table local proto kernel metric 0 pref medium
anycast fe80:: dev br-lan table local proto kernel metric 0 pref medium
anycast fe80:: dev eth0.340 table local proto kernel metric 0 pref medium
anycast fe80:: dev br-Management table local proto kernel metric 0 pref medium
anycast fe80:: dev br-Trusted table local proto kernel metric 0 pref medium
anycast fe80:: dev br-Lab table local proto kernel metric 0 pref medium
anycast fe80:: dev br-VMs table local proto kernel metric 0 pref medium
local fe80::202:c9ff:fe06:a526 dev eth6 table local proto kernel metric 0 pref medium
local fe80::21b:78ff:fe57:bb0e dev eth4 table local proto kernel metric 0 pref medium
local fe80::21b:78ff:fe57:bb0f dev eth5 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev eth0.320 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev eth0 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev eth0.220 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev eth0.4 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev eth0.300 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev br-lan table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev eth0.340 table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev br-Management table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev br-Trusted table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev br-Lab table local proto kernel metric 0 pref medium
local fe80::ec4:7aff:feda:b5d6 dev br-VMs table local proto kernel metric 0 pref medium
local fe80::15ae:79b2:4399:7882 dev pppoe-wan table local proto kernel metric 0 pref medium
local fe80::cc19:b4f:8ad:b78f dev pppoe-wanb table local proto kernel metric 0 pref medium
ff00::/8 dev br-Lab table local metric 256 pref medium
ff00::/8 dev eth6 table local metric 256 pref medium
ff00::/8 dev br-Trusted table local metric 256 pref medium
ff00::/8 dev br-VMs table local metric 256 pref medium
ff00::/8 dev br-lan table local metric 256 pref medium
ff00::/8 dev eth5 table local metric 256 pref medium
ff00::/8 dev eth4 table local metric 256 pref medium
ff00::/8 dev eth0 table local metric 256 pref medium
ff00::/8 dev eth0.4 table local metric 256 pref medium
ff00::/8 dev eth0.220 table local metric 256 pref medium
ff00::/8 dev eth0.300 table local metric 256 pref medium
ff00::/8 dev eth0.320 table local metric 256 pref medium
ff00::/8 dev eth0.340 table local metric 256 pref medium
ff00::/8 dev br-Management table local metric 256 pref medium
ff00::/8 dev pppoe-wan table local metric 256 pref medium
ff00::/8 dev pppoe-wanb table local metric 256 pref medium

[trendy]

I can only speculate here that the common default gateway address in both wan6 interfaces might be causing the offline status and paused tracking.
@aaronjg your insight would be helpful here.

[aaronjg]

The common gateway would not cause the interface to be paused.

The device would show up as paused if:

1. network_get_device is unable to find the device associated with the interface
2. or network_is_up is false.

It looks like your WAN6 and WANB6 are not up. WAN and WANB only a link local IPv6 address. Perhaps you need to change the mwan3 IPV6 interfaces to WAN_6 and WANB_6.

If that doesn't help, you can try to add a bit more logging to help diagnose. Unfortunately, there isn't a good log point in mwan3track for this at the moment. Could you add:

		LOG notice "firstconnect: called on $INTERFACE/$true_iface ($DEVICE). Status is $STATUS."

around line 150 of mwan3track (in the firstconnect function between disabled and return) and then look at the system log and report the corresponding log line?

[aaronjg]

Once you get the interfaces set up correctly, you will need to set up something to translate the LAN device’s ipv6 address to match the outgoing interface. Here is a guide for this:

Using NAT66 with mwan3 on OpenWrt

[jamesmacwhite]

mwan3 and IPv6 has a few caveats on 19.07.x. As others have said metrics are important first and foremost. Ignore the fact that LuCI shows errors related to default route. This is what mine looks like

It's actually more of a false error though, as I don't think luci-app-mwan3 really handles IPv6 interfaces well currently.

@aaronjg Posted something I wrote a while back when I was investigating having IPv6 working with mwan3. It is possible to have mwan3 and IPv6 work, but you will likely need to enable IPv6 masquerading, you will possibly also encounter this specific issue with IPv6 as well:

Ping and traceroute failing for eth0.3 on IPv6

Try this way: ip -6 route get 1:: ip -6 route get 1:: dev pppoe-wan ip -6 route get 1:: dev pppoe-wan from 2804:1b2:206:520:a9f2:ca03:4dec:7c71 ip -6 route get 1:: dev eth0.3 ip -6 route get 1:: dev eth0.3 from 2804:14c:658b:1000:45d7:3d14:cfdc:66cd ping6 -c 3 example.org ping6 -c 3 -I 2804:1b2:206:520:a9f2:ca03:4dec:7c71 example.org ping6 -c 3 -I 2804:14c:658b:1000:45d7:3d14:cfdc:66cd example.org It seems that in some scenarios OpenWrt can't correctly select the source IPv6 address for routin…

[aaronjg]

But it looks like he is on snapshot, and the 2.10 release should have fixed the source IP issue you mentioned.

Of course, do keep an eye on it and let me know if the issue persists.

[jamesmacwhite]

Sorry, didn't know if it was snapshot or stable build. Should of looked at the build string on the screenshots (derp).