MWAN3 setup for specific device preferring one WAN

StanL · March 1, 2023, 12:31am

I see this thread, but to be honest the whole interface vs member vs policy vs rule and metric vs weight things just makes no sense to me.

I have a TV I'd prefer to use on WANb but fallover to WANa as needed.

If I go by the other thread, my current load balance settings aren't correct and I got those from a YouTube video, which was the only thing that explained mwan3 in a way I could work with.

Please advise if you can. My current settings:

        option enabled '1'
        list track_ip '1.0.0.1'
        list track_ip '1.1.1.1'
        list track_ip '208.67.222.222'
        list track_ip '208.67.220.220'
        option family 'ipv4'
        option initial_state 'online'
        option track_method 'ping'
        option reliability '1'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config interface 'wan6'
        list track_ip '2606:4700:4700::1001'
        list track_ip '2606:4700:4700::1111'
        list track_ip '2620:0:ccd::2'
        list track_ip '2620:0:ccc::2'
        option family 'ipv6'
        option enabled '1'
        option initial_state 'online'
        option track_method 'ping'
        option reliability '1'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config interface 'wanb'
        list track_ip '1.0.0.1'
        list track_ip '1.1.1.1'
        list track_ip '208.67.222.222'
        list track_ip '208.67.220.220'
        option family 'ipv4'
        option reliability '1'
        option enabled '1'
        option initial_state 'online'
        option track_method 'ping'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config interface 'wanb6'
        list track_ip '2606:4700:4700::1001'
        list track_ip '2606:4700:4700::1111'
        list track_ip '2620:0:ccd::2'
        list track_ip '2620:0:ccc::2'
        option family 'ipv6'
        option reliability '1'
        option enabled '1'
        option initial_state 'online'
        option track_method 'ping'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config member 'wan_m1_w3'
        option interface 'wan'
        option metric '1'
        option weight '3'

config member 'wanb_m1_w3'
        option interface 'wanb'
        option metric '1'
        option weight '3'

config member 'wan6_m1_w3'
        option interface 'wan6'
        option metric '1'
        option weight '3'

config member 'wanb6_m1_w3'
        option interface 'wanb6'
        option metric '1'
        option weight '3'

config policy 'balanced'
        list use_member 'wan_m1_w3'
        list use_member 'wanb_m1_w3'
        list use_member 'wan6_m1_w3'
        list use_member 'wanb6_m1_w3'
        option last_resort 'unreachable'

config rule 'https'
        option sticky '1'
        option dest_port '443'
        option proto 'tcp'
        option use_policy 'balanced'

config rule 'default_rule_v4'
        option dest_ip '0.0.0.0/0'
        option use_policy 'balanced'
        option family 'ipv4'

config rule 'default_rule_v6'
        option dest_ip '::/0'
        option use_policy 'balanced'
        option family 'ipv6'

StanL · March 2, 2023, 1:19am

This seems right... maybe. I think it's working, but I did have a bout with YouTube lagging badly until I disconnected wana from the router. Dunno.

config globals 'globals'
	option mmx_mask '0x3F00'

config interface 'wan'
        option enabled '1'
        list track_ip '1.0.0.1'
        list track_ip '1.1.1.1'
        list track_ip '208.67.222.222'
        list track_ip '208.67.220.220'
        option family 'ipv4'
        option initial_state 'online'
        option track_method 'ping'
        option reliability '1'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config interface 'wan6'
        list track_ip '2606:4700:4700::1001'
        list track_ip '2606:4700:4700::1111'
        list track_ip '2620:0:ccd::2'
        list track_ip '2620:0:ccc::2'
        option family 'ipv6'
        option enabled '1'
        option initial_state 'online'
        option track_method 'ping'
        option reliability '1'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config interface 'wanb'
        list track_ip '1.0.0.1'
        list track_ip '1.1.1.1'
        list track_ip '208.67.222.222'
        list track_ip '208.67.220.220'
        option family 'ipv4'
        option reliability '1'
        option enabled '1'
        option initial_state 'online'
        option track_method 'ping'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config interface 'wanb6'
        list track_ip '2606:4700:4700::1001'
        list track_ip '2606:4700:4700::1111'
        list track_ip '2620:0:ccd::2'
        list track_ip '2620:0:ccc::2'
        option family 'ipv6'
        option reliability '1'
        option enabled '1'
        option initial_state 'online'
        option track_method 'ping'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'

config member 'wan_m1_w3'
        option interface 'wan'
        option metric '1'
        option weight '3'

config member 'wanb_m1_w3'
        option interface 'wanb'
        option metric '1'
        option weight '3'

config member 'wan6_m1_w3'
        option interface 'wan6'
        option metric '1'
        option weight '3'

config member 'wanb6_m1_w3'
        option interface 'wanb6'
        option metric '1'
        option weight '3'

config member 'wan_m2_w3'
	option interface 'wan'
	option metric '2'
	option weight '3'

config policy 'wanb_first'
	list use_member 'wan_m2_w3'
	list use_member 'wanb_m1_w3'
	option last_resort 'unreachable'

config policy 'balanced'
        list use_member 'wan_m1_w3'
        list use_member 'wanb_m1_w3'
        list use_member 'wan6_m1_w3'
        list use_member 'wanb6_m1_w3'
        option last_resort 'unreachable'

config rule 'https'
        option sticky '1'
        option dest_port '443'
        option proto 'tcp'
        option use_policy 'balanced'

config rule 'sony_tv'
	option src_ip '192.168.3.239/24'
	option proto 'all'
	option sticky '0'
	option use_policy 'wanb_first'

config rule 'default_rule_v4'
        option dest_ip '0.0.0.0/0'
        option use_policy 'balanced'
        option family 'ipv4'

config rule 'default_rule_v6'
        option dest_ip '::/0'
        option use_policy 'balanced'
        option family 'ipv6'

pavelgl · March 2, 2023, 9:26am

Remove the suffix (/24) from the src_ip option and move the sony_tv rule above the https one.

Currently, the rule covers not only the TV, but the entire 192.168.3.0/24 subnet, and every https request uses the balanced policy.

StanL · March 2, 2023, 5:49pm

Thanks. That probably explains the occasional YouTube lagging and why every other device seems to be connecting to wana when I check the IP address at https://speedtest.net.

config rule 'sony_tv'
        option proto 'all'
        option sticky '0'
        option use_policy 'wanb_first'
        option src_ip '192.168.3.239'

config rule 'https'
        option sticky '1'
        option dest_port '443'
        option proto 'tcp'
        option use_policy 'balanced'

config rule 'default_rule_v4'
        option dest_ip '0.0.0.0/0'
        option use_policy 'balanced'
        option family 'ipv4'

config rule 'default_rule_v6'
        option dest_ip '::/0'
        option use_policy 'balanced'
        option family 'ipv6'

Now everything seems to be connecting to wana EVEN the TV. If I disconnect wana, failover works, but that's not a balance or a preference for the TV.

Does the metric of the network interface effect the load balance? It seems there are two metrics for each wan when using mwan3.

config interface 'wan'
        option device 'wan'
        option proto 'dhcp'
        option broadcast '1'
        **option metric '10'**

config interface 'wan6'
        option device 'wan'
        option proto 'dhcpv6'
        option reqaddress 'try'
        option reqprefix 'auto'
        **option metric '30'**

config interface 'wanb'
        option device 'br-lan.9'
        option proto 'dhcp'
        **option metric '20'**

config interface 'wanb6'
        option device 'br-lan.9'
        option proto 'dhcpv6'
        option reqaddress 'try'
        option reqprefix 'auto'
        **option metric '40'**

ALTHOUGH, there is no wanb6. It's a place holder, but the ISP doesn't seem to be providing it. Maybe that's the problem?

pavelgl · March 3, 2023, 7:45am

Run mwan3 rules to check the active rules and their usage based on the counters on the left.

Set a higher weight for wanb and test again.

From the guide:

Run mwan3 interfaces.

For wanb6 you should see something like interface wanb6 is offline...
That shouldn't cause problems.

EDIT:

Also, disable the https rule during tests, or at least set option sticky to 0.

StanL · March 3, 2023, 12:33pm

I'll give those a try later today. I can tell you that, yes, wanb6 shows as offline and MultiWAN Manager-Status shows:

Current ipv4 policies:
balanced:
 wanb (50%)
 wan (50%)

Current ipv6 policies:
balanced:
 wan6 (100%)

If I give a higher weight to WANB is that for all devices or just the TV?
To try to unconfuse myself: lower metric has priority, but higher weight has priority within a metric?
Should there also be a IPv6 rule for the TV?

config member 'wan_m2_w3'
	option interface 'wan6'
	option metric '2'
	option weight '3'

config policy 'wanb6_first'
	list use_member 'wan6_m2_w3'
	list use_member 'wanb6_m1_w3'
	option last_resort 'unreachable'

config rule 'sony_tv'
        option proto 'all'
        option sticky '0'
        option use_policy 'wanb6_first'
        option src_ip '192.168.3.239'

Just as a safety net in case the ISP gets their act together?

-Heck. Should there also be a HTTPS rule for the TV?

pavelgl · March 3, 2023, 2:30pm

Something is missing here.

This should create a second policy.

wanb_first:
 wanb (100%)

That's for the rest of the devices.
The TV should use the first rule in the list (sony_tv) if it and the wanb_first policy exist.

Do you see such a rule looking at the status?

... wanb_first  all  --  *      *       192.168.3.239        0.0.0.0/0

Again from the gude:

The wanb_first policy should always use wanb because it has a lower metric. wan will only be used if wanb goes down. In this case, the weight parameter doesn't matter and there is no load balancing, but failover.

The weight value determines how often the corresponding interface will be used during load balancing:

config member 'wan_m1_w1'
        option interface 'wan'
        option metric '1'
        option weight '1'

config member 'wanb_m1_w1'
        option interface 'wanb'
        option metric '1'
        option weight '1'

config policy 'balanced'
        list use_member 'wan_m1_w1'
        list use_member 'wanb_m1_w1'
        
# Result:

balanced:
 wanb (50%)
 wan (50%)

config member 'wan_m1_w1'
        option interface 'wan'
        option metric '1'
        option weight '1'

config member 'wanb_m1_w3'
        option interface 'wanb'
        option metric '1'
        option weight '3'

config policy 'balanced'
        list use_member 'wan_m1_w1'
        list use_member 'wanb_m1_w3'
        
# Result:

balanced:
 wanb (75%)
 wan (25%)

I don't think so. I'm not familiar with Sony TVs, but I seriously doubt they support dual-stack.
In addition, wanb6 is always down.

No.

StanL · March 4, 2023, 3:59am

I went through a LOT of trial and error. I found rules and policies disappearing from the status when I switched between plain load balancing and adding the TV preference
I finally decided to re-add the TV rules from scratch and in a hurry. I took some short cuts, which is when I found out what I was doing wrong.

You see, the names I've posted in this thread are modified for privacy.
The ACTUAL policy name I was using was 16 characters long (and included the name of my ISP), not 15 or less, which is the limit. So the name was breaking everything.

Names may contain characters A-Z, a-z, 0-9, _ and no spaces.Names must be 15 characters or less.

I added the name with LuCI, but it didn't check that the policy name didn't meet the criteria listed? That's just a bug that needs fixing.
Interface, Member, Policy, Rules all should check and throw an error if you go out of bounds.
Heck, only Policies have the 15 character limit (according to LuCI), but not the other settings.

So it's all fixed. Thanks for all the help.
Btw, do you think it would make a difference if I used
option src_ip '192.168.3.239/32'
vs
option src_ip '192.168.3.239?

pavelgl · March 4, 2023, 8:08am

Glad you have it resolved.
Please close the topic by adding [Solved] to the left of the title as described here.

No, not in this case.

Adding /32 (which represents a single host) is good practice, but if omitted, it will be accepted by most services. However, if you create an ip rule and omit the /32, you will receive an unpleasant surprise.

StanL · March 4, 2023, 2:21pm

So an mwan3 rule reacts different than an IP rule?

StanL · March 7, 2023, 12:54am

Interestingly, I made some changes today, changed my mind and then reverted them. The TV, even after rebooting the router twice was still connected to WAN instead of WANb where it belonged.

I stopped WAN under interfaces and found the TV was still streaming video from that ISP, I assume over WAN6.

I finally disconnected the cable of the modem, reattached it and things for the moment are okay. I have no idea why it wouldn't connect to WANb where it belonged. Also, surprised it was probably using IPv6.

StanL · March 7, 2023, 12:13pm

The more I think I've learn about OpenWrt, the more I find I've screwed up and know nothing.

To recap: WANa has a slower top speed, but has IPv6 and is more reliable. WANb has a faster top speed but no IPv6.
I want my living room TV to have the fastest speed possible for streaming (WANb) with failover to WANa.

I THOUGHT all the stuff above got that working until yesterday when I discovered it was connecting to IPv6... which is WANa.

What I don't think I realized is that the device decides if it's going to use IPv4 or IPv6, then mwan3 chooses the WAN.

That means I can't even have IPv6 failover for the TV, because any device choosing IPv6 will 100% of the time use WANa.

In fact, the TV doesn't lease an IPv6 address, it has hidden static addresses that I had to follow it's MAC address from it's IPv4 in the routing status to find the FOUR IPv6 address it was connecting with.

It would be nice if mwan3 let you choose a device based on its MAC address. It would make things a bit easier. It would also be nice if WANs were balanced as a whole, not IPv4 and 6 separately.

So, here is my CURRENT setup, if any of this is wrong or poorly done, don't be afraid to insult the crap out of my stupidity with this stuff.
(Btw, dunno if it was safer to blank the IPv6 addresses or not since they're local, but I did anyway since they may be unique to the device.)

Oh, and yeah, I still have a balance policy, but I'm not currently using it.


config globals 'globals'
	option mmx_mask '0x3F00'

config interface 'wan'
	option enabled '1'
	option family 'ipv4'
	option initial_state 'online'
	option track_method 'ping'
	option reliability '1'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	list track_ip '1.0.0.1'
	list track_ip '1.1.1.1'

config interface 'wan6'
	option family 'ipv6'
	option enabled '1'
	option initial_state 'online'
	option track_method 'ping'
	option reliability '1'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	list track_ip '2606:4700:4700::1001'
	list track_ip '2606:4700:4700::1111'

config interface 'wanb'
	option family 'ipv4'
	option reliability '1'
	option enabled '1'
	option initial_state 'online'
	option track_method 'ping'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	list track_ip '1.0.0.1'
	list track_ip '1.1.1.1'

config interface 'wanb6'
	option family 'ipv6'
	option reliability '1'
	option enabled '1'
	option initial_state 'online'
	option track_method 'ping'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	list track_ip '2606:4700:4700::1001'
	list track_ip '2606:4700:4700::1111'

config member 'wan_m1_w3'
	option interface 'wan'
	option metric '1'
	option weight '3'

config member 'wanb_m1_w3'
	option interface 'wanb'
	option metric '1'
	option weight '3'

config member 'wan6_m1_w3'
	option interface 'wan6'
	option metric '1'
	option weight '3'

config member 'wanb6_m1_w3'
	option interface 'wanb6'
	option metric '1'
	option weight '3'

config policy 'balanced'
	list use_member 'wan_m1_w3'
	list use_member 'wanb_m1_w3'
	list use_member 'wan6_m1_w3'
	list use_member 'wanb6_m1_w3'
	option last_resort 'unreachable'

config rule 'sony_tv61'
	option family 'ipv6'
	option proto 'all'
	option src_ip '2600:::::::'
	option sticky '0'
	option use_policy 'unreachable'

config rule 'sony_tv62'
	option family 'ipv6'
	option proto 'all'
	option src_ip '2600:::::::'
	option sticky '0'
	option use_policy 'unreachable'

config rule 'sony_tv63'
	option family 'ipv6'
	option proto 'all'
	option src_ip 'fd82:::::::'
	option sticky '0'
	option use_policy 'unreachable'

config rule 'sony_tv64'
	option family 'ipv6'
	option proto 'all'
	option src_ip 'fd82::::::'
	option sticky '0'
	option use_policy 'unreachable'

config rule 'sony_tv'
	option proto 'all'
	option src_ip '192.168.3.239'
	option sticky '0'
	option use_policy 'cable_first'
	option family 'ipv4'

config rule 'https'
	option sticky '1'
	option dest_port '443'
	option proto 'tcp'
	option use_policy '60_40'

config rule 'default_rule_v4'
	option dest_ip '0.0.0.0/0'
	option family 'ipv4'
	option proto 'all'
	option sticky '0'
	option use_policy '60_40'

config rule 'default_rule_v6'
	option dest_ip '::/0'
	option family 'ipv6'
	option proto 'all'
	option sticky '0'
	option use_policy '60_40'

config member 'wan_m2_w3'
	option interface 'wan'
	option metric '2'
	option weight '3'

config policy 'cable_first'
	list use_member 'wan_m2_w3'
	list use_member 'wanb_m1_w3'
	option last_resort 'unreachable'

config member 'wan_m1_w2'
	option interface 'wan'
	option metric '1'
	option weight '2'

config member 'wan6_m1_w2'
	option interface 'wan6'
	option metric '1'
	option weight '2'

config policy '60_40'
	list use_member 'wan_m1_w2'
	list use_member 'wan6_m1_w2'
	list use_member 'wanb_m1_w3'
	list use_member 'wanb6_m1_w3'
	option last_resort 'unreachable'

EDIT: Would cable_first be better configured with last_resort being 'default'? Seems like it means even with all the IPv6's blocked first, if WAN and WANb failed then WAN6 would become an option again.

StanL · March 8, 2023, 7:49pm

Annnnd this has failed. Apparently the IPv6 addresses AREN'T static, even though it's not leasing the addresses from the router. I can think of no way to fix this without completely scrapping ipv6 support on the router.

Edit: Tried changing mwan3. Removed all IPv6 rules and members from the policies. Tried last resort as default and unreachable and nothing will stop devices that WANT IPv6 from using the WAN with IPv6.

The only option seems to be a tunnel or disabling IPv6 completely. Both ideas are a let down.

I could be doing something wrong, but if no one has a useful suggestion, tomorrow I'm reflashing the router to OEM and returning it. I give up.

Is it possible to set up a wifi that's connected to LAN, but only can access WANb? It's the only other answer I can think of.

trendy · March 8, 2023, 8:56pm

NAT66 is another option, not too elegant though.

StanL · March 9, 2023, 4:38am

Tonight I swapped which router was load balancing. The one I'm currently using is FreshTomato on an 11 year old Linksys.

When you select a device to prefer one WAN, you can enter the MAC address of the device. I still need to test if fail over works, but why the heck doesn't mwan3 have that feature?