MWAN3 always offline

I'm encountering an issue with my MWAN3 setup where the WAN2 interface goes offline after being online for the first few seconds or minutes. I have two connections from my prepaid WiFi, and I'm trying to load balance them to achieve a faster connection. Sometimes one connection is slow while the other is fast, and vice versa.

  • Router: MF283+
  • Openwrt version: 23.05.3

Configuration Details:

  1. Interfaces:
    • WAN: Configured on the WAN port
    • LAN1: Configured on LAN1 port using VLAN 2

Note i tried both interfaces using ping from diagnostics and in the mwan manager diagnostics and its always up.

  1. Network Configuration:

    config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'
    
    config globals 'globals'
        option ula_prefix 'fd86:21cb:520e::/48'
    
    config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0.1'
    
    config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.2.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
    
    config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'
    
    config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '6t 2 3'
        option description 'LAN'
    
    config device
        option name 'phy0-sta0'
    
    config interface 'wan'
        option proto 'dhcp'
        option device 'eth0.2'
        option metric '1'
        option type 'bridge'
    
    config switch_vlan
        option device 'switch0'
        option ports '6t 0'
        option vlan '2'
        option description 'WAN'
    
    config switch_vlan
        option device 'switch0'
        option ports '6t 1'
        option vlan '3'
        option description 'WAN 2'
    
    config interface 'wan2'
        option proto 'dhcp'
        option device 'eth0.3'
        option metric '2'
        option type 'bridge'
    
    config interface 'wan3'
        option proto 'dhcp'
        option device 'usb0'
        option metric '2'
        option type 'bridge'
    
    config device
        option name 'wwan0'
    
  2. MWAN3 Configuration:

     config globals 'globals'
         option mmx_mask '0x3F00'
    
     config interface 'wan'
         option enabled '1'
         option family 'ipv4'
         option reliability '1'
         option initial_state 'online'
         option track_method 'ping'
         option count '1'
         option size '56'
         option max_ttl '60'
         option timeout '4'
         option interval '10'
         option failure_interval '5'
         option recovery_interval '5'
         option down '5'
         option up '5'
         list track_ip '8.8.8.8'
         list track_ip '8.8.4.44'
         list flush_conntrack 'ifup'
    
     config member 'wan_m1_w3'
         option interface 'wan'
         option metric '1'
         option weight '3'
    
     config policy 'balanced'
         option last_resort 'unreachable'
         list use_member 'wan_m1_w3'
         list use_member 'wan2_m1_w3'
         list use_member 'wan3_m1_w3'
    
     config rule 'https'
         option sticky '1'
         option dest_port '443'
         option proto 'tcp'
         option use_policy 'balanced'
    
     config rule 'default_rule_v4'
         option dest_ip '0.0.0.0/0'
         option use_policy 'balanced'
         option family 'ipv4'
    
     config rule 'default_rule_v6'
         option dest_ip '::/0'
         option use_policy 'balanced'
         option family 'ipv6'
    
     config interface 'wan2'
         option enabled '1'
         option initial_state 'online'
         option family 'ipv4'
         option track_method 'ping'
         option reliability '1'
         option count '1'
         option size '56'
         option max_ttl '60'
         option timeout '4'
         option interval '10'
         option failure_interval '5'
         option recovery_interval '5'
         option down '5'
         option up '5'
         list track_ip '8.8.8.8'
         list track_ip '8.8.4.4'
         list flush_conntrack 'ifup'
    
     config interface 'wan3'
         option enabled '1'
         option initial_state 'online'
         option family 'ipv4'
         option track_method 'ping'
         option reliability '1'
         option count '1'
         option size '56'
         option max_ttl '60'
         option timeout '4'
         option interval '5'
         option failure_interval '5'
         option recovery_interval '5'
         option down '5'
         option up '5'
         list track_ip '8.8.8.8'
         list track_ip '8.8.4.4'
    
     config member 'wan2_m1_w3'
         option interface 'wan2'
         option metric '1'
         option weight '3'
    
     config member 'wan3_m1_w3'
         option interface 'wan3'
         option metric '1'
         option weight '3'
    

Diagnostics from MWAN manager
Using wan

    Running 'ping -c 5 -W 1 192.168.254.254' with DEVICE=eth0.2 SRCIP=192.168.254.155 FWMARK=0x3f00 FAMILY=ipv4
    PING 192.168.254.254 (192.168.254.254): 56 data bytes
    64 bytes from 192.168.254.254: seq=0 ttl=64 time=2.185 ms
    64 bytes from 192.168.254.254: seq=1 ttl=64 time=1.484 ms
    64 bytes from 192.168.254.254: seq=2 ttl=64 time=1.099 ms
    64 bytes from 192.168.254.254: seq=3 ttl=64 time=3.733 ms
    64 bytes from 192.168.254.254: seq=4 ttl=64 time=3.430 ms

    --- 192.168.254.254 ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 1.099/2.386/3.733 ms

Using wan2

    Running 'ping -c 5 -W 1 192.168.1.1' with DEVICE=eth0.3 SRCIP=192.168.1.125 FWMARK=0x3f00 FAMILY=ipv4
    PING 192.168.1.1 (192.168.1.1): 56 data bytes
    64 bytes from 192.168.1.1: seq=0 ttl=64 time=1.390 ms
    64 bytes from 192.168.1.1: seq=1 ttl=64 time=1.129 ms
    64 bytes from 192.168.1.1: seq=2 ttl=64 time=1.166 ms
    64 bytes from 192.168.1.1: seq=3 ttl=64 time=1.178 ms
    64 bytes from 192.168.1.1: seq=4 ttl=64 time=1.064 ms

    --- 192.168.1.1 ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 1.064/1.185/1.390 ms

Status on MWAN

    Interface status:
    interface wan is online 00h:27m:13s, uptime 00h:42m:18s and tracking is active
    interface wan2 is offline 00h:00m:00s, uptime 00h:42m:16s and tracking is active
    interface wan3 is offline and tracking is paused

    Current ipv4 policies:
    balanced:
    wan (100%)

    Current ipv6 policies:
    balanced:
    unreachable

Overview:

For starters, fix the obvious mistakes.
Remove option type 'bridge' from all wan interfaces.

Fix that too.

Then stop the mwan3 service and try pinging the test hosts directly rather than the default gateways.

ping -I eth0.2 -c 3 8.8.8.8
ping -I eth0.3 -c 3 8.8.8.8

okay, i will try it thanks!

I tried this but no luck!

What have you tried?
Can you ping 8.8.8.8 via eth0.3?

yes.

root@OpenWrt:~# ping -I eth0.3 -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

and for the eth0.2

root@OpenWrt:~# ping -I eth0.2 -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=55 time=65.096 ms
64 bytes from 8.8.8.8: seq=1 ttl=55 time=42.785 ms
64 bytes from 8.8.8.8: seq=2 ttl=55 time=40.823 ms

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 40.823/49.568/65.096 ms

here is my logread

root@OpenWrt:~# logread | grep eth0.3
Mon Jun  3 11:42:24 2024 daemon.notice netifd: VLAN 'eth0.3' link is up
Mon Jun  3 11:42:55 2024 user.notice mwan3-hotplug[3731]: Execute ifup event on interface wan2 (eth0.3)
Mon Jun  3 11:42:58 2024 user.notice mwan3track[1724]: Interface wan2 (eth0.3) is online
Mon Jun  3 11:43:00 2024 user.notice firewall: Reloading firewall due to ifup of wan2 (eth0.3)
Mon Jun  3 11:43:04 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.8.8" on interface wan2 (eth0.3). Current score: 10
Mon Jun  3 11:43:08 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.4.4" on interface wan2 (eth0.3). Current score: 10
Mon Jun  3 11:43:13 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.1.1.1" on interface wan2 (eth0.3). Current score: 10
Mon Jun  3 11:43:17 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.0.0.1" on interface wan2 (eth0.3). Current score: 10
Mon Jun  3 11:43:18 2024 user.notice mwan3track[1724]: Interface wan2 (eth0.3) is disconnecting
Mon Jun  3 11:43:28 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.8.8" on interface wan2 (eth0.3). Current score: 9
Mon Jun  3 11:43:33 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.4.4" on interface wan2 (eth0.3). Current score: 9
Mon Jun  3 11:43:37 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.1.1.1" on interface wan2 (eth0.3). Current score: 9
Mon Jun  3 11:43:41 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.0.0.1" on interface wan2 (eth0.3). Current score: 9
Mon Jun  3 11:43:50 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.8.8" on interface wan2 (eth0.3). Current score: 8
Mon Jun  3 11:43:54 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.4.4" on interface wan2 (eth0.3). Current score: 8
Mon Jun  3 11:43:58 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.1.1.1" on interface wan2 (eth0.3). Current score: 8
Mon Jun  3 11:44:02 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.0.0.1" on interface wan2 (eth0.3). Current score: 8
Mon Jun  3 11:44:11 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.8.8" on interface wan2 (eth0.3). Current score: 7
Mon Jun  3 11:44:15 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.4.4" on interface wan2 (eth0.3). Current score: 7
Mon Jun  3 11:44:19 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.1.1.1" on interface wan2 (eth0.3). Current score: 7
Mon Jun  3 11:44:23 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.0.0.1" on interface wan2 (eth0.3). Current score: 7
Mon Jun  3 11:44:32 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.8.8" on interface wan2 (eth0.3). Current score: 6
Mon Jun  3 11:44:37 2024 user.info mwan3track[1724]: Check (ping) failed for target "8.8.4.4" on interface wan2 (eth0.3). Current score: 6
Mon Jun  3 11:44:41 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.1.1.1" on interface wan2 (eth0.3). Current score: 6
Mon Jun  3 11:44:45 2024 user.info mwan3track[1724]: Check (ping) failed for target "1.0.0.1" on interface wan2 (eth0.3). Current score: 6
Mon Jun  3 11:44:45 2024 user.notice mwan3track[1724]: Interface wan2 (eth0.3) is offline
Mon Jun  3 11:44:46 2024 user.notice mwan3-hotplug[6194]: Execute disconnected event on interface wan2 (eth0.3)

As you can see, the wan2 (eth0.3) connection seems to be down.

but why is it?, i tried my second connection that eth0.3, i can browse and connect to the internet

Stop the primary wan interface (ifdown wan) and confirm that you still have an internet connection.

What is the output of ip route list default?

okay i will try it!

i change instead of that wan2 from vlan i use the wireless connected to my second connection

default via 192.168.1.1 dev phy0-sta0 proto static src 192.168.1.124 metric 2
default via 192.168.254.254 dev eth0.2 proto static src 192.168.254.155 metric 10

also i tried to ping that phy0-sta0

root@OpenWrt:~# ping -I phy0-sta0 -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

so strange that this second router has no internet connection, but if i try to connect directly to my device like phone or laptop it connect perfectly

i notice that i can ping google.com but not 8.8.8.8, is this firewall things blocking?