Mwan3, wan shows online when it is not

Tarmgass · September 6, 2022, 10:37am

Hi
I do have a wndr3700 with openwrt21.02 and mwan3.
It is configured witn wan and wanb. Wan is LTE, and wanb is satelite connection. ( On a ship).
In port everything seems to work as it should with failover, but now testing out of LTE range, it still shows wan connection is online, and didt not swap over to wanb wich is backup connection.

In luci i tested diagnostics and ping tracking ip for wan it does not get response, but still it appears mwan think its online?!

Can someone please check if my config is okey? If i unplug or disable wan interface it connects through wanb as it should, but i hope i could manage to get it to change over automatically when lte connection drops.

config globals 'globals'
	option mmx_mask '0x3F00'

config interface 'wan'
	option enabled '1'
	list track_ip '8.8.4.4'
	list track_ip '8.8.8.8'
	list track_ip '208.67.222.222'
	list track_ip '208.67.220.220'
	option family 'ipv4'
	option initial_state 'online'
	option track_method 'ping'
	option max_ttl '60'
	option check_quality '0'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	option reliability '1'
	option count '3'
	option size '24'

config interface 'wanb'
	list track_ip '8.8.4.4'
	list track_ip '8.8.8.8'
	list track_ip '208.67.222.222'
	list track_ip '208.67.220.220'
	option family 'ipv4'
	option reliability '1'
	option enabled '1'
	option initial_state 'online'
	option track_method 'ping'
	option max_ttl '60'
	option check_quality '0'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	option count '3'
	option size '24'

config rule 'https'
	option sticky '1'
	option dest_port '443'
	option proto 'tcp'
	option use_policy 'wan_to_wanb'
	option family 'ipv4'

config rule 'default_rule_v4'
	option dest_ip '0.0.0.0/0'
	option family 'ipv4'
	option proto 'all'
	option sticky '0'
	option use_policy 'wan_to_wanb'

config rule 'default_rule_v6'
	option dest_ip '::/0'
	option use_policy 'balanced'
	option family 'ipv6'

config member 'wan_member'
	option interface 'wan'
	option metric '1'
	option weight '1'

config member 'wanb_member'
	option interface 'wanb'
	option metric '2'
	option weight '2'

config policy 'wan_to_wanb'
	list use_member 'wan_member'
	list use_member 'wanb_member'
	option last_resort 'unreachable'

network config

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd85:317a:5438::/48'

config interface 'wan'
	option device 'eth1'
	option proto 'dhcp'
	option metric '10'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0'

config device
	option name 'eth0'
	option macaddr 'c6:3d:c7:90:eb:7b'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '192.168.20.1'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'
	option blinkrate '2'
	option enable_vlan4k '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '5 2 1'

config switch_port
	option device 'switch0'
	option port '1'
	option led '6'

config switch_port
	option device 'switch0'
	option port '2'
	option led '9'

config switch_port
	option device 'switch0'
	option port '5'
	option led '2'

config switch_vlan
	option device 'switch0'
	option ports '5t 3'
	option vlan '2'
	option description 'wanb'

config interface 'wanb'
	option device 'eth0.2'
	option proto 'dhcp'
	option metric '5'

config switch_vlan
	option device 'switch0'
	option ports '5t 0'
	option vlan '3'
	option description 'admin'

config interface 'admin'
	option proto 'static'
	option device 'eth0.3'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '10.0.10.1'
	list dns '192.168.20.1'

config interface 'PPDWIFI'
	option proto 'dhcp'

config interface 'kvs'
	option proto 'dhcp'

trendy · September 6, 2022, 12:28pm

What is the output of mwan3 status ?
Also there should be messages in logs logread -e /mwan3/i

Tarmgass · September 6, 2022, 9:51pm

Hello. Here is status. This is with wan interface activated, but no LTE signals. The lte modem is 192.168.30.1 wan.

The satelite connection wanb 192.168.120.1

When i activate wan like now i do get offline, because the mwan thinks wan is online and use it as gateway.

wan is the preferred gateway, but when i am out of lte range i want it so switch over to wanb.
i changed gateway metric on wan to 10, and 20 to wanb, but still same..

I intend to only use ipv4, could it be som ipv6 bugging ?

root@xxxxxx:~# mwan3 status
Interface status:
 interface wan is online 00h:01m:35s, uptime 00h:02m:02s and tracking is active
 interface wanb is online 02h:07m:58s, uptime 02h:08m:35s and tracking is active

Current ipv4 policies:
wan_to_wanb:
 wan (100%)

Current ipv6 policies:
balanced:

wan_to_wanb:
 unreachable

Directly connected ipv4 networks:
192.168.30.0/24
192.168.120.0/24
224.0.0.0/3
127.0.0.0/8
10.0.22.0/24
192.168.20.0/24
10.0.10.0/24

Directly connected ipv6 networks:
fe80::/64

Active ipv4 user rules:
69582 4038K S https  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 443
46096 6079K - wan_to_wanb  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Active ipv6 user rules:
 2582  300K - balanced  all      *      *       ::/0                 ::/0       

root@xxxxxx:~#

Tarmgass · September 6, 2022, 10:05pm

I tried to delete the ipv6 rule i mwan rule and activated wan interface again without drop in connection. I do not get the correct answer before i can reach LTE signals again but hoping that was the problem

New status shows wan offline witch is just correct.

root@xxxxxx:~# mwan3 status
Interface status:
 interface wan is offline 00h:00m:00s, uptime 00h:03m:39s and tracking is active
 interface wanb is online 00h:04m:03s, uptime 02h:23m:36s and tracking is active

Current ipv4 policies:
wan_to_wanb:
 wanb (100%)

Current ipv6 policies:
wan_to_wanb:
 unreachable

Directly connected ipv4 networks:
192.168.30.0/24
10.0.10.0/24
192.168.120.0/24
224.0.0.0/3
192.168.20.0/24
127.0.0.0/8
10.0.22.0/24

Directly connected ipv6 networks:
fe80::/64

Active ipv4 user rules:
  328 23195 S https  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 443
  354 36265 - wan_to_wanb  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Active ipv6 user rules:

Tarmgass · September 7, 2022, 7:11pm

Still not happy..
But looks like i maybe discovered whats making problems.

I have a webserver that runs some commands with php scripts.
The plan was to have a user friendly web page to check mwan3 status and vnstat.
I have also made it possible to force interfaces down, and up with the command: ubus call network.interface.wan down and ubus call network.interface.wan up . This was because when the LTE connection was unstable it would be possible for non administrator users to disable the LTE in a easy and safe way without using luci webif.

The problem is, when i send ubus call network.interface.wan up, it stays up and mwan3 showing ONLINE no matter if there is LTE connection or not.

Have alsto tried mwan3 restart after taking wan up, but still ONLINE.

Same thing if i use uci set network.wan.disabled="1"
Anyone with ideas how i still can use commands to take down and up interfaces and still run mwan3 ?

trendy · September 7, 2022, 7:26pm

This must be equivalent to ifup wan
I am using it often in my mwan3 router and have never faced such issue.
Monitor the logs when the connectivity is lost, you should see the lost pings and the current interface score.

Tarmgass · September 7, 2022, 7:33pm

How do i monitor the logs?

Tarmgass · September 7, 2022, 7:42pm

Please try to explain what you mean with equal to ifup wan.
What commands do you use to take down and up again wan connection ?

trendy · September 8, 2022, 7:40am

With logread

ifup and ifdown

Tarmgass · September 8, 2022, 10:10am

Same thing happens if i use ifdown and ifup.

Now i only have connection on satelite. 4G is out of range.
When i take ifdown 4G i can see in interfaces all looks ok, but when i take ifup 4G again it appears to be online. Check status below

Interface status:
 interface 4G is online 00h:02m:18s, uptime 00h:02m:31s and tracking is active
 interface V_SAT is online 14h:12m:33s, uptime 14h:12m:56s and tracking is active

Current ipv4 policies:
4G_V_SAT:
 4G (100%)

Current ipv6 policies:
4G_V_SAT:
 unreachable

Directly connected ipv4 networks:
192.168.120.0/24
10.0.22.0/24
127.0.0.0/8
192.168.20.0/24
224.0.0.0/3
192.168.30.0/24
10.0.10.0/24

Directly connected ipv6 networks:
fe80::/64

Active ipv4 user rules:
81288 4562K S https  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 443 
73512 8373K - 4G_V_SAT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            

Active ipv6 user rules:

logread just have a lot of this?

Thu Sep  8 09:19:06 2022 authpriv.notice dropbear[4505]: Password auth succeeded for 'root' from 192.168.20.117:43242
Thu Sep  8 09:19:07 2022 authpriv.info dropbear[4505]: Exit (root) from <192.168.20.117:43242>: Disconnect received
Thu Sep  8 09:19:17 2022 authpriv.info dropbear[4580]: Child connection from 192.168.20.117:43244
Thu Sep  8 09:19:19 2022 authpriv.notice dropbear[4580]: Password auth succeeded for 'root' from 192.168.20.117:43244
Thu Sep  8 09:19:20 2022 authpriv.info dropbear[4580]: Exit (root) from <192.168.20.117:43244>: Disconnect received
Thu Sep  8 09:19:30 2022 authpriv.info dropbear[4650]: Child connection from 192.168.20.117:43246
Thu Sep  8 09:19:33 2022 authpriv.notice dropbear[4650]: Password auth succeeded for 'root' from 192.168.20.117:43246
Thu Sep  8 09:19:33 2022 authpriv.info dropbear[4650]: Exit (root) from <192.168.20.117:43246>: Disconnect received

trendy · September 8, 2022, 10:40am

You should see something like this:

root@whale:[~]#logread -e mwan3
Thu Sep  8 11:58:50 2022 user.info mwan3track[9537]: Check (ping) failed for target "1.1.1.1" on interface orange_pl (wwan0). Current score: 6
Thu Sep  8 11:58:56 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.152.34" on interface orange_pl (wwan0). Current score: 6
Thu Sep  8 11:59:02 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.159.1" on interface orange_pl (wwan0). Current score: 6
Thu Sep  8 11:59:02 2022 user.notice mwan3track[9537]: Interface orange_pl (wwan0) is disconnecting
Thu Sep  8 11:59:14 2022 user.info mwan3track[9537]: Check (ping) failed for target "1.1.1.1" on interface orange_pl (wwan0). Current score: 5
Thu Sep  8 11:59:20 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.152.34" on interface orange_pl (wwan0). Current score: 5
Thu Sep  8 11:59:26 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.159.1" on interface orange_pl (wwan0). Current score: 5
Thu Sep  8 11:59:37 2022 user.info mwan3track[9537]: Check (ping) failed for target "1.1.1.1" on interface orange_pl (wwan0). Current score: 4
Thu Sep  8 11:59:43 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.152.34" on interface orange_pl (wwan0). Current score: 4
Thu Sep  8 11:59:49 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.159.1" on interface orange_pl (wwan0). Current score: 4
Thu Sep  8 12:00:00 2022 user.info mwan3track[9537]: Check (ping) failed for target "1.1.1.1" on interface orange_pl (wwan0). Current score: 3
Thu Sep  8 12:00:06 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.152.34" on interface orange_pl (wwan0). Current score: 3
Thu Sep  8 12:00:12 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.159.1" on interface orange_pl (wwan0). Current score: 3
Thu Sep  8 12:00:23 2022 user.info mwan3track[9537]: Check (ping) failed for target "1.1.1.1" on interface orange_pl (wwan0). Current score: 2
Thu Sep  8 12:00:29 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.152.34" on interface orange_pl (wwan0). Current score: 2
Thu Sep  8 12:00:35 2022 user.info mwan3track[9537]: Check (ping) failed for target "194.204.159.1" on interface orange_pl (wwan0). Current score: 2
Thu Sep  8 12:00:35 2022 user.notice mwan3track[9537]: Interface orange_pl (wwan0) is offline
Thu Sep  8 12:00:35 2022 user.notice mwan3-hotplug[4289]: Execute disconnected event on interface orange_pl (wwan0)
Thu Sep  8 12:00:36 2022 user.notice mwan3-hotplug[4516]: Execute ifdown event on interface orange_pl (unknown)
Thu Sep  8 12:00:36 2022 user.info mwan3track[9537]: Detect ifdown event on interface orange_pl (wwan0)
Thu Sep  8 12:00:36 2022 user.notice mwan3track[9537]: Interface orange_pl (wwan0) is offline
Thu Sep  8 12:00:36 2022 user.notice mwan3-hotplug[4656]: Execute disconnected event on interface orange_pl (wwan0)
Thu Sep  8 12:00:37 2022 user.notice mwan3-hotplug[5340]: Execute ifdown event on interface orange_pl (unknown)
Thu Sep  8 12:00:37 2022 user.info mwan3track[9537]: Detect ifdown event on interface orange_pl (wwan0)
Thu Sep  8 12:00:37 2022 user.notice mwan3track[9537]: Interface orange_pl (wwan0) is offline
Thu Sep  8 12:00:38 2022 user.notice mwan3-hotplug[5469]: Execute disconnected event on interface orange_pl (wwan0)

For the other problem, you can try to change option initial_state 'online' at the interface config. It will not be used in the beginning, but you could make less strict the conditions for considering it up.

Tarmgass · September 8, 2022, 12:10pm


root@RM:~# logread -e mwan3
root@RM:~#

Tarmgass · September 8, 2022, 12:29pm

I set both vsat and 4g to offline initial state and then did ifdown and up on both, and looks like its good now! After some some seconds vsat was online as it should.
Only vsat did get online. Will do a new test when i reach 4g signals!

trendy · September 8, 2022, 12:37pm

uci set mwan3.globals.logging='1'
uci set mwan3.globals.loglevel='notice'
uci commit mwan3
service mwan3 restart

Tarmgass · September 9, 2022, 6:54pm

Now i get the logs, but is it possible to debug "live" ? I am not at LTE coverage before monday, but i would like to see it is trying to ping 4G connection, even when it will not respond.

root@RM:~# logread -e mwan3
Fri Sep  9 06:10:33 2022 user.notice mwan3-hotplug[2415]: mwan3 hotplug on lan not called because interface disabled
Fri Sep  9 06:10:41 2022 user.notice mwan3-hotplug[3183]: mwan3 hotplug on AxxA not called because interface disabled
Fri Sep  9 06:10:46 2022 user.notice mwan3-hotplug[3596]: mwan3 hotplug on loopback not called because interface disabled
Fri Sep  9 06:10:48 2022 user.notice mwan3-hotplug[3900]: Execute ifup event on interface V_SAT (eth0.4)
Fri Sep  9 06:10:48 2022 user.warn mwan3-hotplug[3900]: failed to add 192.168.120.0/24 dev eth0.4 proto static scope link metric 20 to table 2
Fri Sep  9 06:10:48 2022 user.warn mwan3-hotplug[3900]: failed to add default via 192.168.120.1 dev eth0.4 proto static src 192.168.120.103 metric 20 to table 2
Fri Sep  9 06:10:49 2022 user.info mwan3track[1627]: Detect ifup event on interface V_SAT ()
Fri Sep  9 06:10:52 2022 user.notice mwan3-hotplug[4135]: Execute ifup event on interface 4G (eth0.3)
Fri Sep  9 06:10:53 2022 user.warn mwan3-hotplug[4135]: failed to add 192.168.30.0/24 dev eth0.3 proto static scope link metric 10 to table 1
Fri Sep  9 06:10:53 2022 user.warn mwan3-hotplug[4135]: failed to add default via 192.168.30.1 dev eth0.3 proto static src 192.168.30.204 metric 10 to table 1
Fri Sep  9 06:10:53 2022 user.info mwan3track[1626]: Detect ifup event on interface 4G ()
Fri Sep  9 06:10:56 2022 user.notice mwan3-hotplug[4409]: mwan3 hotplug on KaS not called because interface disabled
Fri Sep  9 06:10:57 2022 user.info mwan3track[1627]: Check (ping) failed for target "8.8.8.8" on interface V_SAT (). Current score: 10
Fri Sep  9 06:10:57 2022 user.notice mwan3track[1627]: Interface V_SAT () is disconnecting
Fri Sep  9 06:11:01 2022 user.info mwan3track[1626]: Check (ping) failed for target "8.8.8.8" on interface 4G (). Current score: 10
Fri Sep  9 06:11:01 2022 user.notice mwan3track[1626]: Interface 4G () is disconnecting
Fri Sep  9 06:11:04 2022 user.info mwan3track[1627]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 0
Fri Sep  9 06:11:04 2022 user.notice mwan3track[1627]: Interface V_SAT (eth0.4) is connecting
Fri Sep  9 06:11:08 2022 user.info mwan3track[1627]: Lost 5 ping(s) on interface V_SAT (eth0.4). Current score: 0
Fri Sep  9 06:11:18 2022 user.info mwan3track[1627]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 1
Fri Sep  9 06:11:28 2022 user.info mwan3track[1627]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 2
Fri Sep  9 06:11:37 2022 user.info mwan3track[1627]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 3
Fri Sep  9 06:11:47 2022 user.info mwan3track[1627]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 4
Fri Sep  9 06:11:47 2022 user.notice mwan3track[1627]: Interface V_SAT (eth0.4) is online
Fri Sep  9 06:11:48 2022 user.notice mwan3-hotplug[5974]: Execute connected event on interface V_SAT (eth0.4)
Fri Sep  9 06:11:59 2022 user.info mwan3track[1627]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 5
Fri Sep  9 11:03:21 2022 user.info mwan3track[1627]: Check (ping) failed for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 10
Fri Sep  9 11:03:21 2022 user.notice mwan3track[1627]: Interface V_SAT (eth0.4) is disconnecting
Fri Sep  9 11:03:31 2022 user.notice mwan3track[1627]: Interface V_SAT (eth0.4) is connecting
Fri Sep  9 11:03:32 2022 user.info mwan3track[1627]: Lost 5 ping(s) on interface V_SAT (eth0.4). Current score: 9
Fri Sep  9 18:02:54 2022 user.notice mwan3track[1626]: Stopping mwan3track for interface "4G". Status was "offline"
Fri Sep  9 18:02:54 2022 user.notice mwan3track[1627]: Stopping mwan3track for interface "V_SAT". Status was "connecting"
Fri Sep  9 18:02:56 2022 user.notice mwan3-hotplug[12082]: Execute ifup event on interface 4G (eth0.3)
Fri Sep  9 18:02:56 2022 user.notice mwan3-hotplug[12087]: Execute ifup event on interface V_SAT (eth0.4)
Fri Sep  9 18:03:04 2022 user.info mwan3track[12284]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 0
Fri Sep  9 18:03:04 2022 user.notice mwan3track[12284]: Interface V_SAT (eth0.4) is connecting
Fri Sep  9 18:03:14 2022 user.info mwan3track[12284]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 1
Fri Sep  9 18:03:24 2022 user.info mwan3track[12284]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 2
Fri Sep  9 18:03:34 2022 user.info mwan3track[12284]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 3
Fri Sep  9 18:03:43 2022 user.info mwan3track[12284]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 4
Fri Sep  9 18:03:44 2022 user.notice mwan3track[12284]: Interface V_SAT (eth0.4) is online
Fri Sep  9 18:03:44 2022 user.notice mwan3-hotplug[13480]: Execute connected event on interface V_SAT (eth0.4)
Fri Sep  9 18:03:55 2022 user.info mwan3track[12284]: Check (ping) success for target "8.8.8.8" on interface V_SAT (eth0.4). Current score: 5
root@RM:~#

trendy · September 9, 2022, 11:01pm

logread -f -e mwan3

Tarmgass · September 10, 2022, 6:20am

logread -f -e mwan3

Did not work for me. Nothing is showing up.

trendy · September 10, 2022, 7:49am

It will show when there is some new output to show.

Tarmgass · September 10, 2022, 10:32am

Aha, i thought i maybe could monitor the pinging thats happening in the background.

trendy · September 10, 2022, 1:39pm

I think it won't show anything as long as nothing changes in the ping results.