I am running a PC Engines APU1 board with OpenWrt 21.02.2. My board has three network RJ45 network interfaces, a Qualcomm Atheros QCA9880 802.11nac WiFi interface and a Quectel EC25-E LTE modem which runs as a QMI device.
LAN connections
eth0 & eth2 are assigned to br-lan
wlan0 is its own LAN interface
WAN connections
eth1 is configured as wan device with metric 10, wan6 with metric 20
wwan0 is configured as 4G device with metric 30
So far so good, internet is accessible wired and wireless.
For auto-failover and more monitoring and configuration I installed mwan3 (2.10.13-1) and configured it to failover to LTE when wired WAN fails and go back to wired WAN when it returns. I tested it by pulling the WAN cable. Tested the LTE connection by disabling the interface and enabling it again. This also works good.
Now to the problem:
Once a day, mostly at night the LTE connecting drops, you see tracking score going down and eventually wwan0 goes offline. But it will not recover the logging shows no tracking anymore on the offline interface. When I use the diagnose function in Luci to ping the tracking addresses all of them react within a 20-30 ms, when I give an ifup on the wwan0 interface everything is OK again. I tried flushing conntrack but that doesn't make a difference. The connections stays offline until I restart the connection.
What I don't exactly understand is what happens at night with the connection, it looks lik that it start with the ending of the lease time for DHCP address and then it maybe takes longer before it has an internet connection again. It isn't only the IP's lease time because it gets renewed every 2 hours and during the day that works fine. It could be that it is caused by the ISP forcing my connection to another subnet, that could take longer which would cause mwan3 to detect a loss of connection. However that doesn't explain why it wouldn't automatically recover.
To diagnose it a step further I added the virtual DHCP client interface from the wwan0 interface to the mwan3 configuration. What I saw is that both of them lose the connection around the same time, but the virtual adapter recovers and the interface itself doesn't. Unfortunately only the real interface exists in the firewall so you don't have any connection over it...
What can I do to let mwan3 recover the LTE connection automatically. Is my only option to add an ifup for the wwan0 interface when the connection get lost in the mwan3.user script?
Did a make a configuration error, do I miss something else?
I searched the forum and the worldwide web several times but it doesn't show topics outside the flush conntrack solutions.
/etc/config/network:
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fd3b:3e30:3f67::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option ipaddr '10.127.1.1'
config interface 'wan'
option device 'eth1'
option proto 'dhcp'
option metric '10'
config interface 'wan6'
option device 'eth1'
option proto 'dhcpv6'
option reqaddress 'try'
option reqprefix 'auto'
option metric '20'
config interface '4G'
option proto 'qmi'
option device '/dev/cdc-wdm0'
option apn 'internet.tele2.nl'
option pincode '****'
option metric '30'
option auth 'none'
option pdptype 'ipv4'
config interface 'WiFi'
option proto 'static'
option ipaddr '10.127.2.1'
option netmask '255.255.255.0'
option device 'radio0.network1'
/etc/config/mwan3:
config policy 'wan_only'
list use_member 'wan_m1_w3'
list use_member 'wan6_m1_w3'
config policy 'wan_4G'
option last_resort 'unreachable'
list use_member 'wan_m1_w3'
list use_member '4G_m3_w2'
config policy 'balanced'
option last_resort 'unreachable'
list use_member 'wan_m1_w3'
list use_member 'wan6_m1_w3'
list use_member '4G_m1_w1'
config policy '4G_only'
option last_resort 'unreachable'
list use_member '4G_m2_w2'
config member 'wan_m1_w3'
option interface 'wan'
option metric '1'
option weight '3'
config member 'wan_m2_w3'
option interface 'wan'
option metric '2'
option weight '3'
config member '4G_m1_w2'
option interface '4G'
option metric '1'
option weight '2'
config member '4G_m2_w2'
option metric '2'
option weight '2'
option interface '4G'
config member 'wan6_m1_w3'
option interface 'wan6'
option metric '1'
option weight '3'
config member 'wan6_m2_w3'
option interface 'wan6'
option metric '2'
option weight '3'
config globals 'globals'
option mmx_mask '0x3F00'
config interface 'wan'
option enabled '1'
list track_ip '8.8.4.4'
list track_ip '8.8.8.8'
list track_ip '208.67.222.222'
list track_ip '208.67.220.220'
option family 'ipv4'
option reliability '2'
option initial_state 'online'
option track_method 'ping'
option count '1'
option size '56'
option max_ttl '60'
option check_quality '0'
option timeout '4'
option interval '10'
option failure_interval '5'
option recovery_interval '5'
option down '5'
option up '3'
config interface 'wan6'
option enabled '0'
list track_ip '2001:4860:4860::8844'
list track_ip '2001:4860:4860::8888'
list track_ip '2620:0:ccd::2'
list track_ip '2620:0:ccc::2'
option family 'ipv6'
option reliability '2'
option initial_state 'online'
option track_method 'ping'
option count '1'
option size '56'
option max_ttl '60'
option check_quality '0'
option timeout '4'
option interval '10'
option failure_interval '5'
option recovery_interval '5'
option down '5'
option up '3'
config rule 'default_rule_v4'
option dest_ip '0.0.0.0/0'
option family 'ipv4'
option proto 'all'
option sticky '0'
option use_policy 'wan_4G'
config rule 'default_rule_v6'
option dest_ip '::/0'
option family 'ipv6'
option proto 'all'
option sticky '0'
option use_policy 'wan_only'
config interface '4G'
option enabled '1'
option initial_state 'online'
option family 'ipv4'
option track_method 'ping'
option count '1'
option size '56'
option max_ttl '60'
option check_quality '0'
option timeout '4'
option failure_interval '5'
option recovery_interval '5'
option down '5'
option reliability '2'
option up '3'
option interval '10'
list track_ip '8.8.8.8'
list track_ip '208.67.222.222'
list track_ip '208.67.220.220'
list track_ip '1.1.1.1'
list flush_conntrack 'ifup'
list flush_conntrack 'ifdown'
list flush_conntrack 'connected'
list flush_conntrack 'disconnected'
config interface '4G_4'
option enabled '1'
option initial_state 'online'
option family 'ipv4'
option track_method 'ping'
option reliability '2'
option count '1'
option size '56'
option max_ttl '60'
option check_quality '0'
option timeout '4'
option interval '10'
option failure_interval '5'
option recovery_interval '5'
option down '5'
option up '5'
list track_ip '1.1.1.1'
list track_ip '8.8.8.8'
list track_ip '208.67.220.220'
list flush_conntrack 'ifup'
list flush_conntrack 'ifdown'
list flush_conntrack 'connected'
list flush_conntrack 'disconnected'
config member '4G4_m1_w2'
option interface '4G_4'
option metric '1'
option weight '2'
config member '4G4_m2_w2'
option interface '4G_4'
option metric '2'
option weight '2'
config member '4G4_m3_w2'
option interface '4G_4'
option metric '3'
option weight '2'
config member '4G_m3_w2'
option interface '4G'
option metric '3'
option weight '2'
config member '4G_m1_w1'
option interface '4G'
option metric '1'
option weight '1'
config member '4G4_m1_w1'
option interface '4G_4'
option metric '1'
option weight '1'