Mwan3 enabled but not working

xtcbt · June 23, 2019, 7:58pm

Hello, I'm new to openwrt and I'm enjoying using this due to very good features especially the SQM compared to tomato but I'm having problems on configuring mwan3. I just want to make a simple setup with 2 wans with load balancing and failover but I can't seem to make it work after enabling the mwan3 service, I lose all internet connection even though both wan are up.

Here are the data from troubleshooting tab of mwan3:

Software-Version
-------------------------------------------------
OpenWrt - OpenWrt 18.06.2 r7676-cddd7b4c77
LuCI - git-19.020.41695-6f6641d


Output of "ip a show"
-------------------------------------------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.19.1/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 80:37:73:cb:b1:22 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8237:73ff:fecb:b122/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 80:37:73:cb:b1:23 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether c2:ad:d4:9d:b6:7c brd ff:ff:ff:ff:ff:ff
6: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32
    link/ether a2:70:f0:ed:28:c2 brd ff:ff:ff:ff:ff:ff
7: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32
    link/ether 3a:f5:f7:d6:aa:bc brd ff:ff:ff:ff:ff:ff
9: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 80:37:73:cb:b1:22 brd ff:ff:ff:ff:ff:ff
    inet 192.168.19.1/24 brd 192.168.19.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fe80::8237:73ff:fecb:b122/64 scope link 
       valid_lft forever preferred_lft forever
10: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
    link/ether 80:37:73:cb:b1:22 brd ff:ff:ff:ff:ff:ff
11: eth0.3@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 80:37:73:cb:b1:22 brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.1/24 brd 192.168.18.255 scope global eth0.3
       valid_lft forever preferred_lft forever
    inet6 fe80::8237:73ff:fecb:b122/64 scope link 
       valid_lft forever preferred_lft forever
12: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc cake state UP group default qlen 1000
    link/ether 80:37:73:cb:b1:22 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.2/24 brd 192.168.1.255 scope global eth0.2
       valid_lft forever preferred_lft forever
    inet6 fe80::8237:73ff:fecb:b122/64 scope link 
       valid_lft forever preferred_lft forever
13: eth0.4@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 80:37:73:cb:b1:22 brd ff:ff:ff:ff:ff:ff
    inet 112.201.35.244/19 brd 112.201.63.255 scope global eth0.4
       valid_lft forever preferred_lft forever
    inet6 fe80::8237:73ff:fecb:b122/64 scope link 
       valid_lft forever preferred_lft forever
21117: ifb4eth0.2: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc cake state UNKNOWN group default qlen 32
    link/ether c6:af:c4:fe:ce:48 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::c4af:c4ff:fefe:ce48/64 scope link 
       valid_lft forever preferred_lft forever


Output of "ip route show"
-------------------------------------------------
default via 192.168.19.1 dev lo 
default via 192.168.1.1 dev eth0.2 proto static src 192.168.1.2 metric 10 
default via 112.201.32.1 dev eth0.4 proto static src 112.201.35.244 metric 20 
112.201.32.0/19 dev eth0.4 proto static scope link metric 20 
192.168.1.0/24 dev eth0.2 proto static scope link metric 10 
192.168.18.0/24 dev eth0.3 proto kernel scope link src 192.168.18.1 
192.168.19.0/24 dev br-lan proto kernel scope link src 192.168.19.1


Output of "ip rule show"
-------------------------------------------------
0:	from all lookup local 
1001:	from all iif eth0.2 lookup main 
1002:	from all iif eth0.4 lookup main 
2001:	from all fwmark 0x100/0x3f00 lookup 1 
2002:	from all fwmark 0x200/0x3f00 lookup 2 
2061:	from all fwmark 0x3d00/0x3f00 blackhole
2062:	from all fwmark 0x3e00/0x3f00 unreachable
32766:	from all lookup main 
32767:	from all lookup default


Output of "ip route list table 1-250"
-------------------------------------------------
Table 1: default via 192.168.1.1 dev eth0.2
Table 2: default via 112.201.32.1 dev eth0.4


Output of "iptables -L -t mangle -v -n"
-------------------------------------------------
Chain PREROUTING (policy ACCEPT 3622 packets, 2180K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 4439 2631K mwan3_hook  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain INPUT (policy ACCEPT 141 packets, 14383 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 3480 packets, 2165K bytes)
 pkts bytes target     prot opt in     out     source               destination         
   22  1837 TCPMSS     tcp  --  *      eth0.2  0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
   14   832 TCPMSS     tcp  --  *      eth0.4  0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan2 MTU fixing */ TCPMSS clamp to PMTU

Chain OUTPUT (policy ACCEPT 144 packets, 28412 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  225 62838 mwan3_hook  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain POSTROUTING (policy ACCEPT 3622 packets, 2193K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain mwan3_connected (2 references)
 pkts bytes target     prot opt in     out     source               destination         
 2080 1567K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set mwan3_connected dst MARK or 0x3f00

Chain mwan3_hook (2 references)
 pkts bytes target     prot opt in     out     source               destination         
 4664 2694K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore mask 0x3f00
  253 18037 mwan3_ifaces_in  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00
  235 17140 mwan3_connected  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00
  195 14197 mwan3_ifaces_out  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00
  171 12181 mwan3_rules  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00
 4664 2694K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save mask 0x3f00
 2402 1598K mwan3_connected  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match ! 0x3f00/0x3f00

Chain mwan3_iface_in_wan (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  eth0.2 *       0.0.0.0/0            0.0.0.0/0            match-set mwan3_connected src mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
    5   337 MARK       all  --  eth0.2 *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan */ MARK xset 0x100/0x3f00

Chain mwan3_iface_in_wan2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  eth0.4 *       0.0.0.0/0            0.0.0.0/0            match-set mwan3_connected src mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
   13   560 MARK       all  --  eth0.4 *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan2 */ MARK xset 0x200/0x3f00

Chain mwan3_iface_out_wan (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   12  1008 MARK       all  --  *      eth0.2  0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan */ MARK xset 0x100/0x3f00

Chain mwan3_iface_out_wan2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   12  1008 MARK       all  --  *      eth0.4  0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan2 */ MARK xset 0x200/0x3f00

Chain mwan3_ifaces_in (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  241 17353 mwan3_iface_in_wan  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00
  186 13241 mwan3_iface_in_wan2  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00

Chain mwan3_ifaces_out (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  187 13763 mwan3_iface_out_wan  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00
  142 10656 mwan3_iface_out_wan2  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00

Chain mwan3_policy_balanced (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   57  3782 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 statistic mode random probability 0.50000000000 /* wan2 1 2 */ MARK xset 0x200/0x3f00
   66  5351 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan 1 1 */ MARK xset 0x100/0x3f00

Chain mwan3_policy_wan_1_only (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan 1 1 */ MARK xset 0x100/0x3f00

Chain mwan3_policy_wan_2_only (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* wan2 1 1 */ MARK xset 0x200/0x3f00

Chain mwan3_rules (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  119  8773 mwan3_policy_balanced  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0x3f00 /* default_rule */

Chain qos_Default (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore mask 0xf
    0     0 qos_Default_ct  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf0 length 0:500 MARK xset 0x22/0xff
    0     0 MARK       icmp --  *      *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x11/0xff
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf0 tcp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf0 udp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
    0     0 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save mask 0xff

Chain qos_Default_ct (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf tcp multiport ports 22,53 /* ssh, dns */ MARK xset 0x11/0xff
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf udp multiport ports 22,53 /* ssh, dns */ MARK xset 0x11/0xff
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf tcp multiport ports 20,21,25,80,110,443,993,995 /* ftp, smtp, http(s), imap */ MARK xset 0x33/0xff
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf tcp multiport ports 5190 /* AOL, iChat, ICQ */ MARK xset 0x22/0xff
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf udp multiport ports 5190 /* AOL, iChat, ICQ */ MARK xset 0x22/0xff
    0     0 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save mask 0xff

Here is the status from mwan3 (taken from ssh)

root@OpenWrt:~# mwan3 status
Interface status:
 interface wan is online and tracking is active
 interface wan2 is online and tracking is active

Current ipv4 policies:
balanced:
 wan2 (50%)
 wan (50%)

wan_1_only:
 wan (100%)

wan_2_only:
 wan2 (100%)


Current ipv6 policies:
balanced:
 unreachable

wan_1_only:
 unreachable

wan_2_only:
 unreachable


Directly connected ipv4 networks:
 192.168.1.0
 192.168.1.0/24
 192.168.18.255
 112.201.32.0
 112.201.32.0/19
 127.0.0.0
 192.168.19.255
 192.168.18.0
 127.0.0.1
 112.201.63.255
 224.0.0.0/3
 112.201.35.244
 192.168.19.1
 192.168.19.0
 192.168.18.0/24
 192.168.18.1
 192.168.1.2
 192.168.1.255
 192.168.19.0/24
 127.0.0.0/8
 127.255.255.255

Directly connected ipv6 networks:
 fe80::/64

Active ipv4 user rules:
 3137  268K - balanced  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Active ipv6 user rules:
   31  4136 - balanced  all      *      *       ::/0                 ::/0

I think I configured the wan properly since I can ping both of them from the router by following the guide and configuring it first before installing and starting the mwan3 service. As you can see, the other wan gateway is 192.168.1.1 because this is under CGNAT from my ISP. I don't know if this additional info will help but just ask if you guys need additional info so I can add some more.

By the way, I have no experience in using linux so my knowledge is very limited in using it but if someone can guide me I think I can follow well. Thank you in advance.

xtcbt · June 24, 2019, 8:07pm

Additional info: I can also see this error "daemon.err uhttpd[2633]: cut: standard output: Broken pipe" when activating mwan3.

xtcbt · June 25, 2019, 4:26pm

I think my problem lies with the old loopback from 18.06 package. I would like to use this rtmon service but I have no idea on how to configure this

Configuring a routable loopback (master):

This is not necessary anymore . Router initiated traffic is also loadbalanced and failovered. A new service rtmon (https://github.com/openwrt/packages/commits/master/net/mwan3/files/usr/sbin/mwan3rtmon?author=ptpt52) was added. The service is responsible to sync the main routing table with the interface routing tables. Also inbound traffic has no dedicated firewall tables anymore. This is now working out of the box. Thanks to Chen Minqiang https://github.com/ptpt52

Can someone give advice? Thanks in advance.

lleachii · June 25, 2019, 4:37pm

I've seen many reasons user configure a routable loopback...can you better describe your use case?

I know that config is common Cisco devices for example; but I'm certain that the goal can be accomplished another way.

xtcbt · June 25, 2019, 5:01pm

Hello, thanks for the quick response. My router is asus ac56u and I have 2 ISPs both 50/50 DL/UL and both on dhcp. I want to set them both on load balancing with SQM too. Right now I can only use one isp since I can't make mwan3 work on both. I currently have a vlan on 192.168.18.1 and my main address is 192.168.19.1. If you need other info I will try my best to answer but I have limited knowledge on linux and have no idea how to access other settings to paste here. Maybe I can try with screenshots from gui if you need it or you can guide me if you want to. Thank you.

Leylander · August 21, 2020, 10:17am

I have the the same issue. I have a newifi 3 (d2) and a GLi-Net mifi.

The mifi kept rebooting expectantly and I chose to factory reset it. Since that point in time, mwan3 stopped working in load sharing. I had a copy of the 'settings' file I took as a backup before factory reset and applied it - still no joy with mwan3 (even though it's config came back).

I tried the newifi with the same config (load balance across two connections) and it does the exact same behaviour. mwan3 is running, it's saying it's 50-50 distributing and only one wan interface takes the load.

I can pull the ethernet from the first lan port and it will swap over to the second. But when I plug back in the first wan cable, it ignores that then and keeps using the second wan port as if mwan3 isn't working at all.

I have tried everything I can think of (reinstalling, every configuration file of known-working installs and even done the /usr/sbin/ip thing in the mwan3.sh) and nothing makes it work.

Any thoughts?

aaronjg · August 27, 2020, 3:52am

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

Leylander · August 27, 2020, 5:52am

At least I got someone's attention.

openwrt - below
mwan3 - 2.8.12-1 (current version - https://openwrt.org/packages/pkgdata/mwan3)

If there is a new version, how would I get it?

I assume that in your haste to reply (and mark my comment as spam), you meant to say troubleshooting and not diagnostics.

Software-Version
-------------------------------------------------
OpenWrt - OpenWrt 19.07.3 r11063-85e04e9f46
LuCI - git-20.234.16293-a82f67e


Output of "ip a show"
-------------------------------------------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN qlen 1000
    link/ether 20:76:93:51:ee:ef brd ff:ff:ff:ff:ff:ff
    inet6 fe80::2276:93ff:fe51:eeef/64 scope link 
       valid_lft forever preferred_lft forever
3: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
    link/ether 16:bb:31:75:90:68 brd ff:ff:ff:ff:ff:ff
4: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
    link/ether 82:77:e9:0e:3c:ef brd ff:ff:ff:ff:ff:ff
7: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 20:76:93:51:ee:ef brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/24 brd 192.168.0.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fd42:37d1:febb::1/60 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::2276:93ff:fe51:eeef/64 scope link 
       valid_lft forever preferred_lft forever
8: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether 20:76:93:51:ee:ef brd ff:ff:ff:ff:ff:ff
10: eth0.3@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 20:76:93:51:ee:ef brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.105/24 brd 192.168.2.255 scope global eth0.3
       valid_lft forever preferred_lft forever
    inet6 fe80::2276:93ff:fe51:eeef/64 scope link 
       valid_lft forever preferred_lft forever
12: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether 20:76:93:51:ee:f0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::2276:93ff:fe51:eef0/64 scope link 
       valid_lft forever preferred_lft forever
18: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 20:76:93:51:ee:ee brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.56/24 brd 10.0.0.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::2276:93ff:fe51:eeee/64 scope link 
       valid_lft forever preferred_lft forever
19: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 20:76:93:51:ee:f0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.106/24 brd 192.168.1.255 scope global eth0.2
       valid_lft forever preferred_lft forever
    inet6 fe80::2276:93ff:fe51:eef0/64 scope link 
       valid_lft forever preferred_lft forever


Output of "ip route show"
-------------------------------------------------
default via 192.168.1.1 dev eth0.2  src 192.168.1.106  metric 10 
default via 192.168.2.1 dev eth0.3  src 192.168.2.105  metric 20 
default via 10.0.0.138 dev wlan0  src 10.0.0.56  metric 30 
10.0.0.0/24 dev wlan0 scope link  metric 30 
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1 
192.168.1.0/24 dev eth0.2 scope link  metric 10 
192.168.2.0/24 dev eth0.3 scope link  metric 20


Output of "ip rule show"
-------------------------------------------------
0:	from all lookup local 
1001:	from all iif eth0.2 lookup 1 
1002:	from all iif eth0.3 lookup 2 
1003:	from all iif wlan0 lookup 3 
32766:	from all lookup main 
32767:	from all lookup default


Output of "ip route list table 1-250"
-------------------------------------------------
Table 1: default via 192.168.1.1 dev eth0.2  metric 10 
10.0.0.0/24 dev wlan0 scope link  metric 30 
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1 
192.168.1.0/24 dev eth0.2 scope link  metric 10 
192.168.2.0/24 dev eth0.3 scope link  metric 20
Table 2: default via 192.168.2.1 dev eth0.3  metric 20 
10.0.0.0/24 dev wlan0 scope link  metric 30 
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1 
192.168.1.0/24 dev eth0.2 scope link  metric 10 
192.168.2.0/24 dev eth0.3 scope link  metric 20
Table 3: default via 10.0.0.138 dev wlan0  metric 30 
10.0.0.0/24 dev wlan0 scope link  metric 30 
192.168.0.0/24 dev br-lan scope link  src 192.168.0.1 
192.168.1.0/24 dev eth0.2 scope link  metric 10 
192.168.2.0/24 dev eth0.3 scope link  metric 20


Output of "iptables -L -t mangle -v -n"
-------------------------------------------------
Chain PREROUTING (policy ACCEPT 34470 packets, 26M bytes)
 pkts bytes target     prot opt in     out     source               destination         
  19M   17G mwan3_hook  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain INPUT (policy ACCEPT 5500 packets, 1554K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 28341 packets, 24M bytes)
 pkts bytes target     prot opt in     out     source               destination         
  397 22676 TCPMSS     tcp  --  *      eth0.2  0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
  137  7560 TCPMSS     tcp  --  eth0.2 *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
    0     0 TCPMSS     tcp  --  *      wlan0   0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
    0     0 TCPMSS     tcp  --  wlan0  *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
    0     0 TCPMSS     tcp  --  *      eth0.3  0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
    0     0 TCPMSS     tcp  --  eth0.3 *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU

Chain OUTPUT (policy ACCEPT 3306 packets, 8954K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 280K   95M mwan3_hook  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain POSTROUTING (policy ACCEPT 31645 packets, 33M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain mwan3_connected (2 references)
 pkts bytes target     prot opt in     out     source               destination         
  11M   16G MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set mwan3_connected dst MARK or 0xff00

Chain mwan3_hook (2 references)
 pkts bytes target     prot opt in     out     source               destination         
  19M   17G CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore mask 0xff00
 546K   73M mwan3_ifaces_in  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00
 544K   71M mwan3_connected  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00
 274K   23M mwan3_rules  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00
  19M   17G CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save mask 0xff00
  18M   17G mwan3_connected  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match ! 0xff00/0xff00

Chain mwan3_iface_in_wanb (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   10  3616 MARK       all  --  eth0.3 *       0.0.0.0/0            0.0.0.0/0            match-set mwan3_connected src mark match 0x0/0xff00 /* default */ MARK or 0xff00
    3   252 MARK       all  --  eth0.3 *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00 /* wanb */ MARK xset 0x200/0xff00

Chain mwan3_iface_in_wwan (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  wlan0  *       0.0.0.0/0            0.0.0.0/0            match-set mwan3_connected src mark match 0x0/0xff00 /* default */ MARK or 0xff00
    0     0 MARK       all  --  wlan0  *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00 /* wwan */ MARK xset 0x300/0xff00

Chain mwan3_ifaces_in (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 106K   15M mwan3_iface_in_wanb  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00
 1289  223K mwan3_iface_in_wwan  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00

Chain mwan3_policy_drop (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00 /* unreachable */ MARK xset 0xfe00/0xff00

Chain mwan3_policy_loadbal (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  193 25581 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00 statistic mode random probability 0.39999999991 /* wanb 10 25 */ MARK xset 0x200/0xff00
  269 32087 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00 /* wan 15 15 */ MARK xset 0x100/0xff00

Chain mwan3_rules (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  462 57668 mwan3_policy_loadbal  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00