51 (edited by mikma 2012-11-20 13:08:33)

Re: mwan3; multi-wan policy routing (general topic)

I have problems with static routes. I think they should be added to mwan3_default the same way as local interfaces.

Is it possible to stop mwan3 without rebooting openwrt? I.e. how do I remove the route tables and iptables rules it adds?

Re: mwan3; multi-wan policy routing (general topic)

th3marco wrote:

Internet is now thanks for the help but the speed :-(

I reach only 6 Mbit / s

Hi Th3marco,


This is correct. Your download speed will never exceed the max of one interface the download is currently using. So in your scenario it would be a max of 10Mb.

If you want to have higher downloads, you should use more sessions.

Re: mwan3; multi-wan policy routing (general topic)

mikma wrote:

I have problems with static routes. I think they should be added to mwan3_default the same way as local interfaces.

If you want to use static routes, you should also add a mwan3 policy rule. For example if you have setup a static route for 192.168.0.0, you need to add the following policy rule:

config 'rule'
    option 'dest_ip' '192.168.0.0/16'
    option 'use_policy' 'default'
mikma wrote:

Is it possible to stop mwan3 without rebooting openwrt? I.e. how do I remove the route tables and iptables rules it adds?

Kinda, just remove all the ip rules that mwan3 has set. I will make a note of this and maybe in next release you could do a "/etc/init.d/mwan3 stop" to clear all rules.

Re: mwan3; multi-wan policy routing (general topic)

This is correct. Your download speed will never exceed the max of one interface the download is currently using. So in your scenario it would be a max of 10Mb.

If you want to have higher downloads, you should use more sessions.

More sessions? If i have
wan1= 2mb
wan2= 2mb
wan3= 10mb

should i have 14mb?
Thanks for your work!

Re: mwan3; multi-wan policy routing (general topic)

henrique.h12 wrote:

This is correct. Your download speed will never exceed the max of one interface the download is currently using. So in your scenario it would be a max of 10Mb.

If you want to have higher downloads, you should use more sessions.

More sessions? If i have
wan1= 2mb
wan2= 2mb
wan3= 10mb

should i have 14mb?

More or less... If you have for example a usenet download running with lets say 15 threads load-balanced, you should in theory reach a download speed of 14Mb.

Re: mwan3; multi-wan policy routing (general topic)

I tried :

opkg install mwan3_1.1-6_ar71xx.ipk luci-app-mwan3_0.1-1_ar71xx.ipk 
Installing mwan3 (1.1-6) to root...
Installing luci-app-mwan3 (0.1-1) to root...
Installing mwan3 (1.1-6) to root...
Configuring mwan3.
Configuring luci-app-mwan3.
Collected errors:
 * satisfy_dependencies_for: Cannot satisfy the following dependencies for mwan3:
 *      iptables-mod-conntrack-extra *  iptables-mod-ipopt * 
 * opkg_install_cmd: Cannot install package mwan3.

Any idea on what I could do to try it ?

Re: mwan3; multi-wan policy routing (general topic)

Will be asking to much an video tutorial like this one?
> http://www.youtube.com/watch?v=oNLss7btnP4
Thanks!

58 (edited by dir2cas 2012-12-21 00:51:43)

Re: mwan3; multi-wan policy routing (general topic)

Hello Adze, once again - thanks for all your efforts thrown in the mwan* architecture, it is really useful.

Yesterday I set up my network as a real dual wan environment, as I connected second ISP. As I am currently in a process of testing I am thinking about an option to be added to mwan3 that will be really handy - a rule that ensures that a request that come on a partucular interface will be answered by the same interface, despite the preferred default route in the main routing table.

This will be mostly a benefit in cases when you are running a server on the router or on another machine in the LAN that has a port fw set.
Such policy will affect mainly the uplink direction.

Let's say you have a service running on the second WAN real IP address, but the router's default route (main routing table) is pointing  to the GW of the first WAN. In such setup, the client will try to establish a session witht the service listening on WAN2 ,but will be answered with src IP address of WAN1. In most cases it will not allow a session to be established. There is no need to mention cases when the host has rp_filter enabled option in the kernel. A simple ping can prove it.

The idea is to check the incomming traffic on an interface for a new session (unmarked traffic as well to rule out traffic that is already caught by another policy) and mark it with the MARK label dedicated to the WANх only policy.

The option could be applied per interfece (in the interface statement in the mwan3 config)
ex.)
config 'interface' 'wan2'
    option 'enabled' '1'
    option 'persist_interface' '1'
.......

this way we may place rules like this:
.....
iptables -A mwan3_pre -i {WAN2} -t mangle -m mark --mark 0 -j CONNMARK --restore-mark
iptables -A mwan3_pre -i {WAN2} -m mark --mark 0 -m conntrack --ctstate NEW -j MARK --set-mark ${MARK_policy_WAN2}
....
etc.

This is only a suggestion, I do not claim to be accurate in the example as I have not studied mwan3 in details. We simply need to check if the option is enabled on the particular interface and if it is, a custom chain could be created where to place the marking for the new umarked sessions.

If you find this useful I may try to help, doing various tests, etc.

ps. Basically something like this is needed

Re: mwan3; multi-wan policy routing (general topic)

dir2cas wrote:

I am thinking about an option to be added to mwan3 that will be really handy - a rule that ensures that a request that come on a partucular interface will be answered by the same interface, despite the preferred default route in the main routing table.

This is already default behaviour, no need for extra option.

Re: mwan3; multi-wan policy routing (general topic)

Adze, Thank you this is great!

I am trying to use it simply for when my ISP goes down to use my 4g tethered phone.

I was having an issue in testing. When I unplugged the Ethernet cable to the router I did not see the switchover.

When I tacked it down I found mwan3_add_iptables_rule() was dying because ifvalue was null.
ticking Equalize changed the behavior.

My setup looks like:
1) Am I doing something wrong?
2) Is this the correct way to do what I want?

config interface 'wan'
        list track_ip '8.8.4.4'
        list track_ip '8.8.8.8'
        list track_ip '208.67.222.222'
        list track_ip '208.67.220.220'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'
        option enabled '1'
        option reliability '1'

config interface 'wan4G'
        option reliability '1'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'
        list track_ip '8.8.8.8'
        list track_ip '208.67.220.220'
        list track_ip '8.8.4.4'
        list track_ip '208.67.222.222'
        option enabled '1'

config member 'wan_m1_w1'
        option metric '1'
        option interface 'wan'
        option weight '500'

config member 'wan4g_m1_w1'
        option metric '1'
        option interface 'wan4G'
        option weight '500'

config policy 'wan_only'
        list use_member 'wan_m1_w1'

config policy 'wan4g_only'
        list use_member 'wan4g_m1_w1'

config policy 'wan_wan4g_loadbalanced'
        list use_member 'wan_m1_w1'
        list use_member 'wan4g_m1_w1'

config rule 'all_traffic'
        option use_policy 'wan1_wan2_loadbalanced'
        option proto 'all'

Re: mwan3; multi-wan policy routing (general topic)

david_s5 wrote:

My setup looks like:
1) Am I doing something wrong?

You have got an error in your config.

Replace

config rule 'all_traffic'
        option use_policy 'wan1_wan2_loadbalanced'
        option proto 'all'

With:

config rule 'all_traffic'
        option dest_ip '0.0.0.0/0'
        option use_policy 'wan_wan4g_loadbalanced'
        option proto 'all'

david_s5 wrote:

2) Is this the correct way to do what I want?

With your setup you have a load-balanced internet connection, where both interfaces are active.

Re: mwan3; multi-wan policy routing (general topic)

What it the right setup to just switch to 4G when the ISP on the wan is down?

Re: mwan3; multi-wan policy routing (general topic)

Hi Adze. thanks for your mwan3, it is realy amazing when I first use it.
I just changed some settings. But I'm not sure what the problem is.
Now, when I download a big file from the Internet. Almost data only via wan1 rather than both wan1 and wan2.

WAN1
eth0.2
RX: 649.45 MB (774047 Pkts.)
TX: 52.77 MB (458188 Pkts.)
IPv4: 172.24.211.202/23
     
WAN2
eth2
Uptime: 0h 23m 27s
MAC-Address: 0:11:22:33:44:5E
RX: 1.27 MB (9763 Pkts.)
TX: 45.54 KB (1060 Pkts.)
IPv4: 172.24.211.20/23

Here are my config files:

/etc/config/network

config interface 'loopback'
    option ifname 'lo'
    option proto 'static'
    option ipaddr '127.0.0.1'
    option netmask '255.0.0.0'

config interface 'lan'
    option ifname 'eth0.1'
    option type 'bridge'
    option proto 'static'
    option ipaddr '192.168.1.1'
    option netmask '255.255.255.0'
    option macaddr 'c8:60:00:95:56:93'

config interface 'wan1'
    option ifname 'eth0.2'
    option proto 'dhcp'
    option macaddr 'c8:60:00:95:56:92'
    option metric '10'

config interface 'wan2'
    option proto 'dhcp'
    option ifname 'eth2'
    option peerdns '0'
    option dns '8.8.8.8 8.8.4.4'
    option metric '10'

config switch
    option name 'switch0'
    option reset '1'
    option enable_vlan '1'

config switch_vlan
    option device 'switch0'
    option vlan '1'
    option ports '0 1 2 3 8t'

config switch_vlan
    option device 'switch0'
    option vlan '2'
    option ports '4 8t'

Another for firewall:

/etc/config/firewall


config defaults
    option syn_flood '1'
    option output 'ACCEPT'
    option forward 'REJECT'
    option input 'ACCEPT'

config zone
    option name 'lan'
    option network 'lan'
    option input 'ACCEPT'
    option output 'ACCEPT'
    option forward 'REJECT'

config zone
    option name 'wan'
    option input 'REJECT'
    option output 'ACCEPT'
    option forward 'REJECT'
    option masq '1'
    option mtu_fix '1'
    option network 'wan1'

config rule
    option name 'Allow-DHCP-Renew'
    option src 'wan'
    option proto 'udp'
    option dest_port '68'
    option target 'ACCEPT'
    option family 'ipv4'

config rule
    option name 'Allow-Ping'
    option src 'wan'
    option proto 'icmp'
    option icmp_type 'echo-request'
    option family 'ipv4'
    option target 'ACCEPT'

config rule
    option name 'Allow-DHCPv6'
    option src 'wan'
    option proto 'udp'
    option src_ip 'fe80::/10'
    option src_port '547'
    option dest_ip 'fe80::/10'
    option dest_port '546'
    option family 'ipv6'
    option target 'ACCEPT'

config rule
    option name 'Allow-ICMPv6-Input'
    option src 'wan'
    option proto 'icmp'
    list icmp_type 'echo-request'
    list icmp_type 'echo-reply'
    list icmp_type 'destination-unreachable'
    list icmp_type 'packet-too-big'
    list icmp_type 'time-exceeded'
    list icmp_type 'bad-header'
    list icmp_type 'unknown-header-type'
    list icmp_type 'router-solicitation'
    list icmp_type 'neighbour-solicitation'
    list icmp_type 'router-advertisement'
    list icmp_type 'neighbour-advertisement'
    option limit '1000/sec'
    option family 'ipv6'
    option target 'ACCEPT'

config rule
    option name 'Allow-ICMPv6-Forward'
    option src 'wan'
    option dest '*'
    option proto 'icmp'
    list icmp_type 'echo-request'
    list icmp_type 'echo-reply'
    list icmp_type 'destination-unreachable'
    list icmp_type 'packet-too-big'
    list icmp_type 'time-exceeded'
    list icmp_type 'bad-header'
    list icmp_type 'unknown-header-type'
    option limit '1000/sec'
    option family 'ipv6'
    option target 'ACCEPT'

config include
    option path '/etc/firewall.user'

config zone
    option name 'wan2'
    option forward 'REJECT'
    option output 'ACCEPT'
    option network 'wan2'
    option input 'REJECT'
    option masq '1'
    option mtu_fix '1'

config forwarding
    option dest 'wan'
    option src 'lan'

config forwarding
    option dest 'wan2'
    option src 'lan'

And config for mwan3:

/etc/config/mwan3


config interface 'wan1'
    option enabled '1'
    option reliability '2'
    option count '1'
    option timeout '2'
    option interval '5'
    option down '3'
    option up '8'
    list track_ip '8.8.4.4'
    list track_ip '8.8.8.8'

config member 'wan1_m1_w1'
    option interface 'wan1'
    option metric '1'
    option weight '500'

config member 'wan2_m1_w1'
    option interface 'wan2'
    option metric '1'
    option weight '500'

config policy 'wan1_only'
        list use_member 'wan1_m1_w1'

config policy 'wan2_only'
        list use_member 'wan2_m1_w1'

config policy 'wan1_wan2_loadbalanced'
    list use_member 'wan1_m1_w1'
    list use_member 'wan2_m1_w1'

config rule 'all_traffic'
        option dest_ip '0.0.0.0/0'
        option use_policy 'wan1_wan2_loadbalanced'
        option proto 'all'

config interface 'wan2'
    option enabled '1'
    list track_ip '8.8.8.8'
    list track_ip '8.8.4.4'
    option reliability '2'
    option count '1'
    option timeout '2'
    option interval '5'
    option down '3'
    option up '8'

Does my config have any problem?
Or it is because wan2 and wan1 have the same gateway, which make mwan3 not work?

Can you give me any hints? Thanks in advance.

Re: mwan3; multi-wan policy routing (general topic)

yanggis wrote:

Does my config have any problem?.

Yes, both your wan interfaces have the same metric value in /etc/config/network. But changing this won't help you because of your next question..

yanggis wrote:

Or it is because wan2 and wan1 have the same gateway, which make mwan3 not work?

Yes, mwan3 does not work with multiple same subnet wan interfaces.

Re: mwan3; multi-wan policy routing (general topic)

How would one configure mwan3 for simple fail over to 4G when the wan is down?

So No load balancing and not using the 4G when the wan is up at all.

Re: mwan3; multi-wan policy routing (general topic)

david_s5 wrote:

How would one configure mwan3 for simple fail over to 4G when the wan is down?

So No load balancing and not using the 4G when the wan is up at all.

Play with the metric and weight settings in mwan3 config. Interfaces with a lower metric have precedence over interfaces with a higher metric. If multiple online interfaces have the same lowest metric value, the load will be distributed based on the weight value. An example config for you could be:


config interface 'wan'
        list track_ip '8.8.4.4'
        list track_ip '8.8.8.8'
        list track_ip '208.67.222.222'
        list track_ip '208.67.220.220'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'
        option enabled '1'
        option reliability '1'

config interface 'wan4G'
        option reliability '1'
        option count '1'
        option timeout '4'
        option interval '30'
        option down '4'
        option up '8'
        list track_ip '8.8.8.8'
        list track_ip '208.67.220.220'
        list track_ip '8.8.4.4'
        list track_ip '208.67.222.222'
        option enabled '1'

config member 'wan_m1_w1'
        option metric '1'
        option interface 'wan'
        option weight '1'

config member 'wan4g_m1_w1'
        option metric '2'
        option interface 'wan4G'
        option weight '1'

config policy 'wan_only'
        list use_member 'wan_m1_w1'

config policy 'wan4g_only'
        list use_member 'wan4g_m1_w1'

config policy 'wan_wan4g_loadbalanced'
        list use_member 'wan_m1_w1'
        list use_member 'wan4g_m1_w1'

config rule 'all_traffic'
        option use_policy 'wan_wan4g_loadbalanced'
        option proto 'all'
        option dest_ip '0.0.0.0/0'

67 (edited by yanggis 2013-01-05 10:26:11)

Re: mwan3; multi-wan policy routing (general topic)

Dear Adze,
Sorry to bother you again.
After using mwan3 for a few days, I got another problem.
When I power on my router, the cpu load is very low, such as Load: 0.57 0.19 0.07.
However, after the router running some days, for example, one week, the  cpu load becomes very high, such as Load: 3.27 2.45 3.76.
I also tried Multi-wan software, when I used that software, this case did not happen.
Once I use mwan3, this case happened again.
Here are my config files:

/etc/config/network



config interface 'loopback'
    option ifname 'lo'
    option proto 'static'
    option ipaddr '127.0.0.1'
    option netmask '255.0.0.0'

config interface 'lan'
    option ifname 'eth0.1'
    option type 'bridge'
    option proto 'static'
    option ipaddr '192.168.1.1'
    option netmask '255.255.255.0'
    option macaddr 'c8:60:00:95:56:93'

config interface 'wan1'
    option ifname 'eth0.2'
    option proto 'dhcp'
    option macaddr 'c8:60:00:95:56:92'
    option metric '10'

config interface 'wan2'
    option ifname 'eth2'
    option proto 'dhcp'
    option macaddr 'C0:11:22:33:44:5E'
    option metric '9'

config switch
    option name 'switch0'
    option reset '1'
    option enable_vlan '1'

config switch_vlan
    option device 'switch0'
    option vlan '1'
    option ports '0 1 2 3 8t'

config switch_vlan
    option device 'switch0'
    option vlan '2'
    option ports '4 8t'


---------------------------------------------------------------------------------------------------------------

/etc/config/mwan3


config rule
    option dest_ip '0.0.0.0/0'
    option use_policy 'wan1_wan2_loadbalanced'
    option proto 'all'
    option equalize '1'

config rule
    option src_ip '192.168.21.0/24'
    option proto 'tcp'
    option dest_port '563'
    option use_policy 'wan2_only'

config rule
    option src_ip '192.168.21.0/24'
    option proto 'tcp'
    option dest_port '995'
    option use_policy 'wan1_only'

config rule
    option dest_ip '88.154.0.0/16'
    option proto 'tcp'
    option dest_port '1024:65535'
    option equalize '1'
    option use_policy 'wan1_wan2_loadbalanced'

config rule
    option dest_ip '77.11.41.0/24'
    option proto 'tcp'
    option dest_port '1024:65535'
    option use_policy 'wan1_pri_wan2_sec'

config rule
    option dest_ip '112.136.0.0/16'
    option proto 'udp'
    option dest_port '5352'
    option use_policy 'wan2_pri_wan1_sec'

config interface 'wan1'
    option enabled '1'
    list track_ip '8.8.4.4'
    list track_ip '8.8.8.8'
    list track_ip '208.67.222.222'
    list track_ip '208.67.220.220'
    option reliability '2'
    option count '1'
    option timeout '2'
    option interval '5'
    option down '3'
    option up '8'

config interface 'wan2'
    option enabled '1'
    list track_ip '8.8.8.8'
    list track_ip '208.67.220.220'
    option reliability '1'
    option count '1'
    option timeout '2'
    option interval '5'
    option down '3'
    option up '8'

config member 'wan1_m1_w3'
    option interface 'wan1'
    option metric '1'
    option weight '3'

config member 'wan1_m2_w3'
    option interface 'wan1'
    option metric '2'
    option weight '3'

config member 'wan2_m1_w2'
    option interface 'wan2'
    option metric '1'
    option weight '2'

config member 'wan2_m2_w2'
    option interface 'wan2'
    option metric '2'
    option weight '2'

config policy 'wan1_only'
    list use_member 'wan1_m1_w3'

config policy 'wan2_only'
    list use_member 'wan2_m1_w2'

config policy 'wan1_wan2_loadbalanced'
    list use_member 'wan1_m1_w3'
    list use_member 'wan2_m1_w2'

config policy 'wan1_pri_wan2_sec'
    list use_member 'wan1_m1_w3'
    list use_member 'wan2_m2_w2'

config policy 'wan2_pri_wan1_sec'
    list use_member 'wan1_m2_w3'
    list use_member 'wan2_m1_w2'

-------------------------------------------------------------------------------------

/etc/config/firewall


config defaults
    option syn_flood '1'
    option input 'ACCEPT'
    option output 'ACCEPT'
    option forward 'REJECT'

config zone
    option name 'lan'
    option network 'lan'
    option input 'ACCEPT'
    option output 'ACCEPT'
    option forward 'REJECT'

config include
    option path '/etc/firewall.user'

config zone
    option name 'wan1'
    option input 'ACCEPT'
    option forward 'REJECT'
    option output 'ACCEPT'
    option network 'wan1'
    option masq '1'
    option mtu_fix '1'

config zone
    option name 'wan2'
    option input 'ACCEPT'
    option forward 'REJECT'
    option output 'ACCEPT'
    option network 'wan2'
    option masq '1'
    option mtu_fix '1'

config forwarding
    option dest 'wan1'
    option src 'lan'

config forwarding
    option dest 'wan2'
    option src 'lan'

config redirect
    option target 'DNAT'
    option src 'wan'
    option dest 'lan'
    option proto 'tcp'
    option src_dport '3389'
    option dest_ip '192.168.1.223'
    option dest_port '3389'
    option name 'remote control'

config redirect
    option target 'DNAT'
    option src 'wan'
    option dest 'lan'
    option proto 'tcp udp'
    option src_dport '38862'
    option dest_ip '192.168.1.223'
    option dest_port '38862'
    option name 'Utorrent'

-------------------------------------------------------------------------------------------------
Can you kindly give me a hint, what is the problem?
Thanks in advance.

68 (edited by natasia13 2013-01-06 14:11:29)

Re: mwan3; multi-wan policy routing (general topic)

Dear Adze,

I am using OpenWRT trunk r34778.  I have two physical wans, both are using dhcp mode, but I can see only one Gateway on the routing table after "route -n", normally it should be two to get MWAN working, isn't it?

/root$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.20.20   0.0.0.0         UG    0      0        0 eth0.2
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.1
192.168.20.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.2 

You can see that eth0.1 is my wan1 and eth0.2 is my wan2.

and then I tried to ping www.google.com :

ping -c 1 -I eth0.1 www.google.com  not successful    (but in fact both eth0.1 & eth0.2 work fine when I'm trying NWAN)

ping -c 1 -I eth0.2 www.google.com   successful

From you post, I knew that to get mwan3 working, it should be:

/root$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.10.10   0.0.0.0         UG    0      0        0 eth0.1
0.0.0.0         192.168.20.20   0.0.0.0         UG    0      0        0 eth0.2
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.1
192.168.20.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.2 

I don't know why my eth0.1's Gateway was not showing on the routing table, does it have something to do with version of firmware?

/$ ip route
default 
    nexthop via 192.168.10.10  dev eth0.1 weight 1
    nexthop via 192.168.20.20  dev eth0.2 weight 1
192.168.3.0/24 dev br-lan  proto kernel  scope link  src 192.168.3.1 
192.168.10.0/24 dev eth0.1  proto kernel  scope link  src 192.168.10.13 
192.168.20.0/24 dev eth0.2  proto kernel  scope link  src 192.168.20.105 


/$ ip rule show
0:    from all lookup local 
32762:    from all fwmark 0x1e lookup wan2_rt 
32763:    from 192.168.20.105 lookup wan2_rt 
32764:    from all fwmark 0x14 lookup wan1_rt 
32765:    from 192.168.10.13 lookup wan1_rt 
32766:    from all lookup main 
32767:    from all lookup default 

Thanks in advance.

Regards,
Natasia

Re: mwan3; multi-wan policy routing (general topic)

natasia13 wrote:

I have two physical wans, both are using dhcp mode, but I can see only one Gateway on the routing table after "route -n", normally it should be two to get MWAN working, isn't it?

Yes, i believe you did not set different metric values in your network config, as described in OP.

Re: mwan3; multi-wan policy routing (general topic)

Is this Multi-Wan or an other package?
I try to route thru different wan's based on source network, which is not chooseable with Multi-WAN.
Please see https://forum.openwrt.org/viewtopic.php?id=41518

Thanks in advance,
regards Florian.

Re: mwan3; multi-wan policy routing (general topic)

Adze wrote:
natasia13 wrote:

I have two physical wans, both are using dhcp mode, but I can see only one Gateway on the routing table after "route -n", normally it should be two to get MWAN working, isn't it?

Yes, i believe you did not set different metric values in your network config, as described in OP.

Hello Adze, as you can see, already set wan1 metric value 1 & wan2 metric value 2.

/$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.10.10   0.0.0.0         UG    0      0        0 eth0.1
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.10.0    0.0.0.0         255.255.255.0   U     1      0        0 eth0.1
192.168.20.0    0.0.0.0         255.255.255.0   U     2      0        0 eth0.2
config interface 'wan1'
    option _orig_bridge 'false'
    option ifname 'eth0.1'
    option _orig_ifname 'eth0.1'
    option proto 'dhcp'
    option metric '1'

config interface 'wan2'
    option ifname 'eth0.2'
    option _orig_bridge 'false'
    option _orig_ifname 'eth0.2'
    option proto 'dhcp'
    option metric '2'

Re: mwan3; multi-wan policy routing (general topic)

natasia13 wrote:

Hello Adze, as you can see, already set wan1 metric value 1 & wan2 metric value 2.

Hmmm... The outcome of "route -n" does not reflect your network config.. Could you try and remove the lines "option _orig_bridge 'false'" and "option _orig_ifname 'eth0.x'" please?

Re: mwan3; multi-wan policy routing (general topic)

Adze wrote:
natasia13 wrote:

Hello Adze, as you can see, already set wan1 metric value 1 & wan2 metric value 2.

Hmmm... The outcome of "route -n" does not reflect your network config.. Could you try and remove the lines "option _orig_bridge 'false'" and "option _orig_ifname 'eth0.x'" please?


Hello Adze, thanks so much for the prompt response.   The outcome's still the same after removing those two lines.

/etc/config$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.10.10   0.0.0.0         UG    0      0        0 eth0.1
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.10.0    0.0.0.0         255.255.255.0   U     1      0        0 eth0.1
192.168.20.0    0.0.0.0         255.255.255.0   U     2      0        0 eth0.2

my network config as below:

config interface 'loopback'
    option ifname 'lo'
    option proto 'static'
    option ipaddr '127.0.0.1'
    option netmask '255.0.0.0'

config interface 'lan'
    option type 'bridge'
    option proto 'static'
    option netmask '255.255.255.0'
    option ipaddr '192.168.3.1'
    option _orig_ifname 'wlan0 wlan1'
    option _orig_bridge 'true'
    option ifname 'eth0.3'

config interface 'wan1'
    option ifname 'eth0.1'
    option proto 'dhcp'
    option metric '1'

config interface 'wan2'
    option ifname 'eth0.2'
    option proto 'dhcp'
    option metric '2'

config switch
    option name 'eth0'
    option reset '1'
    option enable_vlan '1'

config switch_vlan
    option device 'eth0'
    option vlan '1'
    option vid '1'
    option ports '0t 1'

config switch_vlan
    option device 'eth0'
    option vlan '2'
    option vid '2'
    option ports '0t 2'

config switch_vlan
    option device 'eth0'
    option vlan '3'
    option vid '3'
    option ports '0t 3 4 5'

74 (edited by sagitar 2013-01-14 05:27:04)

Re: mwan3; multi-wan policy routing (general topic)

Hi,Adze.
The mwan3 is very usefull for me, I have used it for several months. Thank you very much.
However,I met some troubles on it today.

I found the IP rules can be created if an interface up,however ,it can NOT be deleted when the interface down.  Could you please check the configure for me ?

(update: I updated a new firmware for my router and re-configured the mwan3 then found the issue.)


root@OpenWrt:~# ip rule show
0:      from all lookup local
1001:   from 192.168.119.145 fwmark 0x0/0x8000 lookup 1001
1002:   from 192.168.111.245 fwmark 0x0/0x8000 lookup 1002
1003:   from 192.168.120.245 fwmark 0x0/0x8000 lookup 1003
1004:   from 192.168.122.145 fwmark 0x0/0x8000 lookup 1004
1008:   from all fwmark 0x100/0xff00 lookup 1001
1009:   from all fwmark 0x200/0xff00 lookup 1002
1010:   from all fwmark 0x300/0xff00 lookup 1003
1011:   from all fwmark 0x400/0xff00 lookup 1004
1016:   from all fwmark 0x1000/0xff00 lookup 1016
1017:   from all fwmark 0x1100/0xff00 lookup 1017
1018:   from all fwmark 0x1200/0xff00 lookup 1018
1019:   from all fwmark 0x1300/0xff00 lookup 1019
1020:   from all fwmark 0x1400/0xff00 lookup 1020
32766:  from all lookup main
32767:  from all lookup default
root@OpenWrt:~# logread|grep mwan3
Jan 13 19:56:36 OpenWrt user.notice root: mwan3: ifup interface wan4 (eth0.4)
Jan 13 19:56:59 OpenWrt user.notice root: mwan3: No gateway found for interface wan (eth0.1)
Jan 13 19:57:18 OpenWrt user.notice root: mwan3: No gateway found for interface wan2 (eth0.2)
Jan 13 19:57:35 OpenWrt user.notice root: mwan3: No gateway found for interface wan3 (eth0.3)
Jan 13 19:57:41 OpenWrt user.notice root: mwan3: ifup interface wan4 (eth0.4)
Jan 13 20:03:14 OpenWrt user.notice root: mwan3: ifdown interface wan4 (eth0.4)
Jan 13 20:03:52 OpenWrt user.notice root: mwan3: ifup interface wan4 (eth0.4)
Jan 13 20:05:28 OpenWrt user.notice root: mwan3: Lost 1 ping(s) on interface wan4 (eth0.4)
Jan 13 20:32:39 OpenWrt user.notice root: mwan3: Lost 1 ping(s) on interface wan4 (eth0.4)
Jan 13 21:21:18 OpenWrt user.notice root: mwan3: Lost 1 ping(s) on interface wan4 (eth0.4)
Jan 13 21:32:03 OpenWrt user.notice root: mwan3: ifup interface wan (eth0.1)
Jan 13 21:32:23 OpenWrt user.notice root: mwan3: ifup interface wan2 (eth0.2)
Jan 13 21:32:34 OpenWrt user.notice root: mwan3: ifup interface wan3 (eth0.3)
Jan 13 21:32:40 OpenWrt user.notice root: mwan3: Interface wan (eth0.1) is offline
Jan 13 21:32:42 OpenWrt user.notice root: mwan3: ifdown interface wan (eth0.1)
Jan 13 21:32:59 OpenWrt user.notice root: mwan3: Interface wan2 (eth0.2) is offline
Jan 13 21:32:59 OpenWrt user.notice root: mwan3: ifdown interface wan2 (eth0.2)
Jan 13 21:33:11 OpenWrt user.notice root: mwan3: Interface wan3 (eth0.3) is offline
Jan 13 21:33:11 OpenWrt user.notice root: mwan3: ifdown interface wan3 (eth0.3)
root@OpenWrt:~#

/etc/config/network


root@OpenWrt:~# cat /etc/config/network
#### VLAN configuration
config switch eth0
        option enable   1

config switch_vlan eth0_0
        option device   "eth0"
        option vlan     0
        option ports    "4 8*"

config switch_vlan eth0_1
        option device   "eth0"
        option vlan     1
        option ports    "0 8"

config switch_vlan eth0_2
        option device   "eth0"
        option vlan     2
        option ports    "1 8"

config switch_vlan eth0_3
        option device   "eth0"
        option vlan     3
        option ports    "2 8"

config switch_vlan eth0_4
        option device   "eth0"
        option vlan     4
        option ports    "3 8"

#### Loopback configuration
config interface loopback
        option ifname   "lo"
        option proto    static
        option ipaddr   127.0.0.1
        option netmask  255.0.0.0


#### LAN configuration
config interface lan
        option type     bridge
        option ifname   "eth0.0"
        option proto    static
        option ipaddr   192.168.115.100
        option netmask  255.255.255.0


#### WAN configuration
config interface        wan
        option ifname   "eth0.1"
        option proto    dhcp
        option metric 40
        option accept_ra 1

config interface        wan2
        option ifname   "eth0.2"
        option proto    dhcp
        option metric 30
        option accept_ra 1

config interface        wan3
        option ifname   "eth0.3"
        option proto    dhcp
        option metric 20
        option accept_ra 1

config interface        wan4
        option ifname   "eth0.4"
        option proto    dhcp
        option metric 10
        option accept_ra 1
root@OpenWrt:~#

/etc/config/firware


root@OpenWrt:~# cat /etc/config/firewall
config defaults
        option syn_flood        1
        option input            DROP
        option output           ACCEPT
        option forward          REJECT
# Uncomment this line to disable ipv6 rules
#       option disable_ipv6     1

config zone
        option name             lan
        option network          'lan'
        option input            ACCEPT
        option output           ACCEPT
        option forward          REJECT

config zone
        option name             wan
        option network          'wan wan2 wan3 wan4'
        option input            REJECT
        option output           ACCEPT
        option forward          REJECT
        option masq             1
        option mtu_fix          1

config forwarding
        option src              lan
        option dest             wan

# We need to accept udp packets on port 68,
# see https://dev.openwrt.org/ticket/4108
config rule
        option name             Allow-DHCP-Renew
        option src              wan
        option proto            udp
        option dest_port        68
        option target           ACCEPT
        option family           ipv4

# Allow IPv4 ping
config rule
        option name             Allow-Ping
        option src              wan
        option proto            icmp
        option icmp_type        echo-request
        option family           ipv4
        option target           ACCEPT

# Allow DHCPv6 replies
# see https://dev.openwrt.org/ticket/10381
config rule
        option name             Allow-DHCPv6
        option src              wan
        option proto            udp
        option src_ip           fe80::/10
        option src_port         547
        option dest_ip          fe80::/10
        option dest_port        546
        option family           ipv6
        option target           ACCEPT

# Allow essential incoming IPv6 ICMP traffic
config rule
        option name             Allow-ICMPv6-Input
        option src              wan
        option proto    icmp
        list icmp_type          echo-request
        list icmp_type          echo-reply
        list icmp_type          destination-unreachable
        list icmp_type          packet-too-big
        list icmp_type          time-exceeded
        list icmp_type          bad-header
        list icmp_type          unknown-header-type
        list icmp_type          router-solicitation
        list icmp_type          neighbour-solicitation
        list icmp_type          router-advertisement
        list icmp_type          neighbour-advertisement
        option limit            1000/sec
        option family           ipv6
        option target           ACCEPT

# Allow essential forwarded IPv6 ICMP traffic
config rule
        option name             Allow-ICMPv6-Forward
        option src              wan
        option dest             *
        option proto            icmp
        list icmp_type          echo-request
        list icmp_type          echo-reply
        list icmp_type          destination-unreachable
        list icmp_type          packet-too-big
        list icmp_type          time-exceeded
        list icmp_type          bad-header
        list icmp_type          unknown-header-type
        option limit            1000/sec
        option family           ipv6
        option target           ACCEPT

# include a file with users custom iptables rules
config include
        option path /etc/firewall.user


### EXAMPLE CONFIG SECTIONS
# do not allow a specific ip to access wan
#config rule
#       option src              lan
#       option src_ip   192.168.45.2
#       option dest             wan
#       option proto    tcp
#       option target   REJECT

# block a specific mac on wan
#config rule
#       option dest             wan
#       option src_mac  00:11:22:33:44:66
#       option target   REJECT

# block incoming ICMP traffic on a zone
#config rule
#       option src              lan
#       option proto    ICMP
#       option target   DROP

# port redirect port coming in on wan to lan
#config redirect
#       option src                      wan
#       option src_dport        80
#       option dest                     lan
#       option dest_ip          192.168.16.235
#       option dest_port        80
#       option proto            tcp

# port redirect of remapped ssh port (22001) on wan
#config redirect
#       option src              wan
#       option src_dport        22001
#       option dest             lan
#       option dest_port        22
#       option proto            tcp

# allow IPsec/ESP and ISAKMP passthrough
#config rule
#       option src              wan
#       option dest             lan
#       option protocol         esp
#       option target           ACCEPT

#config rule
#       option src              wan
#       option dest             lan
#       option src_port         500
#       option dest_port        500
#       option proto            udp
#       option target           ACCEPT

### FULL CONFIG SECTIONS
#config rule
#       option src              lan
#       option src_ip   192.168.45.2
#       option src_mac  00:11:22:33:44:55
#       option src_port 80
#       option dest             wan
#       option dest_ip  194.25.2.129
#       option dest_port        120
#       option proto    tcp
#       option target   REJECT

#config redirect
#       option src              lan
#       option src_ip   192.168.45.2
#       option src_mac  00:11:22:33:44:55
#       option src_port         1024
#       option src_dport        80
#       option dest_ip  194.25.2.129
#       option dest_port        120
#       option proto    tcp
root@OpenWrt:~#

/etc/config/mwan3


root@OpenWrt:~# cat /etc/config/mwan3

config member 'wan_m2_w3'
        option weight '3'
        option interface 'wan'
        option metric '2'

config member 'wan2_m2_w3'
        option interface 'wan2'
        option metric '2'
        option weight '3'

config member 'wan3_m2_w3'
        option interface 'wan3'
        option metric '2'
        option weight '3'

config member 'wan4_m2_w3'
        option interface 'wan4'
        option metric '2'
        option weight '3'

config policy 'wan1_only'
        list use_member 'wan1_m1_w3'

config policy 'wan2_only'
        list use_member 'wan2_m1_w2'

config policy 'wan_wan2_wan3_wan4_loadbalanced'
        list use_member 'wan_m2_w3'
        list use_member 'wan2_m2_w3'
        list use_member 'wan3_m2_w3'
        list use_member 'wan4_m2_w3'

config policy 'wan1_pri_wan2_sec'
        list use_member 'wan1_m1_w3'
        list use_member 'wan2_m2_w2'

config policy 'wan2_pri_wan1_sec'
        list use_member 'wan1_m2_w3'
        list use_member 'wan2_m1_w2'

config rule
        option dest_ip '0.0.0.0/0'
        option proto 'all'
        option use_policy 'wan_wan2_wan3_wan4_loadbalanced'
        option equalize '1'

config interface 'wan'
        option enabled '1'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option reliability '2'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option up '8'

config interface 'wan2'
        option enabled '1'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option reliability '2'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'

config interface 'wan3'
        option enabled '1'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option reliability '2'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'

config interface 'wan4'
        option enabled '1'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option reliability '2'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'

root@OpenWrt:~#

route -n


root@OpenWrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         127.0.0.1       0.0.0.0         UG    0      0        0 lo
0.0.0.0         192.168.122.1   0.0.0.0         UG    10     0        0 eth0.4
0.0.0.0         192.168.120.1   0.0.0.0         UG    20     0        0 eth0.3
0.0.0.0         192.168.111.1   0.0.0.0         UG    30     0        0 eth0.2
0.0.0.0         192.168.119.1   0.0.0.0         UG    40     0        0 eth0.1
192.168.111.0   0.0.0.0         255.255.255.0   U     30     0        0 eth0.2
192.168.115.0   0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.119.0   0.0.0.0         255.255.255.0   U     40     0        0 eth0.1
192.168.120.0   0.0.0.0         255.255.255.0   U     20     0        0 eth0.3
192.168.122.0   0.0.0.0         255.255.255.0   U     10     0        0 eth0.4
root@OpenWrt:~#


Thank you in advance.

Re: mwan3; multi-wan policy routing (general topic)

I uninstalled the mwan3 and re-installed it using the package in r33446, the mwan3 works very well. so, maybe it  is a mwan3 issue in r34640.


sagitar wrote:

Hi,Adze.
The mwan3 is very usefull for me, I have used it for several months. Thank you very much.
However,I met some troubles on it today.

I found the IP rules can be created if an interface up,however ,it can NOT be deleted when the interface down.  Could you please check the configure for me ?

(update: I updated a new firmware for my router and re-configured the mwan3 then found the issue.)


root@OpenWrt:~# ip rule show
0:      from all lookup local
1001:   from 192.168.119.145 fwmark 0x0/0x8000 lookup 1001
1002:   from 192.168.111.245 fwmark 0x0/0x8000 lookup 1002
1003:   from 192.168.120.245 fwmark 0x0/0x8000 lookup 1003
1004:   from 192.168.122.145 fwmark 0x0/0x8000 lookup 1004
1008:   from all fwmark 0x100/0xff00 lookup 1001
1009:   from all fwmark 0x200/0xff00 lookup 1002
1010:   from all fwmark 0x300/0xff00 lookup 1003
1011:   from all fwmark 0x400/0xff00 lookup 1004
1016:   from all fwmark 0x1000/0xff00 lookup 1016
1017:   from all fwmark 0x1100/0xff00 lookup 1017
1018:   from all fwmark 0x1200/0xff00 lookup 1018
1019:   from all fwmark 0x1300/0xff00 lookup 1019
1020:   from all fwmark 0x1400/0xff00 lookup 1020
32766:  from all lookup main
32767:  from all lookup default
root@OpenWrt:~# logread|grep mwan3
Jan 13 19:56:36 OpenWrt user.notice root: mwan3: ifup interface wan4 (eth0.4)
Jan 13 19:56:59 OpenWrt user.notice root: mwan3: No gateway found for interface wan (eth0.1)
Jan 13 19:57:18 OpenWrt user.notice root: mwan3: No gateway found for interface wan2 (eth0.2)
Jan 13 19:57:35 OpenWrt user.notice root: mwan3: No gateway found for interface wan3 (eth0.3)
Jan 13 19:57:41 OpenWrt user.notice root: mwan3: ifup interface wan4 (eth0.4)
Jan 13 20:03:14 OpenWrt user.notice root: mwan3: ifdown interface wan4 (eth0.4)
Jan 13 20:03:52 OpenWrt user.notice root: mwan3: ifup interface wan4 (eth0.4)
Jan 13 20:05:28 OpenWrt user.notice root: mwan3: Lost 1 ping(s) on interface wan4 (eth0.4)
Jan 13 20:32:39 OpenWrt user.notice root: mwan3: Lost 1 ping(s) on interface wan4 (eth0.4)
Jan 13 21:21:18 OpenWrt user.notice root: mwan3: Lost 1 ping(s) on interface wan4 (eth0.4)
Jan 13 21:32:03 OpenWrt user.notice root: mwan3: ifup interface wan (eth0.1)
Jan 13 21:32:23 OpenWrt user.notice root: mwan3: ifup interface wan2 (eth0.2)
Jan 13 21:32:34 OpenWrt user.notice root: mwan3: ifup interface wan3 (eth0.3)
Jan 13 21:32:40 OpenWrt user.notice root: mwan3: Interface wan (eth0.1) is offline
Jan 13 21:32:42 OpenWrt user.notice root: mwan3: ifdown interface wan (eth0.1)
Jan 13 21:32:59 OpenWrt user.notice root: mwan3: Interface wan2 (eth0.2) is offline
Jan 13 21:32:59 OpenWrt user.notice root: mwan3: ifdown interface wan2 (eth0.2)
Jan 13 21:33:11 OpenWrt user.notice root: mwan3: Interface wan3 (eth0.3) is offline
Jan 13 21:33:11 OpenWrt user.notice root: mwan3: ifdown interface wan3 (eth0.3)
root@OpenWrt:~#

/etc/config/network


root@OpenWrt:~# cat /etc/config/network
#### VLAN configuration
config switch eth0
        option enable   1

config switch_vlan eth0_0
        option device   "eth0"
        option vlan     0
        option ports    "4 8*"

config switch_vlan eth0_1
        option device   "eth0"
        option vlan     1
        option ports    "0 8"

config switch_vlan eth0_2
        option device   "eth0"
        option vlan     2
        option ports    "1 8"

config switch_vlan eth0_3
        option device   "eth0"
        option vlan     3
        option ports    "2 8"

config switch_vlan eth0_4
        option device   "eth0"
        option vlan     4
        option ports    "3 8"

#### Loopback configuration
config interface loopback
        option ifname   "lo"
        option proto    static
        option ipaddr   127.0.0.1
        option netmask  255.0.0.0


#### LAN configuration
config interface lan
        option type     bridge
        option ifname   "eth0.0"
        option proto    static
        option ipaddr   192.168.115.100
        option netmask  255.255.255.0


#### WAN configuration
config interface        wan
        option ifname   "eth0.1"
        option proto    dhcp
        option metric 40
        option accept_ra 1

config interface        wan2
        option ifname   "eth0.2"
        option proto    dhcp
        option metric 30
        option accept_ra 1

config interface        wan3
        option ifname   "eth0.3"
        option proto    dhcp
        option metric 20
        option accept_ra 1

config interface        wan4
        option ifname   "eth0.4"
        option proto    dhcp
        option metric 10
        option accept_ra 1
root@OpenWrt:~#

/etc/config/firware


root@OpenWrt:~# cat /etc/config/firewall
config defaults
        option syn_flood        1
        option input            DROP
        option output           ACCEPT
        option forward          REJECT
# Uncomment this line to disable ipv6 rules
#       option disable_ipv6     1

config zone
        option name             lan
        option network          'lan'
        option input            ACCEPT
        option output           ACCEPT
        option forward          REJECT

config zone
        option name             wan
        option network          'wan wan2 wan3 wan4'
        option input            REJECT
        option output           ACCEPT
        option forward          REJECT
        option masq             1
        option mtu_fix          1

config forwarding
        option src              lan
        option dest             wan

# We need to accept udp packets on port 68,
# see https://dev.openwrt.org/ticket/4108
config rule
        option name             Allow-DHCP-Renew
        option src              wan
        option proto            udp
        option dest_port        68
        option target           ACCEPT
        option family           ipv4

# Allow IPv4 ping
config rule
        option name             Allow-Ping
        option src              wan
        option proto            icmp
        option icmp_type        echo-request
        option family           ipv4
        option target           ACCEPT

# Allow DHCPv6 replies
# see https://dev.openwrt.org/ticket/10381
config rule
        option name             Allow-DHCPv6
        option src              wan
        option proto            udp
        option src_ip           fe80::/10
        option src_port         547
        option dest_ip          fe80::/10
        option dest_port        546
        option family           ipv6
        option target           ACCEPT

# Allow essential incoming IPv6 ICMP traffic
config rule
        option name             Allow-ICMPv6-Input
        option src              wan
        option proto    icmp
        list icmp_type          echo-request
        list icmp_type          echo-reply
        list icmp_type          destination-unreachable
        list icmp_type          packet-too-big
        list icmp_type          time-exceeded
        list icmp_type          bad-header
        list icmp_type          unknown-header-type
        list icmp_type          router-solicitation
        list icmp_type          neighbour-solicitation
        list icmp_type          router-advertisement
        list icmp_type          neighbour-advertisement
        option limit            1000/sec
        option family           ipv6
        option target           ACCEPT

# Allow essential forwarded IPv6 ICMP traffic
config rule
        option name             Allow-ICMPv6-Forward
        option src              wan
        option dest             *
        option proto            icmp
        list icmp_type          echo-request
        list icmp_type          echo-reply
        list icmp_type          destination-unreachable
        list icmp_type          packet-too-big
        list icmp_type          time-exceeded
        list icmp_type          bad-header
        list icmp_type          unknown-header-type
        option limit            1000/sec
        option family           ipv6
        option target           ACCEPT

# include a file with users custom iptables rules
config include
        option path /etc/firewall.user


### EXAMPLE CONFIG SECTIONS
# do not allow a specific ip to access wan
#config rule
#       option src              lan
#       option src_ip   192.168.45.2
#       option dest             wan
#       option proto    tcp
#       option target   REJECT

# block a specific mac on wan
#config rule
#       option dest             wan
#       option src_mac  00:11:22:33:44:66
#       option target   REJECT

# block incoming ICMP traffic on a zone
#config rule
#       option src              lan
#       option proto    ICMP
#       option target   DROP

# port redirect port coming in on wan to lan
#config redirect
#       option src                      wan
#       option src_dport        80
#       option dest                     lan
#       option dest_ip          192.168.16.235
#       option dest_port        80
#       option proto            tcp

# port redirect of remapped ssh port (22001) on wan
#config redirect
#       option src              wan
#       option src_dport        22001
#       option dest             lan
#       option dest_port        22
#       option proto            tcp

# allow IPsec/ESP and ISAKMP passthrough
#config rule
#       option src              wan
#       option dest             lan
#       option protocol         esp
#       option target           ACCEPT

#config rule
#       option src              wan
#       option dest             lan
#       option src_port         500
#       option dest_port        500
#       option proto            udp
#       option target           ACCEPT

### FULL CONFIG SECTIONS
#config rule
#       option src              lan
#       option src_ip   192.168.45.2
#       option src_mac  00:11:22:33:44:55
#       option src_port 80
#       option dest             wan
#       option dest_ip  194.25.2.129
#       option dest_port        120
#       option proto    tcp
#       option target   REJECT

#config redirect
#       option src              lan
#       option src_ip   192.168.45.2
#       option src_mac  00:11:22:33:44:55
#       option src_port         1024
#       option src_dport        80
#       option dest_ip  194.25.2.129
#       option dest_port        120
#       option proto    tcp
root@OpenWrt:~#

/etc/config/mwan3


root@OpenWrt:~# cat /etc/config/mwan3

config member 'wan_m2_w3'
        option weight '3'
        option interface 'wan'
        option metric '2'

config member 'wan2_m2_w3'
        option interface 'wan2'
        option metric '2'
        option weight '3'

config member 'wan3_m2_w3'
        option interface 'wan3'
        option metric '2'
        option weight '3'

config member 'wan4_m2_w3'
        option interface 'wan4'
        option metric '2'
        option weight '3'

config policy 'wan1_only'
        list use_member 'wan1_m1_w3'

config policy 'wan2_only'
        list use_member 'wan2_m1_w2'

config policy 'wan_wan2_wan3_wan4_loadbalanced'
        list use_member 'wan_m2_w3'
        list use_member 'wan2_m2_w3'
        list use_member 'wan3_m2_w3'
        list use_member 'wan4_m2_w3'

config policy 'wan1_pri_wan2_sec'
        list use_member 'wan1_m1_w3'
        list use_member 'wan2_m2_w2'

config policy 'wan2_pri_wan1_sec'
        list use_member 'wan1_m2_w3'
        list use_member 'wan2_m1_w2'

config rule
        option dest_ip '0.0.0.0/0'
        option proto 'all'
        option use_policy 'wan_wan2_wan3_wan4_loadbalanced'
        option equalize '1'

config interface 'wan'
        option enabled '1'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option reliability '2'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option up '8'

config interface 'wan2'
        option enabled '1'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option reliability '2'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'

config interface 'wan3'
        option enabled '1'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option reliability '2'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'

config interface 'wan4'
        option enabled '1'
        list track_ip '61.135.169.125'
        list track_ip '202.108.33.70'
        list track_ip '61.135.181.167'
        list track_ip '103.8.222.23'
        option reliability '2'
        option count '1'
        option timeout '2'
        option interval '5'
        option down '3'
        option up '8'

root@OpenWrt:~#

route -n


root@OpenWrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         127.0.0.1       0.0.0.0         UG    0      0        0 lo
0.0.0.0         192.168.122.1   0.0.0.0         UG    10     0        0 eth0.4
0.0.0.0         192.168.120.1   0.0.0.0         UG    20     0        0 eth0.3
0.0.0.0         192.168.111.1   0.0.0.0         UG    30     0        0 eth0.2
0.0.0.0         192.168.119.1   0.0.0.0         UG    40     0        0 eth0.1
192.168.111.0   0.0.0.0         255.255.255.0   U     30     0        0 eth0.2
192.168.115.0   0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.119.0   0.0.0.0         255.255.255.0   U     40     0        0 eth0.1
192.168.120.0   0.0.0.0         255.255.255.0   U     20     0        0 eth0.3
192.168.122.0   0.0.0.0         255.255.255.0   U     10     0        0 eth0.4
root@OpenWrt:~#


Thank you in advance.