1 (edited by Adze 2012-09-10 10:29:26)

Topic: New package: mwan2; testers wanted.

Hi all,

I have made a multiwan script called mwan2, which should resolve some issues i had with multiwan. I'd really like it if you could give it a go and let me know what you think of it.

edit: mwan2 has had an update, so it's now compatible with netifd.
edit2: mwan2 now support session-based load sharing. Please read the mwan2 config file for more information. Also the config has changed a little. You now NEED to define at least one rule to make mwan2 work.
edit3: mwan3 is now available: https://forum.openwrt.org/viewtopic.php?id=39052
edit4: mwan svn repository is up: svn://213.136.13.52/var/svn/mwan, it is now much easier to add mwan to the openwrt trunk. Just add the line "src-svn mwan svn://213.136.13.52/var/svn/mwan" to the file "feeds.conf.default".

The package:
http://213.136.13.52/mwan2_1.4-5.ipk

And the source:
http://213.136.13.52/mwan2_1.4-5.tar.gz
svn://213.136.13.52/var/svn/mwan

What is mwan2:
Mwan2 is a couple of lines of code that simplifies the usage of more (up to 7) WAN interfaces in OpenWRT. It is hotplug driven and it allows for any combination of primary, secondary or more failover interfaces, load balanced or not, for any combination of traffic. Mwan2 can monitor the state of interfaces by sending pings to a configured tracking host and failover if necessary.


Why should i use mwan2 instead of multi-wan ?:
- It is faster; mwan2 uses less iptables-rules.
- It is more configurable; mwan2 can handle multiple levels of backup interfaces, load-balanced or not.
- It is compatible; mwan2 uses flowmask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) and you can configure destinations to fall-back to the default routing table.


Requirements:
Mwan2 is successfully tested on OpenWRT trunk r28731 and up. You need the following packages (which should be installed automatically if missing): ip, iptables, iptables-mod-conntrack, iptables-mod-conntrack-extra, iptables-mod-ipopt.


How does it work:
Mwan2 is triggered by hotplug-events. When an interface comes up it creates new routing tables and new iptables rules. A new routing table is created for each possible combination of wan interfaces. So if you have 4 WAN interfaces, 2^4-1 routing tables are created.

It then sets up iptables rules and uses iptables MARK to mark certain traffic. Traffic that is allowed over WAN interface 1 gets the first mark bit set. Traffic that is allowed over WAN interface 2 gets the second mark bit set. And so forth. Eventually you get a flow with certain 'wan' marks set. The kernel then uses that mark to determine which routing table to use.

When an interface goes down, mwan2 deletes all routes to that interface in all created routing tables.


How to install and configure:
I'll assume here you have a clean install of OpenWRT. [s]Due to a bug in OpenWRT (https://dev.openwrt.org/ticket/10423) you have to edit the file "/usr/share/udhcpc/default.script" and replace line 72 from[/s]

eval $(route -n | awk '

[s]to[/s]

eval $(route -n | awk '$5 == ('${user_metric:-0}')' | awk '

You then configure your network according to your setup. Place a different metric on each WAN interface. This metric has only effect on the default routing table, not on the mwan2 routing tables. If it is configured correctly you should have a default gateway with a different metric set for each WAN interface. Something will look like this:

root@openwrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         31.151.171.1    0.0.0.0         UG    10     0        0 eth0.1
0.0.0.0         195.240.99.254  0.0.0.0         UG    20     0        0 eth0.2
31.151.171.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.1
192.168.33.0    0.0.0.0         255.255.255.0   U     0      0        0 br-lan
195.240.96.0    0.0.0.0         255.255.252.0   U     0      0        0 eth0.2

Check if above configuration works by trying to ping www.google.com form each interface:

root@openwrt:~# ping -c 1 -I eth0.1 www.google.com
PING www.google.com (209.85.148.103): 56 data bytes
64 bytes from 209.85.148.103: seq=0 ttl=54 time=19.637 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 19.637/19.637/19.637 ms

root@openwrt:~# ping -c 1 -I eth0.2 www.google.com
PING www.google.com (209.85.148.99): 56 data bytes
64 bytes from 209.85.148.99: seq=0 ttl=56 time=25.552 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 25.552/25.552/25.552 ms

If above ping tests are successful, you can then continue installing mwan2. If you cannot ping www.google.com from all WAN interfaces, there is a problem with your config and installing mwan2 won't fix that!

After installing mwan2, configure it by editting /etc/config/mwan2. For each WAN interface create an interface with a name that matches the one used in /etc/config/network. Configure weight and metric. Interface with a low metric have precedence over higher metric interfaces. Interfaces with the same metric will load-balance. Load balancing interfaces (with same metric) will distribute load based on those weights values. All other options are for tracking and are optional.

After that, configure mwan2 rules. With the mwan2 rules you can direct traffic to certain WAN interfaces, or use the default routing table by choosing default. The order of the rules is important, as they are loaded in iptables in that order. If a rule is matched, but all listed WAN interfaces are down, traffic is handled by the default routing table.


Troubleshooting (if necessary):

root@openwrt:~# ip rule list
0:    from all lookup local 
256:    from all fwmark 0x100/0xff00 lookup 1 
257:    from all fwmark 0x200/0xff00 lookup 2 
258:    from all fwmark 0x300/0xff00 lookup 3 
512:    from 31.151.171.0/24 lookup 1 
513:    from 195.240.96.0/22 lookup 2 
32766:    from all lookup main 
32767:    from all lookup default 

root@openwrt:~# ip route list table 3
default  metric 1 
    nexthop via 195.240.99.254  dev eth0.2 weight 1
    nexthop via 31.151.171.1  dev eth0.1 weight 1

root@openwrt:~# iptables -L mwan2_pre -t mangle -v
Chain mwan2_pre (2 references)
 pkts bytes target     prot opt in     out     source               destination         
1341K  939M CONNMARK   all  --  any    any     anywhere             anywhere            CONNMARK restore mask 0xff00 
 1887  194K MARK       all  --  eth0.2 any     anywhere             anywhere            MARK xset 0x8200/0xff00 
89251   41M MARK       all  --  eth0.1 any     anywhere             anywhere            MARK xset 0x8100/0xff00 
 176K   11M mwan2_rules  all  --  any    any     anywhere             anywhere            ctstate NEW mark match 0x0/0xff00 

root@openwrt:~# iptables -L mwan2_post -t mangle -v
Chain mwan2_post (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 1853  187K MARK       all  --  any    eth0.2  anywhere             anywhere            MARK xset 0x200/0xff00 
36520 4106K MARK       all  --  any    eth0.1  anywhere             anywhere            MARK xset 0x100/0xff00 
 670K  870M MARK       all  --  any    any     anywhere             anywhere            mark match 0x8000/0x8000 MARK and 0xffff7fff 
1098K  905M CONNMARK   all  --  any    any     anywhere             anywhere            CONNMARK save mask 0xff00 

root@openwrt:~# iptables -L mwan2_rules -t mangle -v
Chain mwan2_rules (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  any    any     anywhere             10.0.0.0/8          mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
13634  827K MARK       all  --  any    any     anywhere             127.0.0.0/8         mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
    0     0 MARK       all  --  any    any     anywhere             172.16.0.0/12       mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
  105 15694 MARK       all  --  any    any     anywhere             192.168.0.0/16      mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
  423 28080 MARK       all  --  any    any     anywhere             base-address.mcast.net/3 mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
 2461  204K MARK       all  --  any    any     anywhere             anywhere            mark match 0x0/0xff00 MARK xset 0x300/0xff00

Re: New package: mwan2; testers wanted.

Really no one is interested? Only 3 downloads and 0 replies. Guess multiple WANs isn't that common and used, as i thought...

3 (edited by xopal 2011-12-28 19:13:20)

Re: New package: mwan2; testers wanted.

Thank you Adze, I'm running your script on my router right now , I'll give you report in a few days.

Re: New package: mwan2; testers wanted.

I find this a great piece of information. Comprehensive and detailed, thanks.

Unfortunately, in spite I intent to build a multiwan environment in future, now I am not able to give this a real test. In my oppinion, this should be inspected and tested and put in the wiki...

I may try it in a virtualbox setup with several openwrt machines that I am currently running for testing purposes.

Keep it going Adze.

Re: New package: mwan2; testers wanted.

Thanks, I'll check it out.

Re: New package: mwan2; testers wanted.

have tried this out and using wan1 = wired, wan2=3g. It works and is a great solution for failover / loadsharing applications where all interfaces are kept up at all times. In my application, I want the 3g interface off until needed. May be useful to have interface config option which takes down unneeded interfaces (such as 3g) when main interfaces up.

For my application, opted for this script:

https://forum.openwrt.org/viewtopic.php … 57#p153957

Re: New package: mwan2; testers wanted.

also noticing this (logread):

Jan 12 19:57:30 OpenWrt user.info firewall: removing wan (eth1) from zone wan
Jan 12 19:57:31 OpenWrt user.notice root: mwan2: Lost 3 ping(s) on interface ppp0 (3g-ppp0)
Jan 12 19:57:35 OpenWrt user.notice root: mwan2: Lost 3 ping(s) on interface wan (eth1)
Jan 12 19:58:09 OpenWrt user.notice root: mwan2: Interface ppp0 (3g-ppp0) is back online
Jan 12 19:58:09 OpenWrt user.notice ifup: Enabling Router Solicitations on ppp0 (3g-ppp0)
Jan 12 19:58:09 OpenWrt user.notice root: mwan2: Adding rules for interface ppp0 (3g-ppp0)
Jan 12 19:58:11 OpenWrt user.notice root: mwan2: Interface wan (eth1) is back online
Jan 12 19:58:11 OpenWrt user.notice ifup: Allowing Router Advertisements on wan (eth1)
Jan 12 19:58:11 OpenWrt user.notice root: mwan2: Adding rules for interface wan (eth1)
Jan 12 19:58:11 OpenWrt user.info firewall: adding wan (eth1) to zone wan
Jan 12 19:59:53 OpenWrt user.notice root: mwan2: Interface ppp0 (3g-ppp0) is offline
Jan 12 19:59:53 OpenWrt user.notice root: mwan2: Deleting rules for interface ppp0 (3g-ppp0)
Jan 12 19:59:57 OpenWrt user.notice root: mwan2: Interface wan (eth1) is offline
Jan 12 19:59:58 OpenWrt user.notice root: mwan2: Deleting rules for interface wan (eth1)
Jan 12 19:59:58 OpenWrt user.info firewall: removing wan (eth1) from zone wan
Jan 12 19:59:59 OpenWrt user.notice root: mwan2: Lost 3 ping(s) on interface ppp0 (3g-ppp0)
Jan 12 20:00:03 OpenWrt user.notice root: mwan2: Lost 3 ping(s) on interface wan (eth1)

Indicating that mwan2 believes both interfaces are unstable. They are not. Looks like some sort of race condition.

Re: New package: mwan2; testers wanted.

rossb wrote:

May be useful to have interface config option which takes down unneeded interfaces (such as 3g) when main interfaces up.

This should be possible i think... You have to remove all tracking options for your 3g connection in "/etc/config/mwan2". And configure dial on demand in "/etc/config/network". Don't have a 3g dongle myself (yet), so can't test it myself right now.

config 'interface' 'ppp0'
        option 'proto' '3g'
        option 'ifname' '3g-ppp0'
        option 'service' 'umts_only'
        option 'apn' '???'
        option 'pincode' '???'
        option 'device' '/dev/ttyUSB0'
        option 'demand' '600'
rossb wrote:

Indicating that mwan2 believes both interfaces are unstable. They are not. Looks like some sort of race condition.

It should be fixed now in the 1.3-2 release. It was a small problem of ip rule order.

Re: New package: mwan2; testers wanted.

... with updated package, still seeing mwan2 lost pings and interface rules being changed. It may be because of 3g which you do not have and therefore, cannot test with. I have other tasks, so have spent as much time as I can on this. Thanks for your work and, I may be able to revisit once stability achieved.

Re: New package: mwan2; testers wanted.

Hi there I have started to get things under test but an issue hads apeared

if i have wan 1 = eth1 and wan 2 = eth2

i get 2 enteries in my routing table


if wan 1 = eth1 and wan 2 = 3g dongle
i get only the route or the 3g dongle. if i look at my routing table as it applies i can see the route or eth0 apears and then gets overwriten by the 3g dongle. also thr routemetric does not stick.



Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.64.64.64     *               255.255.255.255 UH    0      0        0 3g-3g
10.10.1.0       *               255.255.255.0   U     0      0        0 br-lan
192.168.0.0     *               255.255.255.0   U     0      0        0 eth1
default         10.64.64.64     0.0.0.0         UG    0      0        0 3g-3g


config 'interface' 'WAN'
    option 'ifname' 'eth1'
    option '_orig_ifname' 'eth1'
    option '_orig_bridge' 'false'
    option 'proto' 'static'
    option 'ipaddr' '192.168.0.100'
    option 'netmask' '255.255.255.0'
    option 'gateway' '192.168.0.1'
    option 'broadcast' '192.168.0.255'
    option 'dns' '192.168.0.1'
    option 'metric' '10'

config 'interface' '3g'
    option 'proto' '3g'
    option 'device' '/dev/ttyUSB0'
    option 'apn' 'three.co.uk'
    option 'service' 'umts'
    option 'metric' '20'

Thanks in advance Dafydd

11 (edited by Adze 2012-01-15 12:13:31)

Re: New package: mwan2; testers wanted.

babel_fish_42 wrote:

if i look at my routing table as it applies i can see the route or eth0 apears and then gets overwriten by the 3g dongle. also thr routemetric does not stick.

That is a problem indeed. Although the main routing table isn't used for traffic forwarding to WANs, it is used to dynamically create the custom routing tables. Could you please try the setting "option defaultroute 0" in your 3g network config? Could you also paste me the output of "ip route list table 3" please?

Thanx

Edit: i see that your 3g interface in the routing table is named "3g-3g". This will probably be troublesome. Could you also try to add the setting "option iface 3g-3g" in your 3g network config?

Re: New package: mwan2; testers wanted.

Thanks for the swit reply smile

with option set to 0

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.64.64.64     *               255.255.255.255 UH    0      0        0 3g-3g
10.10.1.0       *               255.255.255.0   U     0      0        0 br-lan
192.168.0.0     *               255.255.255.0   U     0      0        0 eth1
default         192.168.0.1     0.0.0.0         UG    10     0        0 eth1


ip route table 3 = Blank

root@Semaphore:/# ip route list table 3
root@Semaphore:/#

Thanks
Dafydd

Re: New package: mwan2; testers wanted.

babel_fish_42 wrote:

...

Thanks
Dafydd

I PM-ed you, because otherwise the forum may become some sort of slow chat.

Re: New package: mwan2; testers wanted.

Hello,

I found this error when recall /reload mwan2track

root@OpenWrt:~# mwan2track
/usr/sbin/mwan2track: line 4: arithmetic syntax error
line 1#!/bin/sh
line 2 echo "$$" > /var/run/mwan2track-$2.pid
line 3
line 4 score=$(($7+$8))
line 5 lost=0

15

Re: New package: mwan2; testers wanted.

Thats because there is no 7th and 8th numeric argument in your commandline.

Re: New package: mwan2; testers wanted.

xopal wrote:

Hello,

I found this error when recall /reload mwan2track

mwan2track shouldn't be started by hand, but by the mwan2 script. I will add some syntax checks and warnings if options are missing or started manually. If you really want to start it by hand:

mwan2track $1 $2 $3 $4 $5 $6 $7 $8

where $1 = interface, $2 = device, $3 = host to ping, $4 = number of echo-requests per test, $5 = timeout per icmp-request, $6 = interval between tests in seconds, $7 = number of failed tests until interface is considered "down", $8 = number of successful tests until interface is considered "up".

Example:

/usr/sbin/mwan2track isp1 eth0.1 www.google.com 1 2 5 3 8

17 (edited by VeNoM 2012-01-25 14:49:47)

Re: New package: mwan2; testers wanted.

Cool. I will test it too.

Re: New package: mwan2; testers wanted.

babel_fish_42 wrote:

Thanks for the swit reply smile

with option set to 0

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.64.64.64     *               255.255.255.255 UH    0      0        0 3g-3g
10.10.1.0       *               255.255.255.0   U     0      0        0 br-lan
192.168.0.0     *               255.255.255.0   U     0      0        0 eth1
default         192.168.0.1     0.0.0.0         UG    10     0        0 eth1


ip route table 3 = Blank

root@Semaphore:/# ip route list table 3
root@Semaphore:/#

Thanks
Dafydd

Hi

This is the way i manage my dual wan wired and 3g dongle, may this help you.

#opkg update
#opkg install multiwan
#wget mwan2 (look at adze link)

#opkg install *.ipk

network

config 'interface' 'wan'
        option 'proto'     'pppoe'
        option 'ifname'    'eth0.2'
        option 'username'  'xxxxxxxxxxxx'
        option 'password'  'xxxxxxxxx'
        option 'timeout'   '10'
     #option 'defaultroute' '0'
     option 'peerdns'    '0'
     option 'dns'        '180.131.144.144 180.131.145.145'

config 'interface' 'ppp0'
        option 'proto' '3g'
        #option 'ifname' '3g-ppp0'
        option 'service' 'evdo'
        option 'username' 'smart'
        option 'password' 'smart'
        option 'device' '/dev/ttyUSB0'
     #option 'defaultroute' '0'
     option 'peerdns'    '0'
     option 'dns'        '180.131.144.144 180.131.145.145'

multiwan

config 'interface' 'ppp0'
    option 'weight' 'disable'
    option 'health_interval' '10'
    option 'icmp_hosts' 'disable'
    # icmp_count is defaulted to 1, and can be increased to reduce
    # false positives.
    # option 'icmp_count' '3'
    option 'timeout' '3'
    option 'health_fail_retries' '3'
    option 'health_recovery_retries' '5'
    option 'failover_to' 'wan'
    option 'dns' 'auto'

config 'interface' 'wan'
    option 'weight' 'disable'
    option 'health_interval' '10'
    option 'icmp_hosts' 'disable'
    option 'timeout' '3'
    option 'health_fail_retries' '3'
    option 'health_recovery_retries' '5'
    option 'failover_to' 'balancer'
    option 'dns' '180.131.144.144 180.131.145.145'

mwan2

config 'interface' 'wan'
    option 'enabled' '1'
    option 'metric' '1'
    option 'weight' '1'
    option 'track_ip' 'www.google.com'
    option 'count' '1'
    option 'timeout' '2'
    option 'interval' '5'
    option 'down' '3'
    option 'up' '8'

config 'interface' 'ppp0'
    option 'enabled' '1'
    option 'metric' '1'
    option 'weight' '6'
    option 'track_ip' 'www.google.com'
    option 'count' '1'
    option 'timeout' '2'
    option 'interval' '5'
    option 'down' '3'
    option 'up' '8'
root@OpenWrt:/tmp# ip route list table 3
default  metric 1
        nexthop via 10.20.31.13  dev 3g-ppp0 weight 6
        nexthop via 180.251.112.1  dev pppoe-wan weight 1

I move my primary connection to 3g and my wired act as a secondary connection.

19 (edited by Adze 2012-02-05 16:54:36)

Re: New package: mwan2; testers wanted.

xopal wrote:

#opkg install multiwan

mwan2 is NOT intended to be used together with multiwan. You can do perfectly without multiwan in a dual wired (load-balanced) and a 3g backup, or any other primary/backup setup. As a matter of fact i'm running it now.

Will update this topic soon with some details and configuration examples.

20 (edited by Adze 2012-02-05 17:22:39)

Re: New package: mwan2; testers wanted.

As i was fiddling with a 3g dongle this weekend, i have managed to setup the following scenario:

- 2 Wired WAN links (Unlimited traffic, 120/10Mbps cable link and 20/1Mbps dsl link)
- 1 Wireless UMTS link (2GB/month, 7,2/2Mbps link)

I wanted both wired links to load-balance traffic and the 3G link only to kick-in when both wired links have failed. I also wanted some specific tcp ports to always traverse the second wired link. This is my config:

/etc/config/network:

config 'interface' 'lan'
    option 'ifname' 'eth1'
    option 'proto' 'static'
    option 'ipaddr' '192.168.33.1'
    option 'netmask' '255.255.255.0'
    option 'ip6addr' '2001:610:64d:1::1/64'

config 'interface' 'isp1'
    option 'ifname' 'eth0.1'
    option 'proto' 'dhcp'
    option 'metric' '10'
    option 'dns' '208.67.222.222 8.8.8.8'

config 'interface' 'isp2'
    option 'ifname' 'eth0.2'
    option 'proto' 'dhcp'
    option 'metric' '20'
    option 'dns' '208.67.220.220 8.8.4.4'
    
config 'interface' 'isp3'
    option 'ifname' 'ppp0'
    option 'device' '/dev/ttyUSB0'
    option 'apn' 'office.vodafone.nl'
    option 'username' 'vodafone'
    option 'password' 'vodafone'
    option 'service' 'umts'
    option 'proto' '3g'
    option 'defaultroute' '0'

config 'route'
    option 'interface' 'isp3'
    option 'target' '0.0.0.0'
    option 'netmask' '0.0.0.0'
    option 'metric' '30'

/etc/config/mwan2:

config 'interface' 'isp1'
    option 'enabled' '1'
    option 'metric' '1'
    option 'weight' '4'
    option 'track_ip' 'www.google.com'
    option 'count' '1'
    option 'timeout' '2'
    option 'interval' '5'
    option 'down' '3'
    option 'up' '8'

config 'interface' 'isp2'
    option 'enabled' '1'
    option 'metric' '1'
    option 'weight' '1'
    option 'track_ip' 'www.google.com'
    option 'count' '1'
    option 'timeout' '2'
    option 'interval' '5'
    option 'down' '3'
    option 'up' '8'

config 'interface' 'isp3'
    option 'enabled' '1'
    option 'metric' '2'
    option 'weight' '1'

config 'rule'
    option 'proto' 'tcp'
    option 'src_ip' '192.168.33.0/24'
    option 'dest_port' '119,563'
    list 'use_interface' 'isp1'

config 'rule'
    option 'proto' 'tcp'
    option 'src_ip' '192.168.33.0/24'
    option 'dest_port' '995'
    list 'use_interface' 'isp2'

/etc/config/firewall

config 'defaults'
    option 'input' 'DROP'
    option 'forward' 'REJECT'
    option 'output' 'ACCEPT'
    option 'syn_flood' '1'
    option 'drop_invalid' '1'

config 'zone'
    option 'name' 'local'
    option 'network' 'lan'
    option 'input' 'ACCEPT'
    option 'forward' 'REJECT'
    option 'output' 'ACCEPT'

config 'zone'
    option 'name' 'internet'
    option 'network' 'isp1 isp2 isp3'
    option 'input' 'DROP'
    option 'forward' 'DROP'
    option 'output' 'ACCEPT'
    option 'masq' '1'

config 'forwarding'
    option 'src' 'local'
    option 'dest' 'internet'

With above config you will get these results:

root@mercurius:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         31.151.166.1    0.0.0.0         UG    10     0        0 eth0.1
0.0.0.0         82.168.15.254   0.0.0.0         UG    20     0        0 eth0.2
0.0.0.0         10.64.64.64     0.0.0.0         UG    30     0        0 3g-isp3
10.64.64.64     0.0.0.0         255.255.255.255 UH    0      0        0 3g-isp3
31.151.166.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.1
82.168.12.0     0.0.0.0         255.255.252.0   U     0      0        0 eth0.2
192.168.33.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1

root@mercurius:~# ip route list table 7
default  metric 1 
    nexthop via 31.151.166.1  dev eth0.1 weight 4
    nexthop via 82.168.15.254  dev eth0.2 weight 1
default via 10.64.64.64 dev 3g-isp3  metric 2
 
root@mercurius:~# ip rule show
0:    from all lookup local 
128:    from 31.151.166.0/24 fwmark 0x0/0x8000 lookup 1 
129:    from 82.168.12.0/22 fwmark 0x0/0x8000 lookup 2 
130:    from 10.64.64.64 fwmark 0x0/0x8000 lookup 4 
256:    from all fwmark 0x100/0xff00 lookup 1 
257:    from all fwmark 0x200/0xff00 lookup 2 
258:    from all fwmark 0x300/0xff00 lookup 3 
259:    from all fwmark 0x400/0xff00 lookup 4 
260:    from all fwmark 0x500/0xff00 lookup 5 
261:    from all fwmark 0x600/0xff00 lookup 6 
262:    from all fwmark 0x700/0xff00 lookup 7 
32766:    from all lookup main 
32767:    from all lookup default

root@mercurius:~# iptables -L -t mangle -v -n
Chain PREROUTING (policy ACCEPT 2991K packets, 2941M bytes)
 pkts bytes target     prot opt in     out     source               destination         
2992K 2941M mwan2_pre  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain INPUT (policy ACCEPT 793K packets, 653M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 2168K packets, 2286M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 485K packets, 43M bytes)
 pkts bytes target     prot opt in     out     source               destination         
 485K   43M mwan2_pre  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain POSTROUTING (policy ACCEPT 2652K packets, 2329M bytes)
 pkts bytes target     prot opt in     out     source               destination         
2653K 2329M mwan2_post  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain mwan2_default (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 398K   33M MARK       all  --  *      *       0.0.0.0/0            127.0.0.0/8         mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
32453 2386K MARK       all  --  *      *       0.0.0.0/0            224.0.0.0/3         mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
    0     0 MARK       all  --  *      *       0.0.0.0/0            10.64.64.64         mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
    0     0 MARK       all  --  *      *       0.0.0.0/0            31.151.166.0/24     mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
    0     0 MARK       all  --  *      *       0.0.0.0/0            82.168.12.0/22      mark match 0x0/0xff00 MARK xset 0x8000/0xff00 
18610 1483K MARK       all  --  *      *       0.0.0.0/0            192.168.33.0/24     mark match 0x0/0xff00 MARK xset 0x8000/0xff00 

Chain mwan2_post (1 references)
 pkts bytes target     prot opt in     out     source               destination         
1072K 1367M MARK       all  --  *      eth0.1  0.0.0.0/0            0.0.0.0/0           MARK xset 0x100/0xff00 
    0     0 MARK       all  --  *      3g-isp3  0.0.0.0/0            0.0.0.0/0           MARK xset 0x400/0xff00 
 349K   50M MARK       all  --  *      eth0.2  0.0.0.0/0            0.0.0.0/0           MARK xset 0x200/0xff00 
1231K  912M MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0           mark match 0x8000/0x8000 MARK and 0xffff7fff 
2653K 2329M CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           CONNMARK save mask 0xff00 

Chain mwan2_pre (2 references)
 pkts bytes target     prot opt in     out     source               destination         
3478K 2985M CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           CONNMARK restore mask 0xff00 
 916K  653M MARK       all  --  eth0.1 *       0.0.0.0/0            0.0.0.0/0           MARK xset 0x8100/0xff00 
    0     0 MARK       all  --  3g-isp3 *       0.0.0.0/0            0.0.0.0/0           MARK xset 0x8400/0xff00 
 689K  878M MARK       all  --  eth0.2 *       0.0.0.0/0            0.0.0.0/0           MARK xset 0x8200/0xff00 
 494K   40M mwan2_default  all  --  *      *       0.0.0.0/0            0.0.0.0/0           mark match 0x0/0xff00 
43553 3426K mwan2_rules  all  --  *      *       0.0.0.0/0            0.0.0.0/0           mark match 0x0/0xff00 

Chain mwan2_rules (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       tcp  --  *      *       192.168.33.0/24      0.0.0.0/0           multiport sports 0:65535 multiport dports 119,563 mark match 0x0/0xff00 MARK xset 0x100/0xff00 
    0     0 MARK       tcp  --  *      *       192.168.33.0/24      0.0.0.0/0           multiport sports 0:65535 multiport dports 995 mark match 0x0/0xff00 MARK xset 0x200/0xff00 
42108 3267K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0           mark match 0x0/0xff00 MARK xset 0x700/0xff00

Re: New package: mwan2; testers wanted.

If i disable multiwan my rout -n will be look like this

root@OpenWrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.20.31.11     0.0.0.0         UG    0      0        0 3g-ppp0
10.20.31.11     0.0.0.0         255.255.255.255 UH    0      0        0 3g-ppp0
180.251.112.1   0.0.0.0         255.255.255.255 UH    0      0        0 pppoe-wan
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan

May be i miss something in my network configuration or iptables i'll check it later.

22 (edited by Adze 2012-02-05 17:39:18)

Re: New package: mwan2; testers wanted.

You're missing two static default route entries and uncomment the "defaultroute" '0" options in your /etc/config/network. Look at my config example above.

Re: New package: mwan2; testers wanted.

Thank's Adze it works !!, you just saved my live

root@OpenWrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.20.31.3      0.0.0.0         UG    30     0        0 3g-isp2
0.0.0.0         180.251.112.1   0.0.0.0         UG    30     0        0 pppoe-isp1
10.20.31.3      0.0.0.0         255.255.255.255 UH    0      0        0 3g-isp2
180.251.112.1   0.0.0.0         255.255.255.255 UH    0      0        0 pppoe-isp1
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
root@OpenWrt:~# ip route list table 3
default  metric 1
        nexthop via 10.20.31.3  dev 3g-isp2 weight 6
        nexthop via 180.251.112.1  dev pppoe-isp1 weight 1

Re: New package: mwan2; testers wanted.

Mind you that your current setup is loadbalanced, not active/standby. Loadbalanced on a factor of 6 to 1. If you want active/standby, you should change the metric values.

Nice to see that its working.

25 (edited by hnsr 2012-02-05 19:44:54)

Re: New package: mwan2; testers wanted.

Adze wrote:

As i was fiddling with a 3g dongle this weekend, i have managed to setup the following scenario:

- 2 Wired WAN links (Unlimited traffic, 120/10Mbps cable link and 20/1Mbps dsl link)
- 1 Wireless UMTS link (2GB/month, 7,2/2Mbps link)

Funny, that's identical to the setup I was hoping to be able to get going (ziggo cable, a backup dsl line, and vodafone 3g) smile

I actually have one complication in my current setup though (not using my OpenWRT router yet), the DSL line is connected to a Vigor modem that does PPPoE pass-through, so I can make use of the native IPv6 connectivity my DSL ISP provdes. Right now I have a non-openwrt router doing the Cable (with NAT and everything), and the Vigor PPPoE connection is terminated on a seperate Gentoo machine that has a very hacky and non-automated setup where it gets the IPv6 prefix-delegation from the DSL ISP using dhcpv6 on the ppp IF, and then it uses that prefix and I have to assign an address of that range to my lan IF so radvd starts announcing it. So right now IPv6 traffic goes (thtough the Gentoo machine) via my DSL line, and IPv4 traffic via my Cable NAT device.

Do you think such a setup is possible to get going with OpenWRT and more specifically with mwan2? I'm especially unsure about the IPv6 bits, getting the PD via dhcpv6 and then making sure that prefix is used and announced on my LAN seems complicated if not supported by uci.

Also, I suppose I don't really need loadbalancing nor an extra 3G backup (maybe I'll add that later, I don't have a 3G dongle yet), just being able to failover to DSL should my cable go down would actually be fine for me.