Hi all,

Here is the first release of mwan3. I will create a new topic soon, but i'm eager to share and hear what you guys think of it.

The package:
http://213.136.13.52/mwan3_1.0-1_ar71xx.ipk

And the source:
http://213.136.13.52/mwan3_1.0-1.tar.gz

The big difference between mwan3 and mwan2 is that you now can define different primary and secondary wan's for different traffic. Please take a look at the config file for more info.

Hi Adze!

You are using Netgear WNDR3700v2, but this is "old" model, and new v3 don't supports OpenWrt. The successor WNDR3800 seems a good choice, but i don't know it can be supports to VLANs works on both LAN and WAN, like WNDR3700. Is there any recommended model for two or more cable wan configuration?

Hi... First of all, thanks Adze for his great work.... Multiwan never worked for us properly, and mwan2 is almost there

I have 2 PPPoE wan links. I set mwan2 to load balance them (metric/weight=1). We use a rule for a custom checkip service, with equalize=1, and we see the reported IP changes every time we reload.

The problem we have is that sometimes, after reboot, wan1 is activated (ping -I pppoe-wan1 8.8.8.8 works), we can see mwan2 adding rules for it in logread, BUT the ip table 1 is empty (and table 3 only refers to wan2). If I restart wan1 (ifdown wan1 ; ifup wan1) everything fixes.

Before:

root@corpe-gt:~# ip route show table 1
root@corpe-gt:~# ip route show table 2
default via 192.168.153.1 dev pppoe-wan2  metric 1 
root@corpe-gt:~# ip route show table 3
default via 192.168.153.1 dev pppoe-wan2  metric 1

After restarting wan1:

root@corpe-gt:~# ip route show table 1
default via 80.58.67.104 dev pppoe-wan1  metric 1 
root@corpe-gt:~# ip route show table 2
default via 192.168.153.1 dev pppoe-wan2  metric 1 
root@corpe-gt:~# ip route show table 3
default  metric 1 
    nexthop via 80.58.67.104  dev pppoe-wan1 weight 1
    nexthop via 192.168.153.1  dev pppoe-wan2 weight 1

We are using a custom built openwrt, r32576, no modifications. Uting qos-scripts with both interfaces defined there. Legacy multiwan not installed.

Any clue on what may be happening?

/etc/config/network  (relevant lines)

config interface 'wan1'
    option proto 'pppoe'
    option ifname 'eth0.1'
    option username 'adslppp@telefonicanetpa'
    option password 'adslppp'
    option peerdns '0'
    option dns '8.8.8.8'

config interface 'wan2'
    option proto 'pppoe'
    option ifname 'eth0.2'
    option username 'adslppp@telefonicanetpa'
    option password 'adslppp'
    option peerdns '0'
    option dns '8.8.8.8'

/etc/config/mwan2

config 'interface' 'wan1'
    option 'enabled' '1'
    option 'metric' '1'
    option 'weight' '1'
    option 'track_ip' '8.8.8.8'
    option 'count' '1'
    option 'timeout' '2'
    option 'interval' '5'
    option 'down' '3'
    option 'up' '8'

config 'interface' 'wan2'
    option 'enabled' '1'
    option 'metric' '1'
    option 'weight' '1'
    option 'track_ip' '8.8.4.4'
    option 'count' '1'
    option 'timeout' '2'
    option 'interval' '5'
    option 'down' '3'
    option 'up' '8'

# We have openvpn routing that subnet. The routes are pushed by openvpn (seems to be working fine)
config 'rule'
    option 'dest_ip' '10.100.0.0/16'
    list 'use_interface' 'default'

# checkip service, for testing equalice=1
config 'rule'
    option 'dest_ip' '188.165.xxx.xxx/32'
    option 'proto' 'tcp'
    option 'equalize' '1'
    list 'use_interface' 'wan1'
    list 'use_interface' 'wan2'

# default rule
config 'rule'
    option 'dest_ip' '0.0.0.0/0'
    #option 'equalize' '1'
    list 'use_interface' 'wan1'
    list 'use_interface' 'wan2'

Thanks in advance,
Julian.

I'm not sure, but maybe i know what is happening...

As your network config does not include metric settings for the wan interfaces, there might be a race condition. As wan1 comes up it first handles the basic networking before mwan2 comes into play, but even before mwan2 can finish completely, wan2 comes up. This might override your default routing table entry of wan1. Which in turn causes mwan2 not to finish correctly for wan1, as it needs that information from the default routing table.

Try to set some metric values on your wan interfaces in your network config.

(Last edited by Adze on 5 Sep 2012, 22:59)

I've done some reboots and so far it hasn't failed. Thanks man.

BTW, I didn't initially set the metric on those interfaces, because acording to the documentation, the "metric" option was not allowed with protocol "pppoe", only "static". http://wiki.openwrt.org/doc/uci/network

Thanks again,
Julian.

mwan3 works

default config + suggested mwan2 static routes

wan1 DHCP
wan2 PPPoE

#/etc/config/network
#...
config interface 'wan1'
        option ifname 'eth0.1'
        option proto 'dhcp'
        option defaultroute '0'

config interface 'wan2'
        option proto 'pppoe'
        option ifname 'eth0.2'
        option username '<ausername>'
        option password '<apassword>'
        option defaultroute '0'

config route 
        option interface 'wan1'
        option target '0.0.0.0'
        option netmask '0.0.0.0'
        option metric '10'
        option gateway '<ISP1 Gateway IP>'
 
config route 
        option interface 'wan2'
        option target '0.0.0.0'
        option netmask '0.0.0.0'
        option metric '20'
        option gateway '<ISP2 Gateway IP>'
#...

But with the current situation we have static routes with metric 10 and 20 configured.

In this case the moment when ISP changes their gateway IP - we are screwed.

Any suggestion to avoid this?

What about the case when we use two lines where the gateway is the same and our IP on both lines is the same? This is the case when the ISP gives us a CPE and private range - same private range on all CPEs.

Thanks.

(Last edited by NetworkPro on 9 Sep 2012, 09:20)

In older versions of OpenWrt, where netifd was not used, you had to do this trick with static routes. This was because you could not set the metric option on ppp interfaces. With recent versions of OpenWrt with netifd, this trick is no longer necessary. Please try the following:

config interface 'wan1'
        option ifname 'eth0.1'
        option proto 'dhcp'
        option metric '10'

config interface 'wan2'
        option proto 'pppoe'
        option ifname 'eth0.2'
        option username '<ausername>'
        option password '<apassword>'
        option metric '20'

I have no simple solution to that... Using two OpenWrt routers, one for each CPE could be a sollution.

Thanks for testing.

You SOLVED it High five !

option metric worked this time - I just had to remove option defaltroute so that its not there at all

Removing defaultroute and leaving only metric '10' and metric '20' for each wan respectively solved the problem.

Can you add e-mail notification on link failure and generally more business features?

In what way DNS failover works? Can we do DNS preference if one ISP DNS is better?

Thank you.

(Last edited by NetworkPro on 9 Sep 2012, 12:36)

Hi again,

I has happened twice, that after some time, one of the wans disappears from table 3 (the load balancer). I can manually ping -I pppoe-wan1 8.8.8.8   (and same for pppoe-wan2), but mwan2 seems to think it's down.

If I restart that wan connection (ifdown wan2 ; ifup wan2), everything is back to normal.

Is there any way to enable logging of what mwan2 does, or even hack it to send notifications on link failures, as previos message suggests?

Thanks
Julian.

Here's another:

5) Convert WLAN to WWAN as 2nd WAN, or add WWAN to WLAN if iface supports it...

Feature Request.
Long-running connections continue to use old route.

When main route becomes available, new connections are beeing directed to the main route,
but long running connection, e.g. OpenVPN UDP session still continues to use backup route.
Can mwan2 enforce those sessions to use main route or forcevely kill this sessions (possibly after some given period of time)?

No, as it would make absolutely no sense.

When a session is routed through wan2 (in the event that wan1 is down) and wan1 comes back online; and you were to send subsequent packages out wan1, the receiving host will see packets from a complete other source and won't accept them for the old session. Only when you make a new session a new routing decision will be made.

You should let OpenVPN do the force-setup of a new session.

(Last edited by Adze on 19 Nov 2012, 10:34)

OpenVPN Client is started on machine from the "lan" zone.
May be the problem is caused by file /etc/hotplug.d/iface/10-routes ?
After upgrade to "OpenWrt Attitude Adjustment 12.09-beta"{1,2} without this file mwan2 stop to function.

After main channel becomes online nf_conntrack still shows that backup channel is used

root@OpenWrt:~# ip ru
0:      from all lookup local
192:    from 192.168.1.148 fwmark 0x0/0x8000 lookup 1
193:    from 10.224.202.28 fwmark 0x0/0x8000 lookup 2
256:    from all fwmark 0x100/0xff00 lookup 1
257:    from all fwmark 0x200/0xff00 lookup 2
258:    from all fwmark 0x300/0xff00 lookup 3
32766:  from all lookup main
32767:  from all lookup default

root@OpenWrt:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.1.1     0.0.0.0         UG    10     0        0 eth1
default         *               0.0.0.0         U     20     0        0 3g-wan2
default         *               0.0.0.0         U     20     0        0 3g-wan2
10.64.64.64     *               255.255.255.255 UH    0      0        0 3g-wan2
192.168.1.0     *               255.255.255.0   U     10     0        0 eth1
192.168.4.0     *               255.255.255.0   U     0      0        0 br-lan


root@OpenWrt:~# cat /proc/net/nf_conntrack|grep udp| grep 1194
ipv4     2 udp      17 178 src=192.168.4.106 dst=95.163.85.124 sport=2759 dport=1194 packets=1107 bytes=310337 src=95.163.85.124 dst=[color=#FF0000]10.224.202.28[/color] sport=1194 dport=2759 packets=887 bytes=535904 [ASSURED] mark=512 use=2

dst=10.224.202.28 -- It does show that backup route is being used

If I mannualy restart OpenVPN client, then rigth route becomes used as expected.

(Last edited by zensey on 19 Nov 2012, 12:20)

Could you paste me your network config, as i suspect that there is something wrong there...

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option ifname 'eth0'
        option type 'bridge'
        option proto 'static'
        option netmask '255.255.255.0'
        option ipaddr '192.168.4.1'

config switch
        option name 'eth0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'eth0'
        option vlan '1'
        option ports '0 1 2 3 4'

config interface 'wan'
        option ifname 'eth1'
        option _orig_ifname 'eth1'
        option _orig_bridge 'false'
        option proto 'dhcp'
        option 'metric' '10'

config interface 'wan2'
        option ifname 'ppp0'
        option proto '3g'
        option 'device' '/dev/ttyUSB0'
        option service 'umts'
        option apn 'internet'
        option 'peerdns' '0'
        option 'metric' '20'

config 'route'
        option 'interface' 'wan2'
        option 'target' '0.0.0.0'
        option 'netmask' '0.0.0.0'
        option 'metric' '20'

It seams that conntrack UDP entries should be removed manually when route changes:
http://www.unixpearls.com/solved-linux- … sk-server/
http://www.iptables.info/en/connection- … ONNECTIONS

Hi Zensey,

You can leave out/remove the last route config section.

Adze

Yes, it works.
But conntrack still shows use of backup route.

Hi Zensey,

I think i don't really understand you...

I assume that you have the following situation. A node on your LAN has an OpenVPN connection through the router. In this current situation your primary link is down and traffic is routed over your secondary connection. So far OK?

Then the primary link comes back online and all new connections are routed through the primary link. All active/established session continue to route through the secondary connection. So far OK?

Your VPN connection stays connected through secondary wan, until you restart your OpenVPN client on your LAN. When you have restarted the new connection is routed through primary wan. So far OK?

If this is the case, it works exactly how it supposed to work. If there is something i'm missing, please inform me.

Absolutely right.
But I wanted to know
where should I issue "conntrack -D ..." in the code of mwan2 to break that vpn connect?

(Last edited by zensey on 20 Nov 2012, 11:11)

You can put it "/etc/hotplug.d/iface". All scripts located in that location will run when an interface goes up or down. You don't have to / want to edit mwan2 script directly.

Thank you.
I have made like you said and now it works as expected.

(Last edited by zensey on 22 Nov 2012, 12:26)