MWAN3 doesn't seem to increase bandwidth - much

Just got a second ISP (moving from one to another) and thought I would do some dual-wan testing to see if I should consider keeping both.

I configured both WAN interfaces and individually tested each. They have similar bandwidth, both in UL and DL speeds. Then I used MWAN3 to load balance the interfaces and did some further tests using a multi-connection download client. What I found surprised me.

I can see each WAN port is handling about half the load. However, the additional bandwidth gained is only about 10% compared to shutting down either WAN interface and doing no load balancing.

Has anyone else seen this? Is there any general direction I should take to try to troubleshoot and hopefully gain the lost bandwidth?

My Configuration:
ISP1: T-Mobile Home Internet
ISP2: Starlink
Router: Linksys WRT1900AC
Architecture: ARMv7 Processor rev 2 (v7l)
Firmware Version: OpenWrt 19.07.7 r11306-c4a6851c72 / LuCI openwrt-19.07 branch git-21.327.65580-ab3595b
Kernel Version: 4.14.221

Thank you!

With a double WAN this does not change the bandwidth which remains that of a single ISP for a computer
at best that allow with two machines to pass the data on one or the other ISP and therefore to keep the maximum bandwidth of each ISP for a machine
I have Netgear Srx 5308 with 4 * WAN ports and it never gave me the ability to quadruple internet speed
For the bandwidth to be doubled or even more it is necessary to implement a line aggregation but it is not free

Since the arrival of fiber to me, with 2 * 1 Gbits down and 600 Mbits up, I have only kept one ISP

It would not be interesting if it broke down which has not been the case for more than a year

1 Like

Hey, thanks for the reply.

I think you are misunderstanding my question. Or maybe I am not understanding my situation. My understanding is that the connection, or session, is the fundamental limiter to one ISP and not the fact I am using a single computer. I could be misunderstanding this...

My download manager intentionally initiates many connections to the server and my expectation is that each of those connections would be load balanced between the two ISPs. I do believe this is true as I do get about a 10% improvement in BW using this scheme. What I don't understand is why this isn't closer to 80-90% or even 100% of the combined ISP BW.

I am not sure where I am going wrong. Thanks!

Loadbalancing is complex - and basically never works properly (without the help of an external data centre recombining the two WANs into a single logical WAN facing the outside). That also applies to expensive commercial loadbalancing solutions.

Disclaimer: I have not looked into the details of mwan3, but:
You really need to test with at least two computers in your LAN concurrently, because defining what a session is (which needs to be kept on one WAN) is difficult, as basically everything these days pulls resources from multiple backing stores, the router can't keep track of that - but it can keep track of multiple internal systems and keep them apart.

1 Like

Roughly speaking, each client can only use one WAN at a time.
The increase of bandwidth comes from multiple clients assigned to different WANs.
This way allows you to increase multi-client bandwidth, but not bandwidth per client.

1 Like

i also need this info man

I have managed to double the speed of downloads with two computers each using a different ISP, two BitTorrent applications that download the same files. It should be noted that this works because Bitorrent is able to download parts of different files over the local network

Ok, so if I am reading what you guys are saying correctly and to put it in terms of the router/switch. Load balancing works best (maybe only), when each of two LAN ports can be hooked up with a (temporarily) dedicated WAN port.

Yes, I can see how this is not terribly valuable for what I am looking for..

Since a common web page might reach out to Google for fonts, one or more CDNs for supporting libraries and then the main content from yet another source, I figured each of those requests would get load balanced between the available WAN connections even though the requests were all coming from a single LAN interface. It seemed like that was, at lease one of the reasons download managers spawned separate connections. Each connection downloading a portion of the content.

I guess what I find distractingly weird is when connected to a single ISP (no load balancing), my download client saturates the connection. But when I turn on load balancing across two WAN interfaces (and two ISPs), neither connection saturates, it appears both contribute to the overall download, but only at about half saturation plus about 5-10%.

I appreciate the discussion. Please let me know if you have any more comments or insight. I will probably play with this for the rest of the month and let you know what I find out.

Thanks!

Check CPU load from luci or commandline and see if it's a CPU issue.

In my experience, classic load balancing is very meh, both on OpenWrt and pfSense/OPNSense. Yes in theory multiple connections should go on different WANs but in practice it's very situational and mysterious.

If you want "true" bandwith aggregation, a project worth looking into is mptcprouter https://www.openmptcprouter.com/ (it's based off OpenWrt but only stronger devices are supported)

It works by truly splitting traffic over both connections (breaking TCP standards, so it is not limited as others said above) and sending it to a server where an application is re-assembling the traffic into normal.
To use it you need to pay for the cheapest possible VPS in the cloud like Linode, OVH or others where you install the application that turns it in the exit point.

I've been using that for a while now and it works well enough, I actually join the bandwith of two LTE modems on two carriers.

1 Like

I haven't looked at the code, but I would GUESS that load balancing works by hashing the source and destination IP addresses. It might also hash the TCP/UDP port number but I don't think so. So if this is true opening multiple connections to the same server will have no benefit as they'll all go over the same WAN. However if you open say 8 connections each to different servers you should see good load balance. So, this really works best when you have a few tens of devices all trying to surf the web to different sites at the same time. Something like a gym or a coffee shop or other small business or an apartment building or dorm.

1 Like

I did a setup recently with mwan3 using two 4G internet modems. Can confirm that I'm utilizing both interfaces (by watching the bandwidth using iftop with some filters) when doing multithreaded downloads and so the combined download or upload is greater than just one of them. Used speedtest and fast.com for measuring.

One thing to note is by default mwan3 comes with a rule for https stickiness, fast.com had slower download and speedtest had slower upload when this was enabled cause it forces the reuse of the interface since its the same destination IP (even if multi threaded connections). I sort of removed that rule for my machine only and that also gave a boost, though I need to mention that YMMV as one should be well aware of the effects for disabling this https stickiness rule.

Started looking into the open mptcp router. Since I am looking for bandwidth, I am wondering if setting up a VPS can be worth the costs. Is there such a thing as a VPS that doesn't charge for consumed data? Since this will be bi-directional, there seems to be a lot of costs associated with monthly BW limits after the 1G or 10G.

Any suggestions on VPS vendors to take a look at?

Thanks!

EDIT: I found www.turnkeyinternet.net that claims UnMetered BW. Any thoughts?

Well, I wanted to report back. I found one problem in the setup that is slowing things down. My Starlink connection is only WIFI, I have the Ethernet adapter on order, but is is "pending". The WIFI isn't the direct problem, rather it is the wireless client I am using. For some reason, neither of the two clients I have tried will stay on 5Ghz. They both fall back to 2.4Ghz . Both are running OpenWRT and I cannot find a way to force either to only use 5Ghz.

Client attempt #1 GL-iNet GL-MT1300
Client attempt #2 RPi4

When I connect any other client to the Starlink router, I get the 5Ghz and full bandwidth, so I know it has promise. I know I should just wait until I have the Ethernet adapter, but I am learning so much and having a great time doing it.

If anyone has any ideas on how to force on OpenbWRT WWAN client to only use 5Ghz, I would love to hear about it. :slight_smile:

Easiest way is to add a 5ghz only SSID on the star link router if it supports that

I have tried. Unfortunately, the Starlink router is locked down tight. There is an option in the app to split up the 2.4 and 5Ghz but is doesn't seem to work. I click the button and hit save. No separate SSIDs show up in any of my clients and when I come back to that page it is deselected again. Not sure what is going on there, but it seems to be a dead end.

Everything I have read, tells me the client is responsible for selecting the freq, but I cannot seem to find a way to prefer 5Ghz.

You might be able to lock it to the MAC address of 5GHz but I don't know the command. Maybe someone else does. Meanwhile wired is always better.

I agree wired is far better. :slight_smile: Looking forward to it. C'mon Starlink shipping!!! :wink:

Ok! I figured out my 2.4 and 5Ghz problem with the GL-iNet GL-MT1300

Somewhat hidden (not really, but I had a blind spot), is the "Saved Networks" link on the main page (not in the menu). In that list was a saved 2.4Ghz connection to my Starlink router - I must have done it without paying attention early on.

Anyway, now I get 5Ghz connection to my client and now the load balancing works as expected. Yay!

When my wired adapter comes in, I will update this info. But for now, it balances and fails over as expected.

Hopefully, all this info can help others.

Thank you to all who contributed. This was a fun learning experience!

If your problem is solved, please consider marking this topic as [Solved]. See How to mark a topic as [Solved] for a short how-to.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.