Setting up LACP (Bonding, Link Aggregation) on OpenWrt

Hello,

I want to setup LACP between my router (WDR4300) and my NAS. The NAS is already set, but I don't know how to configure the router? I don't know where to start. I took a look at kmod-bonding, but I don't know how to use it. Is somebody using bonding this way and can explain to me how to configure it?

Kind regards,
pearmint

I don't think OpenWrt has a way to automatically set up bonding, so you need to run some IP commands to bond things manually. Be aware that built in switches on these devices probably don't support bonding... so that's an issue.

You probably need to set up two internal VLANs in the switch, untagged at the CPU and at the port... so that the switch acts like two Ethernet Jack's nothing else.

Then install ip-full package, and the kmod for bonding. Read the man pages for ip to find out how to create a bond0 and enslave the two Ethernet.

search the forum, we have discussed bonding at least once before, and I think there are some examples.

hopefully your switch won't swallow the LACP packets

Anything that needs to go through the CPU port (the kernel) and can't be done in hardware by the switch(-driver) is limited by the CPU capabilities, which is in the range of ~150-200 MBit/s max.

I used to use a modified kernel bonding driver to bond two point to point internet connections to each other in order to double my internet bandwidth, which is a very advanced use case of bonding (and not supported by the vanilla kernel bonding driver which does not allow bonding of devices that don't have a MAC address).

Your bonding case is how the bonding driver is originally intended to be used and is much simpler.

That said, you can glean a lot of info from the scripts on how to set it up and use the bonding driver (you need to create a bond interface and slave the devices to it).

My solution was built for 15.05, you can look at it here, but the commands should be much the same on 19.07

@Pearmint the point by @slh is a good one. It's not clear what you are trying to accomplish, but if you are trying to get somehow "fast filesharing" to the rest of your LAN, what you want is a managed switch with bonding support. If LACP is an important feature (as opposed to "static LAG") then you should look at the gs1900-24e from ZyXEL or the TP-Link t2600G

if a static LAG is sufficient, you could maybe get away with tl-sg1016de, and if you only need 8 ports, tl-sg108e

The bonding driver when used in round robin mode has about a 10% efficiency loss when bonding two links. As you add links, the performance goes down. The practical limit is about 4 to 5 interfaces before performance degrades enough to make it not worthwhile.

The driver is fairly efficient as it's kernel mode, so there's no context switching which would nail performance. The packets are going through the TCP/IP stack anyway, the only difference is they're just being round robined by the kernel onto two output interfaces, so there is only a little bit of additional overhead (performance analyses are available on the net)

I benchmarked the efficiency on my setup at a 10% loss (ie., combined bandwidth = 2x per link bandwidth less 10%) and this included two VPN tunnels, although the crypto was being handled by hardware acceleration, so probably was a relatively small percentage of the 10% loss, although with faster links than I had, this may well be higher. 10%, however, does accord with other analyses I've seen.

If your CPU and device can handle, let's say, a real world throughput of 1Gbps on one interface, you'll absolutely get a minimum of 1Gbps - 10% when bonding two interfaces, assuming that the CPU is the bottleneck. If the CPU has additional headroom to process more packets (independent of the bonding), then you'll get higher throughput.

The point is valid though, that a lot of consumer router devices, even though they have a 1Gbps interface, cannot sustain much more than a few hundred Mbps throughput. So you'll need capable router hardware.

Thank you all for the answers and insights!
The bonding mode adaptive load balancing should work without switch support so I might have to use this one.

Thanks again!

I just did a quick Google for reviews of your device. This Tweaktown review got 840Mbps out of the interface doing a file transfer. It's arguable that your device might indeed be CPU bound and not be able to sustain enough additional throughput that bonding two interfaces would actually make a measurable difference. That said, the solution is free and not that hard to setup, so worth a try

https://openwrt.org/toh/tp-link/tl-wdr4300#hardware

It's a MIPS 74kc, 560MHz single core processor. It's not "maybe" CPU bound it is very CPU bound to on the order of 100-200Mbps as mentioned. The number of devices that can push 1G through their CPU is limited to basically things considered top end devices in the last 5 years, like WRT3200 which doesn't quite do it.

People pushing 800+Mbps through this device are either using some kind of stock firwmware with hardware acceleration, or are just pushing it through the switch from LAN to LAN... anyone can do that.

RPI4 will do it with lots of spare change.

The thing I don't really understand is what is the goal here? Is the router itself really consuming huge quantities of data from the NAS? Or is the NAS downloading huge quantities of data over a multi-gigabit internet? Not through this router it isn't no matter what you do.

If the goal is to provide excess bandwidth to the LAN clients so that 10 devices can each get say 200Mbps of downloading and uploading files off the NAS... then you want a switch that supports LAG, that's it. At the low end we're talking sg108e which supports static LAG and will do all this no problem for $35. There is no need for the router to bond its ethernets.

From the OpenWrt page it's not even clear if there are two CPU ports on this device. In fact it appears with all information that there are NOT... the example configs on that page linked above show WAN as a separate VLAN on a single eth0 so bonding is irrelevant.

Perhaps @Pearmint can provide more explanation as to what is the broader goal.

1 Like

The goal is that I want to increase the bandwidth to my NAS which is directly connected to the router. The NAS should be able to be serve LAN with as much bandwidth as possible. And I thought that adding a spare NIC to the server while enabling bonding would help me reach this goal.

Correct, this device only has a single CPU port.

…but even if it had two (and were fast enough - neither of which is true), the device has only 4+1 ethernet ports to begin with. Assuming that WAN remains WAN, two ports 'lost' for the link aggregation would leave you with only 2 'free' LAN ports. You could either use them as another bonded interface to a switch (in which case, why bother to include the totally underpowered router into this mix) or serve only two wired clients with 1 GBit/s each (very unlikely to max out LACP that way).

What you need in this case is a smart switch. for example the tp-link sg108e or some better switch mentioned above. You can then bond the NAS to the switch and the NAS will be able to push up to 2Gbps into the switch, which will then distribute it to the various LAN clients. The router will not get involved at all, as it only gets involved when packets need to go WAN <--> LAN

For LAN <-> LAN traffic the switch is the only device involved, and that is what needs the link aggregation group (LAG).

Indeed, much better solution.

Thank you for the suggestion! I think that's the way I will go.

hi i want try something like that, is the wrt3200acm good to do that?

I think adding bonding or LACP to the OpenWRT project would be worthwhile. It was added to the Linux kernel as it was deemed 'useful'. With any setup, there are merits and demerits. Some router switches might not be capable of it. But this should be part of any experimentation or growing-pains.

Opinions aside, there are some facts here. Why would anyone want this?

Redundancy. Okay, you have bigger problems if a cable dies, since your router is still the biggest SPOF. And it's not a given that any common router available today switch back-plane can manage e.g. greater than 1GBit in order to make bonding 2x 1GBit interfaces speed-worthwhile. But redundancy is always a win. Keep in mind that for many users of OpenWRT, the Router is the only device at home which is the unifying switch to connect other devices. Investing in, and configuring an additional switch is often the best thing to do, but is also beyond many users.

But laying the groundwork for this will make future projects smoother. There will definitely be a need for this with newer routers. 1GBit LAN infrastructure will benefit from LAGs now that devices with 2.5GBit and 5GBit connection standards are becoming available. Internet link speeds will not sit at 1GBit forever. Commodity ethernet link speeds are now going up to 10GBit. Weird things like two bonded 5GBit WAN ports might be a possible use-case.

The linked repo here: Bonding with cable modem 802.3ad LACP is a promising lead.

Bonding is already supported by OpenWrt, you just install the bonding drivers. But you'll need two ethernet ports. Some all-in-one devices have this, others don't.

It's a "manual set-up" sort of thing, but I think given its a feature that would probably not be usable by the average person (they'd be pretty confused, most people don't even know the difference between say eth0 and eth1 or eth0.1 and eth0.2) it really isn't a problem that there's no GUI to set it up. Also most switches on all-in-ones don't support bonding modes other than fallback type modes.

Correct - your points aside, I meant to say that the link provides the LuCI GUI bits which do lots of the heavy lifting.

Using a WRT3200ACM, where should 2 LACP ports be physically plugged for speed, processing and security coming from a SB8200 with link aggregation setup and two Ethernet cables? Wouldn't it create a bottle neck if you used CPU(eth0) ports 0,1 for WAN(LACP) and CPU(eth0) ports 2,3 for LAN? Is it possible to use CPU(eth0) and CPU(eth1) as the WAN(LACP)? Thank you to anyone who can read through my poor knowledge of the matter and correct me with guidance.