NIC as Switch - true hardware offload?

Hardware Questions and Recommendations
1

Hi folks,

Is it possible to configure a PCIE multi-port NIC to function as a layer 2 switch doing all the packet processing on the NIC controller without sending to the CPU?

Does configuring two interfaces as 'bridged' and using hardware offload do this?

Thanks.

2

No, the data will flow through CPU and kernel.

--
that isn't quite true, depending on the network chipset in question and its drivers, there might be DMA operations going on, but still far way from switching in a hardware fabric.

3

So bridging two 40Gigabit interfaces on a NIC is going to put extreme load on the CPU?

I ask because 40GB QSFP NICs are super cheap (like $40) where as 40GB QSFP switches seem to be thousands of dollars.

4

Yeah - well the reason I ask is because the datasheet for the X710 adapters that I'm looking suggests they have an internal switch exactly for this fuction. However elsewhere in the document it also states that the internal switch is not a 'learning switch' and has to be configured by the host system. It has the command reference for how to program the switch (very low level) and I wonder if this is something the hardware offload feature of the kernel is doing in the background.

image
image1008×867 80.7 KB


The switch is not a learning switch and is managed by the host. The programming interface exposed to
the operating system is that of a managed switch. Each switch element can be configured either via a
software device driver running on a PF or via the EMP.
5

10 GBit/s ethernet cards and faster are quite different from 'normal' 1 GBit/s ones, because they have to do a lot of offloading to the hardware to keep up at all. Answering this would really depend on the details and the exact hardware in question, sorry I've to pass on this question (it wasn't clear in your original post that you were referring to 40 GBit/s cards) - my gut feeling is still not very positive though.

6

No need to apologise :slight_smile: thanks for your thoughts.

I am getting the hardware anyway, so will do some tests and see what the throughput is over bridged interfaces and the impact on CPU.

7

So initial test results...

Bridging two 10GB port with a standard Linux L2 bridge, I get throughput of 9.2Gbps (which is about max with ethernet overhead anyway) and it consumes approx. 2 whole cores of my 8 core Atom processor (Intel C2758).

8

Intel C2758
doesn't' it have power related bug?

9

Early revisions did. Intel fixed it in later revisions. Supermicro replaced the CPU on my board free of charge.