PCIe/mPCIe managed switch

I want to extend APU's functionality by adding a managed switch to it over mini-PCIe for a project and some learning with fun.

Have been looking around the Internet but couldn't find any suitable card readily available. Most of the results are for the controllers and custom design solutions by manufacturers like Realtek and Broadcom.

Does anyone here know of a readily available switch card which are supported by OpenWrt. One of the result mentioned access to the switch registers via SPI or i2c bus.

Also, what could be the probable pitfalls related to the speeds when using a PCIe interface for say a 5-8 ports gigabit switch.

Thanks in advance.

I've never heard of this...but doing searching, I could only find one:

But it only has 2 ports...and I don't think its a switch PHY.

Why not use a "smart" switch with a CLI? 8-port or larger switches available for under $100 both new and used. My Cisco SG300-series switches, as one example, can use SSH certificates for authentication and authorization and have a richer command set through the CLI than through the GUI.

PCIe can certainly handle gigabit speeds. The number of lanes depends on the rate and number of interfaces. However, the point of using a switch would seem to be to keep the data from ever flowing through the CPU.

Switch hardware isn't connected over PCIe. They usually physically connect one of the ethernet "ports" of the switch to an ethernet controller of the CPU (skipping some steps as this connection never becomes external), and then have some way to communicate between the CPU and the switch controller (like SPI or i2c as you found out)

You would probably want to look for switch controller devboards, but you are probably better off buying a device supported by OpenWrt that has a switch integrated already, and use that instead of the APU.

However, the point of using a switch would seem to be to keep the data from ever flowing through the CPU.

This is correct. A switch controller is simply faster and better than processing packets on a generic CPU running an operating system, that's the whole point of its existence.

Afaik there have been zero attempts at making an opensource switch firmware.

1 Like

If you're referring as specifically branded and marketed to switches and third-party switch firmware replacements, no. But there are a few devices with 8+ ports that are compatible with OpenWrt.

I should also note, that's because most switches are already unmanaged, or there is likely low or no flash. There's not a lot to a switch, most is done in hardware.

That's not the same thing. Managed switches are dedicated networking microcontrollers with a web interface, it's not a Linux system. It's not like routers with integrated switches supported by OpenWrt, where you have a "primary" CPU running Linux and a dumb switch controller where the configuration can be changed by the main CPU.

Besides, devices in OpenWrt with an hardware switch using upstream DSA switch driver treat each physical port as if it was a single ethernet controller.
The driver will implement bridging or VLAN you define "in software" in the actual hardware switch, but configuring them works like configuring a Linux server's ethernet interfaces.

Open a circa 2008-2014 Cisco Catalyst switch...I think you'll be surprised.

I do agree there is other MAC, TCP, etc. processing going on in a actual switch...that is indeed the software...using the available options.

...agreed

Afaik if it's running CatOS it's a switch microcontroller.

If they run Cisco's IOS then yeah they have a general-purpose CPU too and an operating system, and they are layer 3 devices, and can actually do routing and more advanced functions.

Anyway, point taken, the midrange and high-end businness-grade switches do have a general-purpose CPU and some kind of OS.

Admittedly I had the most common and cheap kind of managed switches in mind when I posted that.

1 Like

I found an implementation of what I was searching for here:

The chip on the switch's board has Realtek's logo. Still, I am not able to understand the connection from mini-PCIe to the switch completely. It seems an additional interface is visible to the OS while switch is an unmanaged one.

There is a project - https://cumulusnetworks.com/community/

It is based on Debian Linux and runs on high configuration datacenter switches. Was just wondering if something like it can be used to create open hardware switches?

That is such poorly made kit, who would tie mains voltage wires to ELV DC power wires :frowning:

1 Like

OH MY! Good catch! That circuit could have a lot of mains noise on it!

But yes...larger switches simply have rows of these, albeit better wired and connected together. I also think that must be a 100 Mbps switch, or it cannot be capable of transferring data to the Ethernet ports on the mainboard.

I'm 90% certain that this is some chinese router-ish garbage where the switch is just a dumb switch (not managed)

The minipcie looks a lot like an ethernet controller. The square chip is the controller, the large rectangular chip is for the phisical layer, see this minipcie ethernet controller card http://www.dpie.com/mini-pcie/ethernet/connect-tech-inc-mini-pcie-gbe

Also, if you see the Ethernet switch board, you see that the cables for the minipcie card come from an unpopulated Ethernet port slot. I'm 99% sure that it is just an ethernet connection, and you can't control a switch directly with that.

It is based on Debian Linux and runs on high configuration datacenter switches. Was just wondering if something like it can be used to create open hardware switches?

I don't know what you mean with "open hardware switches", but they are doing something similar to what OpenWrt does, they replace the firmware running on the general-purpose CPU for layer 3 features, while the actual hardware switch microcontroller is still running its own blob.

In their hardware list, they specify what brand and product name is this switch microcontroller for each of their supported devices. https://cumulusnetworks.com/products/hardware-compatibility-list/
"Broadcom Tomahawk" or "Mellanox Spectrum", for example, but there are others.

I have no idea if they are using firmware/drivers that can be redistributed and used by OpenWrt too, but if they are, then you could port OpenWrt on these swittches too.

2 Likes