Hello
I am currently using a Mikrotik Rb5009 with an SFP+ module as 10G fiber uplink, it works but I am less than happy with router OS.
So my quest is to build a passively cooled router with dual SFP+ connections (one to the ISP, other to a local switch). Does anybody have a pointer to suitable hardware? I assume a Intel n100 or even n6005 would be enough (basically, it needs to NAT at reasonable speed, not necessarily even 10G) but I need a board with pci-e for the dual SFP+ card...
Should be, all those dual X520 are x8. I think even x4 would be enough given that pci-e 3 is 1GB/s per lane.
There's that Chinese R86s thing that even comes with (I think Mellanox) dual SFP+ but that one has a fan of unknown noise level...
I am either way not terribly fussed if I can max out the 10G, I have all gig-e after that (in wall wiring will do at most 2.5G it seems). It's actually more that the uplink is crucial (bad 4G reception means I have no real backup) and getting a spare router with suitable optics is tricky.
That would be the fall back at the price of presumably using significantly more power. That is the real upside of those x86 SoCs. And given I only need NAT, the CPU power goes to waste (seeing that I really rather not use border gateway for other duties such as NAS)...
I have something similar to this many years ago, the conclusion is you still need a case fan for cooling (those ~1200rpm slow spinning one is suffice), using it completely without any fan will get overheat when the loading is high. But of course such case fan is usually running less noisy than a normal CPU fan.
I've bought couple of fujitsu siemens futro s920 4x 2.4GHz thin clients for cheap, i've had some nice nics to play . When i try with intel x520 i can top with 5.7Gbps betwean two sfp+ ports without aby tweaks. Since there is pcie 2.0 x4 so bus bandwith is 16Gbps there is room for optimisation. Maybe with mellanox or Chelsio it can go faster.
X520 is PCI-E v2.0, so with x4 dual port you cannot run both ports at full speed (I have this card on Synology NAS which is exactly PCI-E 3.0 + x4, the total throughput when running both ports is ~13Gbps, which means your NAT will hit max. throughput < 7Gbps). Usually Mellanox ConnectX-3 (which is the one being used on your mentioned R86S) is preferred, noted that Pentium Silver 6005 has PCI-E 3.0 x8 but your board peripherals like USB3/storage have shared PCI-E bandwidth so you can never get all x8 for a dedicated PCI-E card, and I believe this is why Mellanox Connect-3 being used on R86S.
Given the PCI-E lanes issue on usual motherboards (in consumer market), without a custom made things like R86S (most non-critical components cut down to save PCI-E lanes), you can only look at platforms with CPU that has more PCI-E lanes built-in, and you'll see that those need to use CPU fan. I won't say there is no exception, the Intel Denverton (Atom C35x8 series) from Supermicro can fit your need but you might give up once you know the price.
4 x 500MB/s minus coding is ~ 1.8GB/s in both side so if it negotiate link pcie 2.0 x4 i see no problem to saturate ~ 18 Gbps , i'm not in home so i can't show You prove today. This poor cpu is capable after some tweaks to do nat at speed of ~7Gbps or more, need to tweak interrupts and some buffers.
Someone already explained why you can't get > 14Gbps for Intel X520-DA2 on a 2.0 x4, and this is almost the same as what I tested. My Synology NAS is 4C8T AMD Ryzen CPU which is a lot faster than 6005 so it's definitely not a CPU issue.
Agreed. It can be passively cooled, but I did choose to put a decent Noctua case fan in my box and a little PWM fan on the passive cooler (I don't use this exact model, but very similar - the A2SDi-8C-HLN4F).
To be honest, I think the case fan is necessary but the small CPU fan is unnceccesary - the temps on the CPU are pretty low (screenshot from IPMI web interface of this board)
I'd be alright with 7gbit but still no real luck finding a board with even 4x pci-e aside of the super-micro stuff which is expensive enough that regular desktop hw looks like a better deal, power consumption be damned.
Maybe I will end up with a R86s after all. Can anybody comment on the noise level?
Only other option Ive seen is mcbin but software for that one is ancient. Or trying to somehow get a spare Rb5009 and suffer from routeros...
Yes, agreed that the Supermicro boards look expensive. From memory, the one I'm using cost about ÂŁ350 GBP, although that does include both motherboard and CPU.
For me, the principal attraction of this board is its much lower power consumption, mini-ITX form factor and IPMI management stack.
The mini-ITX form factor results in a footprint that is about the same size as a typical desktop consumer router and doesn't make a great big noise sitting a foot away from my head.
The other major benefit is that has a bunch of ethernet interfaces on it. more than you'd get on a regular desktop motherboard. There are many configurations available including multiple SFP+ ports and 10Gbe ports
Well realistically mine will live in the utility cabinet (where fiber and cat5 trunk is) but that is in the middle of the apartment. A really quiet fan (<25db say) would be alright there... But less heat would definitely be welcome as there is no real airflow through the cabinet.
As for openwrt on the rB009, seeing that I have no spare I did not dare. If I could find another RB5009, I would. Even upgrading routeros makes me queasy everytime taking into account that 4G/5G backup is nigh useless in that spot...
The ISP is semi helpful in this regard: they let you use whatever you like (I even got my own optics from FS) and only offer a 10g Zyxel thingy but that does not seem to have gotten a firmware update since 2020...
