OPNSense DEC740 with OpenWrt

Hi,

I've always wanted to have a device that can handle two times (or more) 10G and OpenWrt.
This looks like an affordable option and I was wondering if someone tried it already?
As this is X86 it should work with OpenWrt out of the box, but the SFP+ I don't know if this might be a show stopper.

I would really love to buy one though.

It has

  • Ryzen Embedded V1500B with 4 Cores and 8 Threads with up to 2,2 GHz.
  • 2 x 10G SFP+
  • 3 x 1G Copper
  • 128 GB SSD
  • 4 GB RAM
  • 12-16 W TDP

for ~700 €

What do you think?

1 Like

Expensive :slight_smile:

If you want dual 10G paired with a low power consumption and an OKish CPU performance then I think it's really worth the money.
Of course it doesn't compare to all the typical low end MIPS devices.

Another option but with only one 10G interface would be the Mochabin, which will probably surface in march next year. It's roughly a third of the price. but if you want to route 10 between networks it will not be good enough.

If you're going to build one yourself, it's going to be larger, have fans, and perhaps a higher TDP.

Used dual sftp+ NICs are $25 on ebay, quad port gbit NICs are less.

Building a low end pc hosting the two above, shouldn't cost more than $250-300, depending on what parts you use, and if they're used or new.

Yeah that's what I currently do. :stuck_out_tongue_winking_eye:

Then I suggest you keep doing it :wink:
I do it too.

That's some fine looking piece of hardware, but most of the premium you pay is for the size.

on x86 most of the low level magic wizardry bs that makes SFP ports hard to support with OpenWrt on an embedded device is handled by the BIOS/UEFI, or the network controller firmware.
Also OPNSense is FreeBSD and that means their driver support is very limited, so it's either an Intel or a Broadcomm network chipset.

So I'd say 99% likely to "just work".

Yes but not by a whole lot. Especially if you get a low profile chassis.

A slightly bulkier case, especially if it doesn't need to be rack-mountable (1U/ 2U devices are particular difficult to cool, unless you mount several jet engines inside), tends to be much easier (and therefore less noisy-) to keep cool (vertical tends to be easier than horizontal). Very few devices can really operate passively (and that usually comes at a premium), but an 85-120 mm fan running at low speed can be pretty silent (especially higher priced/ better fans) and keep the case within safe margins.

I mean common x86 hardware of course is an option. On the other hand I really think you won't even get near a power consumption of around 25 W, paired with those kind of specs. My current sollution is an Intel 10700t, marked as 35W TDP. I know TDP doesn't translate to power consumption but the actuall power consumption is way higher with around 55W and peaks over 100W.
With AMD current low power CPUs are equal to not available and Intel really doesn't do a good job when it comes to low power CPU paired with performance in range of desktop CPU (Atoms snails aside).

So all options taken into account, I really think this is an excellent piece of hardware if price isn't the limiting factor. If I will get my hands on some of those boxes I will be happy to share my findings with you.

It depends, if you can set power targets or not. On AMD you can usually do that in BIOS, I don't know for Intel. This will of course tank the performance because that's what determines how much and how long they run at burst clocks, but it's also what they do in embedded devices and laptops.

Also it depends from how big of a power hog is the NIC cards. Older (first/second gen) dual port 10Gbit cards can easly burn 15W on idle (and get hot as the Sun), while more modern cards like mellanox connectx3 can go down to a few watts per port and actually not need a fan to not melt down

Also fun fact, the motherboard choice (and how much you can disable in the BIOS) makes a difference. Well-optimized boards can go down to 5w on idle, badly optimized boards can waste 20w doing nothing.

For most uses (aside from very busy enterprises, keeping the router loaded all time), you mostly need to optimize for the idle use case, as that's the state your router will remain in for most of the day. How much it may burn under full load is much less relevant, it quickly switches to full power, gets its job done and then continues to idle (or close to that).

With AMD, there is the option to leave out the southbridge on small, SOC-like, boards, saving a lot of (idle-) power - but that also kills of your extensibility with additional ethernet cards.

The mere presence of a dedicated (not CPU internal-) graphics card usually costs you ~20-30 watts idle alone.

So the major powersaving aspect to take care of is indeed the mainboard, less extensibility == less power - and with >2.5 GBit/s ethernet, the network card(s) matters a lot as well.

Allright guys I have one DEC740 in my hands and I'm going to test it to the bones. Firstly I will do some benchmarks with the stock OPNsense firmware and publish (german blog) them.

Secondly I plan to install OpenWrt on it but I just figured those 10G SFP+ interfaces are those of the AMD Ryzen Embedded SoC. For FreeBSD there has been a new driver called axp. Does someone know if those might work on Kernel 5.4?

Edit:
I just tried it. As the hardware only supports EFI boot one has to choose the efi image.
21.02.1 doesn't not work (Kernel 5.4)
snapshot seems to work (Kernel 5.10)
some amd xgbe driver is being loaded.

1 Like

Is it fine i do not believe in Ryzen CPU due to massive latency issues with 1st and 3rd generation? I would think it will affect performance, even i would not notice it..
I prefer use Intel instead of, but if we talk platform, you got my like, i was first one here in this forum posted one simple thing -
OpenWrt should drop major part of platforms and focuse on x86 and according to the residual principle ath and mediatek only.

what latency issues? The only latency issues I'm aware of are on Windows, either due to issues in the platform support package that AMD has provided them or because Microsoft has optimized their kernel to run on Intel and this is falling flat on AMD that isn't exactly the same

why though. They are maintained by different people anyway so it's not like reducing platforms will increase amount of developers on your favourite archs

1 Like

thanks for taking the time to document your tests

I was wondering if this device supports ECC RAM, as the CPU should support it so it's possible. Do you see any ECC RAM options in the UEFI setup?

I assume the DIMM module isn't ECC.

OpenWrt lacks the EDAC packages/modules to test if ECC is enabled (and log ECC errors), but if you install Debian or OpenSUSE you should get something like this if you write

dmesg | grep EDAC

in the terminal

[    0.294934] EDAC MC: Ver: 3.0.0
[    5.901368] EDAC amd64: MCT channel count: 2
[    5.910834] EDAC MC0: Giving out device to module amd64_edac controller F17h_M10h: DEV 0000:00:18.3 (INTERRUPT)
[    5.910840] EDAC amd64: F17h_M10h detected (node 0).
[    5.910846] EDAC MC: UMC0 chip selects:
[    5.910847] EDAC amd64: MC: 0:  8192MB 1:  8192MB
[    5.910849] EDAC amd64: MC: 2:     0MB 3:     0MB
[    5.910853] EDAC MC: UMC1 chip selects:
[    5.910854] EDAC amd64: MC: 0:  8192MB 1:  8192MB
[    5.910855] EDAC amd64: MC: 2:     0MB 3:     0MB
[    5.910856] EDAC amd64: using x8 syndromes.
[    5.910867] EDAC PCI0: Giving out device to module amd64_edac controller EDAC PCI controller: DEV 0000:00:18.0 (POLLED)
[    5.910869] AMD64 EDAC driver v3.5.0

I cannot find any ECC memory option in the EFI menu. Since the installed module is Very Low Profile I doubt it would fit into the case anyway.

Edit:
Sadly the 10G interfaces won't come up. I file a bug report for this.

Thanks for checking. Very Low Profile Unbuffered ECC DDR4 modules exist at up to 32GB per module (but aren't easy to get and probably not cheap, a normal height 32GB ECC DDR4 module costs around 200 euros) https://us.transcend-info.com/embedded/product/embedded-dram-modules/ddr4-ecc-dimms-very-low-profile
but without firmware support in the UEFI they would just run in non-ECC mode.