I'm looking for recommendations on a device taking my 8MB/sec (megabyte/sec) connection over OpenVPN.
The device should be cheap (of course as cheap as possible) and use as little power as possible.
5GHZ is not a requirement (mostly using ethernet), multiple ethernet ports is not a requirement (1 lan and 1 wan is all I use), can be old/second hand.
Which devices would you suggest?
Consider Jalapeno from 8devices. Two Gigabit interfaces, hardware NAT engine and crypto engine, max power consumption 12,75W. Price 85€ and another 6,5€ for the shipping.
8 MB/s, which you clarify as "megabyte/sec" works out to 64 mbit/sec. Unfortunately that puts you well out of the "cheap" and "low power" range as many might define it. A MIPS device, on a good day, can handle 10-15 mbps of OpenVPN encryption, so the typical "cheap" devices are an order of magnitude too slow.
You're pretty much squarely in the very high-end, multi-core ARM or x86_64 arena.
Other VPN approaches, such as WireGuard, have lower CPU requirements, likely due to their choice of encryption algorithms. If you find their security meeting your needs, one commercial manufacturer claims performance on their MIPS-based device running recent OpenWRT 17 Mbps for OpenVPN and 68 Mbps for WireGuard. I won't say that those numbers are achievable in real-world conditions, but they're likely under the same conditions and can provide some useful guidance.
64 Mbps of openvpn is a nontrivial amount. Consider switching to wireguard which will cut your CPU cycles in half I'm guessing.
The jalapeno thing sounds like a possibility, espressobin maybe, and a decent mini PC with AES-NI is another option (will handle probably 300Mbps OVPN)
Done? (99$)
A raspberry pi?
Yeah at $99 the WRT32X seems like a good choice, and would be better than an R-pi for sure. Not quite sure if it'll handle 64Mbps of OpenVPN but I suspect it would handle that much wireguard.
Thanks for all the suggestions.
Does openwrt handle the AES instructions in ARM v8 such as A53, A57, cortex 8 ?
jalapeno and espressobin look nice, pcengines too, but I'd like to try squeeze the budget to the max with this exercise, so searching benchmarks on devices like the PI. Anyone seen any OVPN benchmarks?
I know and have used wireguard, super fast on sub-par hardware. I stopped using it when I realised there are privacy issues with it such as it being designed to have the client ISP IP and server to client delivered IP in the clear on the server, or it not having a randomization feature such as OVPN's remote-random.
a test made by me on mips with wireguard
https://lists.zx2c4.com/pipermail/wireguard/2018-June/003079.html
[ 0.000000] SoC: Qualcomm Atheros QCA9533 ver 1 rev 1
[ 0.000000] CPU clock: 550.000 MHz
Max down 37.4 Mbits/sec
Max up 53.7 Mbits/sec
[ 0.000000] SoC: Qualcomm Atheros QCA956X ver 1 rev 0
[ 0.000000] CPU clock: 775.000 MHz
Max down 82.8 Mbits/sec
Max up 73.6 Mbits/sec
is doing like 500mbps no sweat
That ~75 Mbits/s for WireGuard on a QCA956x is consistent with the OEM-advertised 68 Mbps for WireGuard for their device based on a QCA9563 at 775 MHz. As they are selling routers rather than promoting a specific VPN, I would tend to believe that their claim of 17 Mbps for OpenVPN on the same hardware is a reasonable estimate.
Not with bandwidth shaping/latency control/SQM though right?
Thanks for the wireguard datapoints. I was looking for performance datapoints of OpenVPN on boards like raspberry pi. Anyone?
Raspberry Pi 3B (64bit Ubuntu):
OpenSSL 1.1.0g 2 Nov 2017
built on: reproducible build, date unspecified
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr)
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/aarch64-linux-gnu/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-cbc 38586.71k 49584.69k 52905.56k 53966.17k 54255.62k 54417.89k
Raspberry Pi 3B+ (32bit Raspbian):
OpenSSL 1.1.0j 20 Nov 2018
built on: reproducible build, date unspecified
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr)
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/arm-linux-gnueabihf/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-cbc 34707.90k 48316.67k 53932.95k 55729.83k 56227.16k 56273.58k
Ran using openssl speed -evp aes-128-cbc
For comparison, Archer C7 v2 (MIPS QCA9558 at 720 MHz) on ath79 from recent master
OpenSSL 1.0.2p 14 Aug 2018
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,2,long) aes(partial) blowfish(ptr)
compiler: ccache_cc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -I/home/jeff/devel/openwrt/staging_dir/target-mips_24kc_musl/usr/include -I/home/jeff/devel/openwrt/staging_dir/target-mips_24kc_musl/include -I/home/jeff/devel/openwrt/staging_dir/toolchain-mips_24kc_gcc-7.3.0_musl/usr/include -I/home/jeff/devel/openwrt/staging_dir/toolchain-mips_24kc_gcc-7.3.0_musl/include/fortify -I/home/jeff/devel/openwrt/staging_dir/toolchain-mips_24kc_gcc-7.3.0_musl/include -znow -zrelro -DOPENSSL_SMALL_FOOTPRINT -DOPENSSL_NO_ERR -DTERMIOS -Os -pipe -mno-branch-likely -mips32r2 -mtune=24kc -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -msoft-float -iremap/home/jeff/devel/openwrt/build_dir/target-mips_24kc_musl/openssl-1.0.2p:openssl-1.0.2p -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -fpic -I/home/jeff/devel/openwrt/package/libs/openssl/include -ffunction-sections -fdata-sections -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DAES_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 9656.96k 11439.76k 12160.98k 12217.73k 12265.24k
about 4x slower than the Raspberry Pi, and a single-core device isn't going to reach those numbers if it's managing an interface, NAT-ing, and who knows what else at the same time.
Same command on an Orange Pi Zero Plus (note onboard wifi doesn't work with OpenWRT… but you can BYO-Wifi via USB).
Orange Pi Zero+ with H5 Chip (64bit Armbian):
OpenSSL 1.1.0j 20 Nov 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr)
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/aarch64-linux-gnu/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-cbc 54027.92k 175891.56k 368803.84k 525083.99k 603652.10k 608305.15k
cat /proc/cpuinfo tells me this has the AES instructions that the Raspberry Pi lacks. If raw encryption speed is your thing I think this is by far the absolute best price to speed you're going to find on any device out there that can run OpenWRT...
And last but not least to be complete here is my router, a Zotac CI327 running OpenWRT 18.06.0 with an Intel Celeron N3450 that has the AES-NI instruction set.
Celeron N3450 (OpenWRT x86-64):
OpenSSL 1.0.2p 14 Aug 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(idx,cisc,2,int) aes(partial) blowfish(idx)
compiler: x86_64-openwrt-linux-musl-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/target-x86_64_musl/usr/include -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/target-x86_64_musl/include -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/toolchain-x86_64_gcc-7.3.0_musl/usr/include -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/toolchain-x86_64_gcc-7.3.0_musl/include/fortify -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/toolchain-x86_64_gcc-7.3.0_musl/include -znow -zrelro -DOPENSSL_SMALL_FOOTPRINT -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -DOPENSSL_NO_ERR -m64 -DL_ENDIAN -DTERMIOS -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -iremap/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/build_dir/target-x86_64_musl/openssl-1.0.2p:openssl-1.0.2p -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -fpic -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/feeds/base/package/libs/openssl/include -ffunction-sections -fdata-sections -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 407676.64k 514757.82k 535363.24k 556226.56k 562492.76k
Thanks for all those benchmarks !
I could be tempted by the Orange Pi Zero+ with H5 Chip but am worried as it has only 1 ethernet and no wifi. How does one go about installing and more importantly configuring openwrt on such a device?
SD card, just like a Raspberry Pi. You mess it up, just take the card out and dd a new image.
PCEngines APU2C4 (amd64 at 1 GHz with AES-NI, 4 cores)
OpenSSL 1.1.1a-freebsd 20 Nov 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(8x,int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-cbc 71025.29k 142958.72k 196000.35k 210281.13k 219384.53k 223712.60k
though quite a bit more expensive than the Pi-class options (~US$150), it does come with three Intel nics, 4 GB of RAM, and several other niceties.
...or just buy a Up Squared board instead, newer hardware, 2 NICs (Realtek) however