What cheap low power OVPN for 10MBps?


#1

I'm looking for recommendations on a device taking my 8MB/sec (megabyte/sec) connection over OpenVPN.

The device should be cheap (of course as cheap as possible) and use as little power as possible.

5GHZ is not a requirement (mostly using ethernet), multiple ethernet ports is not a requirement (1 lan and 1 wan is all I use), can be old/second hand.

Which devices would you suggest?


#2

Consider Jalapeno from 8devices. Two Gigabit interfaces, hardware NAT engine and crypto engine, max power consumption 12,75W. Price 85€ and another 6,5€ for the shipping.


#3

8 MB/s, which you clarify as "megabyte/sec" works out to 64 mbit/sec. Unfortunately that puts you well out of the "cheap" and "low power" range as many might define it. A MIPS device, on a good day, can handle 10-15 mbps of OpenVPN encryption, so the typical "cheap" devices are an order of magnitude too slow.

You're pretty much squarely in the very high-end, multi-core ARM or x86_64 arena.

Other VPN approaches, such as WireGuard, have lower CPU requirements, likely due to their choice of encryption algorithms. If you find their security meeting your needs, one commercial manufacturer claims performance on their MIPS-based device running recent OpenWRT 17 Mbps for OpenVPN and 68 Mbps for WireGuard. I won't say that those numbers are achievable in real-world conditions, but they're likely under the same conditions and can provide some useful guidance.


#4

64 Mbps of openvpn is a nontrivial amount. Consider switching to wireguard which will cut your CPU cycles in half I'm guessing.

The jalapeno thing sounds like a possibility, espressobin maybe, and a decent mini PC with AES-NI is another option (will handle probably 300Mbps OVPN)


#5


Done? (99$)


#6

A raspberry pi?


#7

Yeah at $99 the WRT32X seems like a good choice, and would be better than an R-pi for sure. Not quite sure if it'll handle 64Mbps of OpenVPN but I suspect it would handle that much wireguard.


#8

Thanks for all the suggestions.

Does openwrt handle the AES instructions in ARM v8 such as A53, A57, cortex 8 ?

jalapeno and espressobin look nice, pcengines too, but I'd like to try squeeze the budget to the max with this exercise, so searching benchmarks on devices like the PI. Anyone seen any OVPN benchmarks?

I know and have used wireguard, super fast on sub-par hardware. I stopped using it when I realised there are privacy issues with it such as it being designed to have the client ISP IP and server to client delivered IP in the clear on the server, or it not having a randomization feature such as OVPN's remote-random.


#9

a test made by me on mips with wireguard

https://lists.zx2c4.com/pipermail/wireguard/2018-June/003079.html

[    0.000000] SoC: Qualcomm Atheros QCA9533 ver 1 rev 1
[    0.000000] CPU clock: 550.000 MHz

Max down 37.4 Mbits/sec
Max up   53.7 Mbits/sec

[    0.000000] SoC: Qualcomm Atheros QCA956X ver 1 rev 0
[    0.000000] CPU clock: 775.000 MHz

Max down  82.8 Mbits/sec
Max up    73.6 Mbits/sec

is doing like 500mbps no sweat


#10

That ~75 Mbits/s for WireGuard on a QCA956x is consistent with the OEM-advertised 68 Mbps for WireGuard for their device based on a QCA9563 at 775 MHz. As they are selling routers rather than promoting a specific VPN, I would tend to believe that their claim of 17 Mbps for OpenVPN on the same hardware is a reasonable estimate.


#11

Not with bandwidth shaping/latency control/SQM though right?


#12

Thanks for the wireguard datapoints. I was looking for performance datapoints of OpenVPN on boards like raspberry pi. Anyone?


#13

Raspberry Pi 3B (64bit Ubuntu):

OpenSSL 1.1.0g  2 Nov 2017
built on: reproducible build, date unspecified
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr)
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/aarch64-linux-gnu/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      38586.71k    49584.69k    52905.56k    53966.17k    54255.62k    54417.89k

Raspberry Pi 3B+ (32bit Raspbian):

OpenSSL 1.1.0j  20 Nov 2018
built on: reproducible build, date unspecified
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr)
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/arm-linux-gnueabihf/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      34707.90k    48316.67k    53932.95k    55729.83k    56227.16k    56273.58k

Ran using openssl speed -evp aes-128-cbc


#14

For comparison, Archer C7 v2 (MIPS QCA9558 at 720 MHz) on ath79 from recent master

OpenSSL 1.0.2p  14 Aug 2018
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,2,long) aes(partial) blowfish(ptr) 
compiler: ccache_cc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -I/home/jeff/devel/openwrt/staging_dir/target-mips_24kc_musl/usr/include -I/home/jeff/devel/openwrt/staging_dir/target-mips_24kc_musl/include -I/home/jeff/devel/openwrt/staging_dir/toolchain-mips_24kc_gcc-7.3.0_musl/usr/include -I/home/jeff/devel/openwrt/staging_dir/toolchain-mips_24kc_gcc-7.3.0_musl/include/fortify -I/home/jeff/devel/openwrt/staging_dir/toolchain-mips_24kc_gcc-7.3.0_musl/include -znow -zrelro -DOPENSSL_SMALL_FOOTPRINT -DOPENSSL_NO_ERR -DTERMIOS -Os -pipe -mno-branch-likely -mips32r2 -mtune=24kc -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -msoft-float -iremap/home/jeff/devel/openwrt/build_dir/target-mips_24kc_musl/openssl-1.0.2p:openssl-1.0.2p -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -fpic -I/home/jeff/devel/openwrt/package/libs/openssl/include -ffunction-sections -fdata-sections -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DAES_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc       9656.96k    11439.76k    12160.98k    12217.73k    12265.24k

about 4x slower than the Raspberry Pi, and a single-core device isn't going to reach those numbers if it's managing an interface, NAT-ing, and who knows what else at the same time.


#15

Same command on an Orange Pi Zero Plus (note onboard wifi doesn't work with OpenWRT… but you can BYO-Wifi via USB).

Orange Pi Zero+ with H5 Chip (64bit Armbian):

OpenSSL 1.1.0j  20 Nov 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr)
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/aarch64-linux-gnu/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      54027.92k   175891.56k   368803.84k   525083.99k   603652.10k   608305.15k

cat /proc/cpuinfo tells me this has the AES instructions that the Raspberry Pi lacks. If raw encryption speed is your thing I think this is by far the absolute best price to speed you're going to find on any device out there that can run OpenWRT...


#16

And last but not least to be complete here is my router, a Zotac CI327 running OpenWRT 18.06.0 with an Intel Celeron N3450 that has the AES-NI instruction set.

Celeron N3450 (OpenWRT x86-64):

OpenSSL 1.0.2p  14 Aug 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(idx,cisc,2,int) aes(partial) blowfish(idx)
compiler: x86_64-openwrt-linux-musl-gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/target-x86_64_musl/usr/include -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/target-x86_64_musl/include -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/toolchain-x86_64_gcc-7.3.0_musl/usr/include -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/toolchain-x86_64_gcc-7.3.0_musl/include/fortify -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/staging_dir/toolchain-x86_64_gcc-7.3.0_musl/include -znow -zrelro -DOPENSSL_SMALL_FOOTPRINT -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -DOPENSSL_NO_ERR -m64 -DL_ENDIAN -DTERMIOS -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -iremap/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/build_dir/target-x86_64_musl/openssl-1.0.2p:openssl-1.0.2p -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -fpic -I/build/openwrt-18.06/slaves/phase2/x86_64/build/sdk/feeds/base/package/libs/openssl/include -ffunction-sections -fdata-sections -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc     407676.64k   514757.82k   535363.24k   556226.56k   562492.76k

#17

Thanks for all those benchmarks !
I could be tempted by the Orange Pi Zero+ with H5 Chip but am worried as it has only 1 ethernet and no wifi. How does one go about installing and more importantly configuring openwrt on such a device?


#18

SD card, just like a Raspberry Pi. You mess it up, just take the card out and dd a new image.


#19

PCEngines APU2C4 (amd64 at 1 GHz with AES-NI, 4 cores)

OpenSSL 1.1.1a-freebsd  20 Nov 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(8x,int) des(int) aes(partial) idea(int) blowfish(ptr) 
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      71025.29k   142958.72k   196000.35k   210281.13k   219384.53k   223712.60k

though quite a bit more expensive than the Pi-class options (~US$150), it does come with three Intel nics, 4 GB of RAM, and several other niceties.


#20

...or just buy a Up Squared board instead, newer hardware, 2 NICs (Realtek) however