Comparing CPU/SOC performance?

That, and QCA KRAIT300 ~= Cortex A15 has very different performance levels than ARM Cortex A7.

Edit: Btw., I wouldn't trust the actual (example?) figures in that chart too much, yes mvebu is much faster than ipq806x at plain routing, but when it comes to crypto operations (and that is the limiting factor for VPN usage), both mvebu and ipq806x are pretty close to each other (actually with ipq8065 slightly in the lead).

1 Like

mvebu has CESA hardware acceleration crypto engine whose driver is in-tree. With OpenSSL and cryptodev, OpenVPN can offload AES-ECB/CBC en/decryption to it.
ipq806x also has a crypto engine in its NSS core but no driver is available (yet)

1 Like

And for MT7621 and MT7628, there is an out-of-tree and WIP driver available, authored by @drbrains


https://github.com/vschagen/mtk-eip93 (MT7621)
By the time I commented, the crypto driver for MT7628 is stable while MT7621 one needs more work.

1 Like

OpenVPN does not use ECB and even with hardware crypto, OpenVPN throughput is not even close to as high as that chart shows on mvebu (which kind of looks to be just OpenSSL performance) due to the massive overhead OpenVPN has.

To compare the performance of multiple vendor's SOCs, either you have to have hands own experience on them or read reviews by users. Reviews may reflect the the personal preferences of the users.

Nah, ipq40xx does not have a NSS (Network Subsystem - Dedicated CPU-cores + 3 Ethernet-Macs + Switch). Instead it's called ESS (Ethernet Subsystem) over there which just includes one Ethernet-Mac and the internal Switch + transceivers.

So the crypto on the ipq40xx is just a on a DMA channel and can used by standard ALG interface.

This is a list of the currently supported cryptos (from /proc/crypto)

name         : hmac(sha256)
driver       : hmac-sha256-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ahash
async        : yes
blocksize    : 64
digestsize   : 32

name         : hmac(sha1)
driver       : hmac-sha1-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ahash
async        : yes
blocksize    : 64
digestsize   : 20

name         : sha256
driver       : sha256-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ahash
async        : yes
blocksize    : 64
digestsize   : 32

name         : sha1
driver       : sha1-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ahash
async        : yes
blocksize    : 64
digestsize   : 20

name         : cbc(des3_ede)
driver       : cbc-3des-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 8 
min keysize  : 24
max keysize  : 24
ivsize       : 8 
geniv        : <default>

name         : ecb(des3_ede)
driver       : ecb-3des-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 8 
min keysize  : 24
max keysize  : 24
ivsize       : 0 
geniv        : <default>

name         : cbc(des)
driver       : cbc-des-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 8 
min keysize  : 8 
max keysize  : 8 
ivsize       : 8 
geniv        : <default>

name         : ecb(des)
driver       : ecb-des-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 8 
min keysize  : 8 
max keysize  : 8 
ivsize       : 0 
geniv        : <default>

name         : xts(aes)
driver       : xts-aes-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 16
min keysize  : 16
max keysize  : 32
ivsize       : 16
geniv        : <default>

name         : ctr(aes)
driver       : ctr-aes-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 16
min keysize  : 16
max keysize  : 32
ivsize       : 16
geniv        : <default>

name         : cbc(aes)
driver       : cbc-aes-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 16
min keysize  : 16
max keysize  : 32
ivsize       : 16
geniv        : <default>

name         : ecb(aes)
driver       : ecb-aes-qce
module       : kernel
priority     : 300
refcnt       : 1 
selftest     : passed
internal     : no
type         : ablkcipher
async        : yes
blocksize    : 16
min keysize  : 16
max keysize  : 32
ivsize       : 16
geniv        : <default>

So there are "direct" offloads available for hmac-sha256, sha256, xts-aes, ctr-aes, cbc-aes and ecb-aes.
(I skipped the (3)des and sha1 since they should not be used anymore).

1 Like

So that means it doesn't require additional (out-of-tree) crypto driver to run on master OpenWrt?

The qce driver is available in the upstream vanilla linux for some time now:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/crypto/qcom-qce.txt
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/drivers/crypto/qce

1 Like

openssl engine -pre DUMP_INFO devcrypto detects the qce driver as software crypto driver:

root@OpenWrt:~# openssl engine -pre DUMP_INFO devcrypto
(devcrypto) /dev/crypto engine
Information about ciphers supported by the /dev/crypto engine:
Cipher DES-CBC, NID=31, /dev/crypto info: id=1, driver=cbc-des-qce (software)
Cipher DES-EDE3-CBC, NID=44, /dev/crypto info: id=2, driver=cbc-3des-qce (software)
Cipher BF-CBC, NID=91, /dev/crypto info: id=3, CIOCGSESSION (session open call) failed
Cipher CAST5-CBC, NID=108, /dev/crypto info: id=4, CIOCGSESSION (session open call) failed
Cipher AES-128-CBC, NID=419, /dev/crypto info: id=11, driver=cbc-aes-qce (software)
Cipher AES-192-CBC, NID=423, /dev/crypto info: id=11, driver=cbc-aes-qce (software)
Cipher AES-256-CBC, NID=427, /dev/crypto info: id=11, driver=cbc-aes-qce (software)
Cipher RC4, NID=5, /dev/crypto info: id=12, CIOCGSESSION (session open call) failed
Cipher AES-128-CTR, NID=904, /dev/crypto info: id=21, driver=ctr-aes-qce (software)
Cipher AES-192-CTR, NID=905, /dev/crypto info: id=21, driver=ctr-aes-qce (software)
Cipher AES-256-CTR, NID=906, /dev/crypto info: id=21, driver=ctr-aes-qce (software)
Cipher AES-128-ECB, NID=418, /dev/crypto info: id=23, driver=ecb-aes-qce (software)
Cipher AES-192-ECB, NID=422, /dev/crypto info: id=23, driver=ecb-aes-qce (software)
Cipher AES-256-ECB, NID=426, /dev/crypto info: id=23, driver=ecb-aes-qce (software)

Information about digests supported by the /dev/crypto engine:
Digest MD5, NID=4, /dev/crypto info: id=13, driver=md5-generic (software), CIOCCPHASH capable
Digest SHA1, NID=64, /dev/crypto info: id=14, driver=sha1-qce (software), CIOCCPHASH capable
Digest RIPEMD160, NID=117, /dev/crypto info: id=102, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA224, NID=675, /dev/crypto info: id=103, driver=sha224-generic (software), CIOCCPHASH capable
Digest SHA256, NID=672, /dev/crypto info: id=104, driver=sha256-qce (software), CIOCCPHASH capable
Digest SHA384, NID=673, /dev/crypto info: id=105, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA512, NID=674, /dev/crypto info: id=106, driver=unknown. CIOCGSESSION (session open) failed

[Success]: DUMP_INFO

@cotequeiroz How to make it detect them as hw accelerated?

@LGA1150, the linux module has to set the CRYPTO_ALG_KERN_DRIVER_ONLY flag in alg->cra_flags to be detected as a hardware driver. I'm looking at drivers/crypto/qce/ablkcipher.c and drivers/crypto/qce/sha.c, and they are indeed missing.

The meaning of the flag is that the cipher/hash is is only available via a kernel driver (that means direct access to hardware, which should not be allowed outside of the kernel). If the cipher implementation might be available by using an instruction set (which can be accessed straight from userspace) or by porting the kernel code, then it must not be set. I'm not certain if this is a mistake or not, but it does seem like it. You can circumvent it by enabling software drivers and then selecting only the accelerated algorithms. Make sure digests are disabled! They don't work with everything, especially across forks, so you'll get locked out of openssh if you enable them.
My suggestion:

openssl_conf=openssl_conf

[openssl_conf]
engines=engines

[engines]
devcrypto=devcrypto

[devcrypto]
USE_SOFTDRIVERS=1
CIPHERS=DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC
DIGESTS=NONE

I know that there are more ciphers available, but ECB-mode is mostly used for PRNG, in 16-byte blocks, so it will slow you down and cause trouble with forks as well. I have my doubts that CTR-mode is really updating the IV correctly--I'm not sure who should be responsible for it, cryptodev, the hardware driver, or somewhere else in the kernel, but at least the regular software driver does not do it right. It may be worth testing it.

Edit:
It seems that the CRYPTO_ALG_KERN_DRIVER_ONLY flag should have been set. I will try my luck in submitting a patch upstream.

1 Like

@LGA1150
Can you, or someone with access to IPQ40XX/IPQ806X hardware, give https://patchwork.ozlabs.org/patch/1165442/ a try?

1 Like

Is that patch “complete” or are some of the configuration changes outlined above also needed?

The patch should do it by itself. The output of the DUMP_INFO command above should change the *-qce drivers from (software) to (hw accelerated). You can apply the configuration changes as well, and they should not change the DUMP_INFO output. If it works, then the USE_SOFTDRIVERS=1 line won't be necessary anymore. I would recommend to use the CIPHERS= and DIGESTS= lines for the reasons I explained earlier.

1 Like

ipq8065/ nbg6817 (kernel 4.19):

# openssl engine -pre DUMP_INFO devcrypto 
(devcrypto) /dev/crypto engine
Information about ciphers supported by the /dev/crypto engine:
Cipher DES-CBC, NID=31, /dev/crypto info: id=1, driver=cbc(des-generic) (software)
Cipher DES-EDE3-CBC, NID=44, /dev/crypto info: id=2, driver=cbc(des3_ede-generic) (software)
Cipher BF-CBC, NID=91, /dev/crypto info: id=3, CIOCGSESSION (session open call) failed
Cipher CAST5-CBC, NID=108, /dev/crypto info: id=4, CIOCGSESSION (session open call) failed
Cipher AES-128-CBC, NID=419, /dev/crypto info: id=11, driver=cbc(aes-generic) (software)
Cipher AES-192-CBC, NID=423, /dev/crypto info: id=11, driver=cbc(aes-generic) (software)
Cipher AES-256-CBC, NID=427, /dev/crypto info: id=11, driver=cbc(aes-generic) (software)
Cipher RC4, NID=5, /dev/crypto info: id=12, CIOCGSESSION (session open call) failed
Cipher AES-128-CTR, NID=904, /dev/crypto info: id=21, driver=ctr(aes-generic) (software)
Cipher AES-192-CTR, NID=905, /dev/crypto info: id=21, driver=ctr(aes-generic) (software)
Cipher AES-256-CTR, NID=906, /dev/crypto info: id=21, driver=ctr(aes-generic) (software)
Cipher AES-128-ECB, NID=418, /dev/crypto info: id=23, driver=ecb(aes-generic) (software)
Cipher AES-192-ECB, NID=422, /dev/crypto info: id=23, driver=ecb(aes-generic) (software)
Cipher AES-256-ECB, NID=426, /dev/crypto info: id=23, driver=ecb(aes-generic) (software)

Information about digests supported by the /dev/crypto engine:
Digest MD5, NID=4, /dev/crypto info: id=13, driver=md5-generic (software), CIOCCPHASH capable
Digest SHA1, NID=64, /dev/crypto info: id=14, driver=sha1-generic (software), CIOCCPHASH capable
Digest RIPEMD160, NID=117, /dev/crypto info: id=102, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA224, NID=675, /dev/crypto info: id=103, driver=sha224-generic (software), CIOCCPHASH capable
Digest SHA256, NID=672, /dev/crypto info: id=104, driver=sha256-generic (software), CIOCCPHASH capable
Digest SHA384, NID=673, /dev/crypto info: id=105, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA512, NID=674, /dev/crypto info: id=106, driver=unknown. CIOCGSESSION (session open) failed

[Success]: DUMP_INFO
root@nbg6817:~# openssl engine -pre DUMP_INFO devcrypto speed md5 sha1 sha256 sha512 des des-ede3 aes-128-cbc aes-192-cbc aes-256-cbc rsa2048 dsa2048
(devcrypto) /dev/crypto engine
Information about ciphers supported by the /dev/crypto engine:
Cipher DES-CBC, NID=31, /dev/crypto info: id=1, driver=cbc(des-generic) (software)
Cipher DES-EDE3-CBC, NID=44, /dev/crypto info: id=2, driver=cbc(des3_ede-generic) (software)
Cipher BF-CBC, NID=91, /dev/crypto info: id=3, CIOCGSESSION (session open call) failed
Cipher CAST5-CBC, NID=108, /dev/crypto info: id=4, CIOCGSESSION (session open call) failed
Cipher AES-128-CBC, NID=419, /dev/crypto info: id=11, driver=cbc(aes-generic) (software)
Cipher AES-192-CBC, NID=423, /dev/crypto info: id=11, driver=cbc(aes-generic) (software)
Cipher AES-256-CBC, NID=427, /dev/crypto info: id=11, driver=cbc(aes-generic) (software)
Cipher RC4, NID=5, /dev/crypto info: id=12, CIOCGSESSION (session open call) failed
Cipher AES-128-CTR, NID=904, /dev/crypto info: id=21, driver=ctr(aes-generic) (software)
Cipher AES-192-CTR, NID=905, /dev/crypto info: id=21, driver=ctr(aes-generic) (software)
Cipher AES-256-CTR, NID=906, /dev/crypto info: id=21, driver=ctr(aes-generic) (software)
Cipher AES-128-ECB, NID=418, /dev/crypto info: id=23, driver=ecb(aes-generic) (software)
Cipher AES-192-ECB, NID=422, /dev/crypto info: id=23, driver=ecb(aes-generic) (software)
Cipher AES-256-ECB, NID=426, /dev/crypto info: id=23, driver=ecb(aes-generic) (software)

Information about digests supported by the /dev/crypto engine:
Digest MD5, NID=4, /dev/crypto info: id=13, driver=md5-generic (software), CIOCCPHASH capable
Digest SHA1, NID=64, /dev/crypto info: id=14, driver=sha1-generic (software), CIOCCPHASH capable
Digest RIPEMD160, NID=117, /dev/crypto info: id=102, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA224, NID=675, /dev/crypto info: id=103, driver=sha224-generic (software), CIOCCPHASH capable
Digest SHA256, NID=672, /dev/crypto info: id=104, driver=sha256-generic (software), CIOCCPHASH capable
Digest SHA384, NID=673, /dev/crypto info: id=105, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA512, NID=674, /dev/crypto info: id=106, driver=unknown. CIOCGSESSION (session open) failed

[Success]: DUMP_INFO
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/speed.so): Error loading shared library /usr/lib/engines-1.1/speed.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=speed
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/md5.so): Error loading shared library /usr/lib/engines-1.1/md5.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=md5
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/sha1.so): Error loading shared library /usr/lib/engines-1.1/sha1.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=sha1
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/sha256.so): Error loading shared library /usr/lib/engines-1.1/sha256.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=sha256
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/sha512.so): Error loading shared library /usr/lib/engines-1.1/sha512.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=sha512
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/des.so): Error loading shared library /usr/lib/engines-1.1/des.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=des
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/des-ede3.so): Error loading shared library /usr/lib/engines-1.1/des-ede3.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=des-ede3
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/aes-128-cbc.so): Error loading shared library /usr/lib/engines-1.1/aes-128-cbc.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=aes-128-cbc
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/aes-192-cbc.so): Error loading shared library /usr/lib/engines-1.1/aes-192-cbc.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=aes-192-cbc
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/aes-256-cbc.so): Error loading shared library /usr/lib/engines-1.1/aes-256-cbc.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=aes-256-cbc
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/rsa2048.so): Error loading shared library /usr/lib/engines-1.1/rsa2048.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=rsa2048
3070199140:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/engines-1.1/dsa2048.so): Error loading shared library /usr/lib/engines-1.1/dsa2048.so: No such file or directory
3070199140:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
3070199140:error:260B6084:engine routines:dynamic_load:dso not found:crypto/engine/eng_dyn.c:414:
3070199140:error:2606A074:engine routines:ENGINE_by_id:no such engine:crypto/engine/eng_list.c:334:id=dsa2048

But, as I've never looked into cryptodev before, I might be missing something:

$ grep --colour -i -e openssl -e crypto .config
# CONFIG_BUSYBOX_DEFAULT_FEATURE_WGET_OPENSSL is not set
# CONFIG_PACKAGE_openwisp-config-openssl is not set
# CONFIG_ZABBIX_OPENSSL is not set
# Cryptographic API modules
CONFIG_PACKAGE_kmod-crypto-acompress=m
CONFIG_PACKAGE_kmod-crypto-aead=m
# CONFIG_PACKAGE_kmod-crypto-arc4 is not set
CONFIG_PACKAGE_kmod-crypto-authenc=m
CONFIG_PACKAGE_kmod-crypto-cbc=m
# CONFIG_PACKAGE_kmod-crypto-ccm is not set
# CONFIG_PACKAGE_kmod-crypto-cmac is not set
CONFIG_PACKAGE_kmod-crypto-crc32=m
CONFIG_PACKAGE_kmod-crypto-crc32c=m
# CONFIG_PACKAGE_kmod-crypto-ctr is not set
# CONFIG_PACKAGE_kmod-crypto-cts is not set
CONFIG_PACKAGE_kmod-crypto-deflate=m
CONFIG_PACKAGE_kmod-crypto-des=m
# CONFIG_PACKAGE_kmod-crypto-ecb is not set
# CONFIG_PACKAGE_kmod-crypto-ecdh is not set
CONFIG_PACKAGE_kmod-crypto-echainiv=m
# CONFIG_PACKAGE_kmod-crypto-fcrypt is not set
# CONFIG_PACKAGE_kmod-crypto-gcm is not set
# CONFIG_PACKAGE_kmod-crypto-gf128 is not set
# CONFIG_PACKAGE_kmod-crypto-ghash is not set
CONFIG_PACKAGE_kmod-crypto-hash=m
CONFIG_PACKAGE_kmod-crypto-hmac=m
# CONFIG_PACKAGE_kmod-crypto-hw-ccp is not set
# CONFIG_PACKAGE_kmod-crypto-hw-geode is not set
# CONFIG_PACKAGE_kmod-crypto-hw-hifn-795x is not set
# CONFIG_PACKAGE_kmod-crypto-hw-padlock is not set
# CONFIG_PACKAGE_kmod-crypto-hw-talitos is not set
CONFIG_PACKAGE_kmod-crypto-iv=m
CONFIG_PACKAGE_kmod-crypto-manager=m
# CONFIG_PACKAGE_kmod-crypto-md4 is not set
CONFIG_PACKAGE_kmod-crypto-md5=m
# CONFIG_PACKAGE_kmod-crypto-michael-mic is not set
# CONFIG_PACKAGE_kmod-crypto-misc is not set
CONFIG_PACKAGE_kmod-crypto-null=m
# CONFIG_PACKAGE_kmod-crypto-pcbc is not set
CONFIG_PACKAGE_kmod-crypto-pcompress=m
# CONFIG_PACKAGE_kmod-crypto-rmd160 is not set
CONFIG_PACKAGE_kmod-crypto-rng=m
# CONFIG_PACKAGE_kmod-crypto-seqiv is not set
CONFIG_PACKAGE_kmod-crypto-sha1=m
CONFIG_PACKAGE_kmod-crypto-sha256=m
# CONFIG_PACKAGE_kmod-crypto-sha512 is not set
# CONFIG_PACKAGE_kmod-crypto-test is not set
CONFIG_PACKAGE_kmod-crypto-user=m
CONFIG_PACKAGE_kmod-crypto-wq=m
# CONFIG_PACKAGE_kmod-crypto-xcbc is not set
# CONFIG_PACKAGE_kmod-crypto-xts is not set
CONFIG_PACKAGE_kmod-cryptodev=m
# CONFIG_PACKAGE_erlang-crypto is not set
# CONFIG_PACKAGE_lua-openssl is not set
# CONFIG_PACKAGE_python-asn1crypto is not set
# CONFIG_PACKAGE_python-asn1crypto-src is not set
# CONFIG_PACKAGE_python-crypto is not set
# CONFIG_PACKAGE_python-crypto-src is not set
# CONFIG_PACKAGE_python-cryptodome is not set
# CONFIG_PACKAGE_python-cryptodome-src is not set
# CONFIG_PACKAGE_python-cryptodomex is not set
# CONFIG_PACKAGE_python-cryptodomex-src is not set
# CONFIG_PACKAGE_python-cryptography is not set
# CONFIG_PACKAGE_python-cryptography-src is not set
# CONFIG_PACKAGE_python-openssl is not set
# CONFIG_PACKAGE_python-openssl-src is not set
# CONFIG_PACKAGE_python-pyopenssl is not set
# CONFIG_PACKAGE_python-pyopenssl-src is not set
# CONFIG_PACKAGE_python3-asn1crypto is not set
# CONFIG_PACKAGE_python3-asn1crypto-src is not set
# CONFIG_PACKAGE_python3-crypto is not set
# CONFIG_PACKAGE_python3-crypto-src is not set
# CONFIG_PACKAGE_python3-cryptodome is not set
# CONFIG_PACKAGE_python3-cryptodome-src is not set
# CONFIG_PACKAGE_python3-cryptodomex is not set
# CONFIG_PACKAGE_python3-cryptodomex-src is not set
# CONFIG_PACKAGE_python3-cryptography is not set
# CONFIG_PACKAGE_python3-cryptography-src is not set
# CONFIG_PACKAGE_python3-openssl is not set
# CONFIG_PACKAGE_python3-openssl-src is not set
# CONFIG_PACKAGE_python3-pyopenssl is not set
# CONFIG_PACKAGE_python3-pyopenssl-src is not set
# CONFIG_PACKAGE_libelektra-crypto is not set
# CONFIG_PACKAGE_libuhttpd-openssl is not set
# CONFIG_PACKAGE_libuwsc-openssl is not set
CONFIG_PACKAGE_libopenssl=m
CONFIG_OPENSSL_OPTIMIZE_SPEED=y
CONFIG_OPENSSL_WITH_ASM=y
# CONFIG_OPENSSL_WITH_DEPRECATED is not set
CONFIG_OPENSSL_NO_DEPRECATED=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_TLS13=y
# CONFIG_OPENSSL_WITH_DTLS is not set
# CONFIG_OPENSSL_WITH_NPN is not set
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_CMS=y
# CONFIG_OPENSSL_WITH_EC2M is not set
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
CONFIG_OPENSSL_PREFER_CHACHA_OVER_GCM=y
CONFIG_OPENSSL_WITH_PSK=y
# CONFIG_OPENSSL_WITH_ARIA is not set
# CONFIG_OPENSSL_WITH_CAMELLIA is not set
# CONFIG_OPENSSL_WITH_IDEA is not set
# CONFIG_OPENSSL_WITH_SEED is not set
# CONFIG_OPENSSL_WITH_SM234 is not set
# CONFIG_OPENSSL_WITH_BLAKE2 is not set
# CONFIG_OPENSSL_WITH_MDC2 is not set
# CONFIG_OPENSSL_WITH_WHIRLPOOL is not set
# CONFIG_OPENSSL_WITH_COMPRESSION is not set
# CONFIG_OPENSSL_WITH_RFC3779 is not set
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_ENGINE_BUILTIN=y
CONFIG_OPENSSL_ENGINE_BUILTIN_AFALG=y
CONFIG_OPENSSL_ENGINE_BUILTIN_DEVCRYPTO=y
# CONFIG_OPENSSL_WITH_GOST is not set
CONFIG_PACKAGE_libopenssl-conf=m
# CONFIG_PACKAGE_libarchive-noopenssl is not set
# CONFIG_PACKAGE_libevent2-openssl is not set
CONFIG_PACKAGE_libustream-openssl=m
# CONFIG_PACKAGE_libwebsockets-openssl is not set
CONFIG_PACKAGE_luci-ssl-openssl=m
# CONFIG_PACKAGE_luci-lib-nixio_openssl is not set
# CONFIG_PACKAGE_transmission-cli-openssl is not set
# CONFIG_PACKAGE_transmission-daemon-openssl is not set
# CONFIG_PACKAGE_transmission-remote-openssl is not set
# CONFIG_PACKAGE_wifidog-ng-openssl is not set
CONFIG_DNSDIST_OPENSSL=y
# CONFIG_PACKAGE_umurmur-openssl is not set
# CONFIG_PACKAGE_openvpn-openssl is not set
CONFIG_PACKAGE_strongswan-mod-openssl=m
# CONFIG_PACKAGE_eapol-test-openssl is not set
# CONFIG_PACKAGE_hnetd-openssl is not set
# CONFIG_PACKAGE_hostapd-openssl is not set
# CONFIG_PACKAGE_wpa-supplicant-mesh-openssl is not set
# CONFIG_PACKAGE_wpa-supplicant-openssl is not set
# CONFIG_PACKAGE_wpad-mesh-openssl is not set
CONFIG_PACKAGE_wpad-openssl=m
# CONFIG_PACKAGE_shairport-sync-openssl is not set
# CONFIG_PACKAGE_bsdtar-noopenssl is not set
# CONFIG_PACKAGE_rtty-openssl is not set
CONFIG_PACKAGE_openssl-util=m

IPQ806x needs a different driver for the HW crypto. @cotequeiroz's patch is only for IPQ40xx.

1 Like

EA8300 (IPQ4019)

aca7542e9a (HEAD -> jmk-ea8300) kernel: fix hw-crypto detection of qce driver
a28ff57c73 JMK: Squashed build-system changes through June, 27, 2019
c5b10c8282 (openwrt/master, openwrt/HEAD, master) kernel: bump 4.19 to 4.19.74

gives

root@OpenWrt:~# openssl engine -pre DUMP_INFO devcrypto 
(devcrypto) /dev/crypto engine
Information about ciphers supported by the /dev/crypto engine:
Cipher DES-CBC, NID=31, /dev/crypto info: id=1, driver=cbc-des-qce (hw accelerated)
Cipher DES-EDE3-CBC, NID=44, /dev/crypto info: id=2, driver=cbc-3des-qce (hw accelerated)
Cipher BF-CBC, NID=91, /dev/crypto info: id=3, CIOCGSESSION (session open call) failed
Cipher CAST5-CBC, NID=108, /dev/crypto info: id=4, CIOCGSESSION (session open call) failed
Cipher AES-128-CBC, NID=419, /dev/crypto info: id=11, driver=cbc-aes-qce (hw accelerated)
Cipher AES-192-CBC, NID=423, /dev/crypto info: id=11, driver=cbc-aes-qce (hw accelerated)
Cipher AES-256-CBC, NID=427, /dev/crypto info: id=11, driver=cbc-aes-qce (hw accelerated)
Cipher RC4, NID=5, /dev/crypto info: id=12, CIOCGSESSION (session open call) failed
Cipher AES-128-CTR, NID=904, /dev/crypto info: id=21, driver=ctr-aes-qce (hw accelerated)
Cipher AES-192-CTR, NID=905, /dev/crypto info: id=21, driver=ctr-aes-qce (hw accelerated)
Cipher AES-256-CTR, NID=906, /dev/crypto info: id=21, driver=ctr-aes-qce (hw accelerated)
Cipher AES-128-ECB, NID=418, /dev/crypto info: id=23, driver=ecb-aes-qce (hw accelerated)
Cipher AES-192-ECB, NID=422, /dev/crypto info: id=23, driver=ecb-aes-qce (hw accelerated)
Cipher AES-256-ECB, NID=426, /dev/crypto info: id=23, driver=ecb-aes-qce (hw accelerated)

Information about digests supported by the /dev/crypto engine:
Digest MD5, NID=4, /dev/crypto info: id=13, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA1, NID=64, /dev/crypto info: id=14, driver=sha1-qce (hw accelerated), CIOCCPHASH capable
Digest RIPEMD160, NID=117, /dev/crypto info: id=102, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA224, NID=675, /dev/crypto info: id=103, driver=sha224-generic (software), CIOCCPHASH capable
Digest SHA256, NID=672, /dev/crypto info: id=104, driver=sha256-qce (hw accelerated), CIOCCPHASH capable
Digest SHA384, NID=673, /dev/crypto info: id=105, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA512, NID=674, /dev/crypto info: id=106, driver=unknown. CIOCGSESSION (session open) failed

[Success]: DUMP_INFO

Edit: If I remove the USE_SOFTDRIVERS=1 line, I additionally get in the output

3069633892:error:260AC089:engine routines:int_ctrl_helper:invalid cmd name:crypto/engine/eng_ctrl.c:87:
3069633892:error:260AB089:engine routines:ENGINE_ctrl_cmd_string:invalid cmd name:crypto/engine/eng_ctrl.c:255:
3069633892:error:260BC066:engine routines:int_engine_configure:engine configuration error:crypto/engine/eng_cnf.c:141:section=devcrypto, name=HOME, value=.
3069633892:error:0E07606D:configuration file routines:module_run:module initialization error:crypto/conf/conf_mod.c:177:module=engines, value=engines, retcode=-1 
root@OpenWrt:~# diff -u /rom/etc/ssl/openssl.cnf /etc/ssl/openssl.cnf 
--- /rom/etc/ssl/openssl.cnf	2019-09-20 23:39:07.000000000 +0000
+++ /etc/ssl/openssl.cnf	2019-09-21 01:56:01.000000000 +0000
@@ -1,3 +1,14 @@
+openssl_conf=openssl_conf
+
+[openssl_conf]
+engines=engines
+
+[engines]
+devcrypto=devcrypto
+
+[devcrypto]
+CIPHERS=DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC
+DIGESTS=NONE
 #
 # OpenSSL example configuration file.
 # This is mostly being used for generation of certificate requests.

What do I need to do to have OpenSSL and OpenVPN take advantage of this?

At least from what I can tell, openssl speed aes-256-cb and testing of OpenVPN throughput is pretty much the same with or without the patch in place.

/etc/ssl/openssl.cnf as given above. diffconfig:

CONFIG_TARGET_ipq40xx=y
CONFIG_TARGET_ipq40xx_generic=y
CONFIG_TARGET_ipq40xx_generic_DEVICE_linksys_ea8300=y
CONFIG_DEVEL=y
CONFIG_BUSYBOX_CUSTOM=y
CONFIG_BUILD_LOG=y
CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVEHISTORY=y
CONFIG_BUSYBOX_CONFIG_FEATURE_REVERSE_SEARCH=y
CONFIG_BUSYBOX_CONFIG_FEATURE_VERBOSE_CP_MESSAGE=y
CONFIG_CCACHE=y
CONFIG_DOWNLOAD_FOLDER="/home/jeff/devel/openwrt_dl"
CONFIG_ETHTOOL_PRETTY_DUMP=y
CONFIG_NGINX_HEADERS_MORE=y
CONFIG_NGINX_HTTP_ACCESS=y
CONFIG_NGINX_HTTP_AUTH_BASIC=y
CONFIG_NGINX_HTTP_AUTOINDEX=y
CONFIG_NGINX_HTTP_BROWSER=y
CONFIG_NGINX_HTTP_CACHE=y
CONFIG_NGINX_HTTP_CHARSET=y
CONFIG_NGINX_HTTP_EMPTY_GIF=y
CONFIG_NGINX_HTTP_FASTCGI=y
CONFIG_NGINX_HTTP_GEO=y
CONFIG_NGINX_HTTP_GZIP=y
CONFIG_NGINX_HTTP_LIMIT_CONN=y
CONFIG_NGINX_HTTP_LIMIT_REQ=y
CONFIG_NGINX_HTTP_MAP=y
CONFIG_NGINX_HTTP_MEMCACHED=y
CONFIG_NGINX_HTTP_PROXY=y
CONFIG_NGINX_HTTP_REFERER=y
CONFIG_NGINX_HTTP_REWRITE=y
CONFIG_NGINX_HTTP_SCGI=y
CONFIG_NGINX_HTTP_SPLIT_CLIENTS=y
CONFIG_NGINX_HTTP_SSI=y
CONFIG_NGINX_HTTP_UPSTREAM_HASH=y
CONFIG_NGINX_HTTP_UPSTREAM_IP_HASH=y
CONFIG_NGINX_HTTP_UPSTREAM_KEEPALIVE=y
CONFIG_NGINX_HTTP_UPSTREAM_LEAST_CONN=y
CONFIG_NGINX_HTTP_USERID=y
CONFIG_NGINX_HTTP_UWSGI=y
CONFIG_NGINX_NAXSI=y
CONFIG_NGINX_PCRE=y
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_PREFER_CHACHA_OVER_GCM=y
CONFIG_OPENSSL_WITH_ASM=y
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
CONFIG_OPENSSL_WITH_CMS=y
CONFIG_OPENSSL_WITH_DEPRECATED=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_PSK=y
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_TLS13=y
CONFIG_OPENVPN_openssl_ENABLE_DEF_AUTH=y
CONFIG_OPENVPN_openssl_ENABLE_FRAGMENT=y
CONFIG_OPENVPN_openssl_ENABLE_LZ4=y
CONFIG_OPENVPN_openssl_ENABLE_LZO=y
CONFIG_OPENVPN_openssl_ENABLE_MULTIHOME=y
CONFIG_OPENVPN_openssl_ENABLE_PF=y
CONFIG_OPENVPN_openssl_ENABLE_PORT_SHARE=y
CONFIG_OPENVPN_openssl_ENABLE_SERVER=y
CONFIG_OPENVPN_openssl_ENABLE_SMALL=y
CONFIG_PACKAGE_build-details=y
CONFIG_PACKAGE_ca-bundle=y
CONFIG_PACKAGE_cgi-io=y
CONFIG_PACKAGE_diffutils=y
CONFIG_PACKAGE_ethtool=y
CONFIG_PACKAGE_findutils=y
CONFIG_PACKAGE_findutils-find=y
CONFIG_PACKAGE_findutils-locate=y
CONFIG_PACKAGE_findutils-xargs=y
CONFIG_PACKAGE_git=y
CONFIG_PACKAGE_htop=y
CONFIG_PACKAGE_i2c-tools=y
CONFIG_PACKAGE_ip-bridge=y
CONFIG_PACKAGE_ip-full=y
CONFIG_PACKAGE_iperf3=y
CONFIG_PACKAGE_iptables-mod-conntrack-extra=y
CONFIG_PACKAGE_iptables-mod-ipopt=y
CONFIG_PACKAGE_irqbalance=y
CONFIG_PACKAGE_jansson=y
CONFIG_PACKAGE_kmod-crypto-aead=y
CONFIG_PACKAGE_kmod-crypto-authenc=y
CONFIG_PACKAGE_kmod-crypto-crc32c=y
CONFIG_PACKAGE_kmod-crypto-hash=y
CONFIG_PACKAGE_kmod-crypto-manager=y
CONFIG_PACKAGE_kmod-crypto-null=y
CONFIG_PACKAGE_kmod-crypto-pcompress=y
CONFIG_PACKAGE_kmod-cryptodev=y
CONFIG_PACKAGE_kmod-fs-ext4=y
CONFIG_PACKAGE_kmod-ifb=y
CONFIG_PACKAGE_kmod-ipt-conntrack-extra=y
CONFIG_PACKAGE_kmod-ipt-ipopt=y
CONFIG_PACKAGE_kmod-ipt-raw=y
CONFIG_PACKAGE_kmod-lib-crc16=y
CONFIG_PACKAGE_kmod-sched-cake=y
CONFIG_PACKAGE_kmod-sched-core=y
CONFIG_PACKAGE_kmod-tun=y
CONFIG_PACKAGE_kmod-udptunnel4=y
CONFIG_PACKAGE_kmod-udptunnel6=y
CONFIG_PACKAGE_kmod-wireguard=y
CONFIG_PACKAGE_less=y
CONFIG_PACKAGE_libacl=y
CONFIG_PACKAGE_libattr=y
CONFIG_PACKAGE_libcap=y
CONFIG_PACKAGE_libelf=y
CONFIG_PACKAGE_libi2c=y
CONFIG_PACKAGE_libiwinfo-lua=y
CONFIG_PACKAGE_liblua=y
CONFIG_PACKAGE_liblucihttp=y
CONFIG_PACKAGE_liblucihttp-lua=y
CONFIG_PACKAGE_liblzo=y
CONFIG_PACKAGE_libmnl=y
CONFIG_PACKAGE_libncurses=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libopenssl-conf=y
CONFIG_PACKAGE_libopenssl-devcrypto=y
CONFIG_PACKAGE_libpcap=y
CONFIG_PACKAGE_libpcre=y
CONFIG_PACKAGE_libpopt=y
CONFIG_PACKAGE_librt=y
CONFIG_PACKAGE_libubus-lua=y
CONFIG_PACKAGE_libusb-1.0=y
CONFIG_PACKAGE_libuuid=y
CONFIG_PACKAGE_lua=y
CONFIG_PACKAGE_luci-app-firewall=y
CONFIG_PACKAGE_luci-app-openvpn=y
CONFIG_PACKAGE_luci-app-sqm=y
CONFIG_PACKAGE_luci-app-wireguard=y
CONFIG_PACKAGE_luci-base=y
CONFIG_PACKAGE_luci-lib-ip=y
CONFIG_PACKAGE_luci-lib-jsonc=y
CONFIG_PACKAGE_luci-lib-nixio=y
CONFIG_PACKAGE_luci-mod-admin-full=y
CONFIG_PACKAGE_luci-mod-network=y
CONFIG_PACKAGE_luci-mod-status=y
CONFIG_PACKAGE_luci-mod-system=y
CONFIG_PACKAGE_luci-proto-ipv6=y
CONFIG_PACKAGE_luci-proto-ppp=y
CONFIG_PACKAGE_luci-proto-wireguard=y
CONFIG_PACKAGE_luci-ssl-nginx=y
CONFIG_PACKAGE_luci-theme-bootstrap=y
CONFIG_PACKAGE_nand-utils=y
CONFIG_PACKAGE_nginx-mod-luci-ssl=y
CONFIG_PACKAGE_nginx-ssl=y
CONFIG_PACKAGE_openssl-util=y
CONFIG_PACKAGE_openvpn-openssl=y
CONFIG_PACKAGE_procps-ng=y
CONFIG_PACKAGE_procps-ng-free=y
CONFIG_PACKAGE_procps-ng-kill=y
CONFIG_PACKAGE_procps-ng-pgrep=y
CONFIG_PACKAGE_procps-ng-pkill=y
CONFIG_PACKAGE_procps-ng-pmap=y
CONFIG_PACKAGE_procps-ng-ps=y
CONFIG_PACKAGE_procps-ng-pwdx=y
CONFIG_PACKAGE_procps-ng-skill=y
CONFIG_PACKAGE_procps-ng-slabtop=y
CONFIG_PACKAGE_procps-ng-snice=y
CONFIG_PACKAGE_procps-ng-tload=y
CONFIG_PACKAGE_procps-ng-top=y
CONFIG_PACKAGE_procps-ng-uptime=y
CONFIG_PACKAGE_procps-ng-vmstat=y
CONFIG_PACKAGE_procps-ng-w=y
CONFIG_PACKAGE_procps-ng-watch=y
CONFIG_PACKAGE_rpcd=y
CONFIG_PACKAGE_rpcd-mod-file=y
CONFIG_PACKAGE_rpcd-mod-iwinfo=y
CONFIG_PACKAGE_rpcd-mod-rrdns=y
CONFIG_PACKAGE_rsync=y
CONFIG_PACKAGE_sqm-scripts=y
CONFIG_PACKAGE_tc=y
CONFIG_PACKAGE_tcpdump-mini=y
CONFIG_PACKAGE_terminfo=y
CONFIG_PACKAGE_usbutils=y
CONFIG_PACKAGE_uwsgi-cgi=y
CONFIG_PACKAGE_uwsgi-cgi-luci-support=y
CONFIG_PACKAGE_wireguard=y
CONFIG_PACKAGE_wireguard-tools=y
CONFIG_PACKAGE_zlib=y
CONFIG_RSYNC_acl=y
CONFIG_RSYNC_xattr=y
CONFIG_RSYNC_zlib=y

Add engine cryptodev to /etc/openvpn/client.conf.

My directions were too terse. At least the first line must be fore any session listed in []. Also the newly created sessions must not break the original unnamed session (that's what caused your error message). So an appropriate patch for the original openssl.cnf may be:

--- a/etc/ssl/openssl.cnf       2019-09-20 09:09:59.918495075 -0300
+++ b/etc/ssl/openssl.cnf       2019-09-21 15:35:27.813684608 -0300
@@ -22,6 +22,18 @@
 # (Alternatively, use a configuration file that has only
 # X.509v3 extensions in its main [= default] section.)

+openssl_conf=openssl_conf
+
+[openssl_conf]
+engines=engines
+
+[engines]
+devcrypto=devcrypto
+
+[devcrypto]
+CIPHERS=DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC
+DIGESTS=NONE
+
 [ new_oids ]

 # We can add new OIDs in here for use by 'ca', 'req' and 'ts'.

Your output shows the qce driver patch worked! Thank you.
As for performance, it will vary. The openssl speed measurements will give you an idea of "best case scenario". The problem with actually using the crypto engine is that it only works with CBC ciphers. In my case it improves performance for larger blocks, so vpn would benefit from it for sure. Depending on your config, IPSEC may be handled by the kernel, in which case it will use the hw-crypto driver, and should not be troubled by context-switching overhead. I'm using strongswan, and it works that way. I imagine OpenVPN does not work that way, but I'm not experienced enough with it to guide you through it. If it uses cryptodev, then it runs in userspace, and will suffer from context switches.
As for openssl speed, it should give you different results.
Test it. First, without engines. Change the CIPHERS line to CIPHERS=none. Then confirm that there are not algorithms registered with openssl engine -c. Here's an empty list:

# openssl engine -c
(dynamic) Dynamic engine loading support
(devcrypto) /dev/crypto engine
# time -v openssl speed -elapsed -evp AES-256-CBC
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 7543255 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 2246599 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 591612 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 149572 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 18803 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 16384 size blocks: 9354 aes-256-cbc's in 3.00s
OpenSSL 1.1.1d  10 Sep 2019
built on: Fri Sep  6 13:29:54 2019 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: arm-openwrt-linux-muslgnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -pipe -g3 -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -mfloat-abi=hard -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -O3 -fpic -ffunction-sections -fdata-sections -znow -zrelro -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DZLIB_SHARED -DNDEBUG -DOPENSSL_PREFER_CHACHA_OVER_GCM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc      40230.69k    47927.45k    50484.22k    51053.91k    51344.73k    51085.31k
        Command being timed: "openssl speed -elapsed -evp AES-256-CBC"
        User time (seconds): 18.00
        System time (seconds): 0.01
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 18.10s
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 14512
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 162
        Voluntary context switches: 19
        Involuntary context switches: 301
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Notice the ~100% usage of user time.
Then, change the ciphers to enable at least the one you're going to test. I'm going with my suggestion above:

# openssl engine -c
(dynamic) Dynamic engine loading support
(devcrypto) /dev/crypto engine
 [DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC]
# time -v openssl speed -elapsed -evp AES-256-CBC
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 286951 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 281872 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 247273 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 162764 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 34669 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 16384 size blocks: 18295 aes-256-cbc's in 3.00s
OpenSSL 1.1.1d  10 Sep 2019
built on: Fri Sep  6 13:29:54 2019 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: arm-openwrt-linux-muslgnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -pipe -g3 -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -mfloat-abi=hard -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -O3 -fpic -ffunction-sections -fdata-sections -znow -zrelro -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DZLIB_SHARED -DNDEBUG -DOPENSSL_PREFER_CHACHA_OVER_GCM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc       1530.41k     6013.27k    21100.63k    55556.78k    94669.48k    99915.09k
        Command being timed: "openssl speed -elapsed -evp AES-256-CBC"
        User time (seconds): 0.08
        System time (seconds): 5.32
        Percent of CPU this job got: 29%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 18.10s
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 14576
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 161
        Voluntary context switches: 1031843
        Involuntary context switches: 32
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Notice the shift from user to system time, and the reduction in CPU usage. This is one benefit you'll have besides the throughput. Notice the poor performance with small blocks, so SSL performance is not going to be great, unless you're mostly downloading large files. This is just a theoretical benchmark. If you have means to measure your actual software, it will give you a real answer.

1 Like

I believe things changed along the way:

root@bsaedgy:/etc/openvpn# openvpn --show-engines
OpenSSL Crypto Engines

Dynamic engine loading support [dynamic]
/dev/crypto engine [devcrypto]
AF_ALG engine [afalg]

1 Like