Ngnix with devcrypto HW crypto accelerator

Psychesnet · April 9, 2021, 12:26am

Hi all,

We are working on openwrt-19.07.7 branch and our device support HW crypto accelerator.

driver version is cryptodev-linux-1.10 and enable devcrypto support on openssl.
openssl engine -t -c

(dynamic) Dynamic engine loading support
     [ unavailable ]
(devcrypto) /dev/crypto engine
 [AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-128-CTR, AES-192-CTR, AES-256-CTR, AES-128-ECB, AES-192-ECB, AES-256-ECB]
     [ available ]

openssl engine -pre DUMP_INFO devcrypto

(devcrypto) /dev/crypto engine
Information about ciphers supported by the /dev/crypto engine:
Cipher DES-CBC, NID=31, /dev/crypto info: id=1, driver=cbc(des-generic) (software)
Cipher DES-EDE3-CBC, NID=44, /dev/crypto info: id=2, driver=cbc(des3_ede-generic) (software)
Cipher BF-CBC, NID=91, /dev/crypto info: id=3, CIOCGSESSION (session open call) failed
Cipher CAST5-CBC, NID=108, /dev/crypto info: id=4, CIOCGSESSION (session open call) failed
Cipher AES-128-CBC, NID=419, /dev/crypto info: id=11, driver=cbc-aes-infinity (hw accelerated)
Cipher AES-192-CBC, NID=423, /dev/crypto info: id=11, driver=cbc-aes-infinity (hw accelerated)
Cipher AES-256-CBC, NID=427, /dev/crypto info: id=11, driver=cbc-aes-infinity (hw accelerated)
Cipher RC4, NID=5, /dev/crypto info: id=12, CIOCGSESSION (session open call) failed
Cipher AES-128-CTR, NID=904, /dev/crypto info: id=21, driver=ctr-aes-infinity (hw accelerated)
Cipher AES-192-CTR, NID=905, /dev/crypto info: id=21, driver=ctr-aes-infinity (hw accelerated)
Cipher AES-256-CTR, NID=906, /dev/crypto info: id=21, driver=ctr-aes-infinity (hw accelerated)
Cipher AES-128-ECB, NID=418, /dev/crypto info: id=23, driver=ecb-aes-infinity (hw accelerated)
Cipher AES-192-ECB, NID=422, /dev/crypto info: id=23, driver=ecb-aes-infinity (hw accelerated)
Cipher AES-256-ECB, NID=426, /dev/crypto info: id=23, driver=ecb-aes-infinity (hw accelerated)

Information about digests supported by the /dev/crypto engine:
Digest MD5, NID=4, /dev/crypto info: id=13, driver=md5-generic (software), CIOCCPHASH capable
Digest SHA1, NID=64, /dev/crypto info: id=14, driver=unknown. CIOCGSESSION (session open) failed
Digest RIPEMD160, NID=117, /dev/crypto info: id=102, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA224, NID=675, /dev/crypto info: id=103, driver=sha224-generic (software), CIOCCPHASH capable
Digest SHA256, NID=672, /dev/crypto info: id=104, driver=sha256-infinity (software), CIOCCPHASH capable
Digest SHA384, NID=673, /dev/crypto info: id=105, driver=unknown. CIOCGSESSION (session open) failed
Digest SHA512, NID=674, /dev/crypto info: id=106, driver=unknown. CIOCGSESSION (session open) failed

[Success]: DUMP_INFO

time openssl speed -evp aes-128-cbc -engine devcrypto

engine "devcrypto" set.
Doing aes-128-cbc for 3s on 16 size blocks: 188248 aes-128-cbc's in 0.10s
Doing aes-128-cbc for 3s on 64 size blocks: 175358 aes-128-cbc's in 0.09s
Doing aes-128-cbc for 3s on 256 size blocks: 137635 aes-128-cbc's in 0.03s
Doing aes-128-cbc for 3s on 1024 size blocks: 74389 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 8192 size blocks: 13721 aes-128-cbc's in 0.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 6909 aes-128-cbc's in 0.01s
OpenSSL 1.1.1i  8 Dec 2020
built on: Mon Feb 15 15:22:37 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr) 
compiler: arm-linux-gnueabihf-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -Os -pipe -march=armv7-a -fno-caller-saveT
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      30119.68k   124699.02k  1174485.33k  1904358.40k         infk 11319705.60k
real    0m 18.17s
user    0m 0.29s
sys     0m 15.84s

Question:
A. According to 2, we know what ciphers can be supported on engine, but how can I get correct cipher suite for nginx web server?
B. I tried to use "openssl ciphers -s" to show cipher suite, but the output is fully supported by engine?

# openssl ciphers -s
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES2
56-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY
1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDH
E-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE
-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SH
A:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA

C. Once I got correct cipher suite support list from devcrypto engine, how can I verify it via nginx server?

Please advise. Thanks.

Psychesnet · April 12, 2021, 12:33am

Hi all,

Anyone can help? Thanks.

jring · April 26, 2021, 2:10pm

Hi,

not really an nginx person, but you should do an openssl ciphers -V and take all ciphers which fulfill the following criteria:

Enc=AES(128|256) ... AESGCM would be more modern and secure but your accelerator only supports CBC and CTR - the latter having been ignored for TLS in favour of GCM
Kx=RSA|DH|ECDH ... RSA is old and insecure, both DH and ECDH are secure given sufficient parameter lengths. ECDH is faster.
Au=RSA|ECDSA ... the latter only if you really have an EC server cert...

and the set it via ssl_ciphers and with ssl_prefer_server_ciphers on...

The result will be a cipher suite that will not do well in ssllabs server test due to lack of modern crypto (chacha20 stream cipher or AES in Galois Counter Mode).

Joachim

Psychesnet · April 27, 2021, 12:12am

Hi jring,

Thanks for replying. But, I still have some questions need to ask, please help if you could, thanks.

Sound like HW SSL engine did not support GCM, basically, this HW SSL engine is not secure enough, right?
If we use cipher suite like this one, what Enc mode is? CBC or CTR or any kind of it is okay?

0x00,0x33 - DHE-RSA-AES128-SHA SSLv3 Kx=DH Au=RSA Enc=AES(128) Mac=SHA1

Thanks.

jring · April 27, 2021, 5:17am

Hi,

ad 1) GCM is pretty much the encryption mode of choice for block ciphers in TLS 1.2 and 1.3. So yes, if your engine can't do that (or even less likely, CHACHA20), you have the choice of hardware accelerated but not too secure crypto or secure modern crypto in software...

ad 2) Enc mode for AES is pretty much limited to CBC (Enc=AES(n) or GCM (AESGCM(n) in TLS. ECB is insecure anyways for more than one block of data and CTR has never been used for TLS.

Joachim

Psychesnet · April 27, 2021, 5:55am

Hi jring,

Thanks. I think I will try to use openssl server/client to verify cipher suite first and see how to works on my device. Thanks.

jring · April 27, 2021, 6:20am

Hi,

if you got a server running, you could probably test it with the ssllabs server test (if it is internet facing or you can make it for the test).
It will of course have a lot to say about certificates not being trusted (unless you get a cert from a well-known CA - letsencrypt should work), but in the cipher suite section it will tell you what it thinks about your cipher suite in general and each cipher...

For a good score you want at least a few ciphers flagged as green at the top of the cipher list as shown here... Those are basically ciphers with PFS (that is not using RSA as Kex) and using either AES in GCM or CHACHA20.

https://www.ssllabs.com/ssltest/analyze.html?d=forum.openwrt.org&s=2a03%3Ab0c0%3A3%3Ad0%3A0%3A0%3A168b%3A9001&hideResults=on

Joachim

Psychesnet · May 7, 2021, 5:55am

Hi jring,

I try to follow up this page to test engine more, https://www.cloudinsidr.com/content/tls-1-3-and-tls-1-2-cipher-suites-demystified-how-to-pick-your-ciphers-wisely/
But, I don't know how to dig in. If possible, please give me some tips. Thanks.

On my device, the ciphers are

root@OpenWrt:/etc/nginx# openssl ciphers
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES2
56-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:................

I try to use openssl to try software encryption first. Looks like it works.

# server side
root@OpenWrt:/etc/nginx# openssl s_server -accept 4433 -key nginx.key -cert ngin
x.cer -debug -msg -ciphersuites TLS_AES_256_GCM_SHA384
Using default temp DH parameters
ACCEPT

# client side
$ openssl s_client -connect 172.31.5.204:4433 -tls1_3 -ciphersuites TLS_AES_256_GCM_SHA384

I try to use another cipher suites to try, it is wrong.

root@OpenWrt:/etc/nginx# openssl s_server -accept 4433 -key nginx.key -cert ngin
x.cer -debug -msg -ciphersuites ECDHE-ECDSA-AES256-GCM-SHA384
Error with command: "-ciphersuites ECDHE-ECDSA-AES256-GCM-SHA384"
548029798368:error:1426E0B9:SSL routines:ciphersuite_cb:no cipher match:ssl/ssl_ciph.c:1294:

Why I cannot use ECDHE-ECDSA-AES256-GCM-SHA384 as parameter?
ECDHE-ECDSA-AES256-GCM-SHA384 is not a cipher suite?
It confuses me. Please help. Thanks.

jring · May 7, 2021, 8:27am

Hi,

the ECDSA in ECDHE-ECDSA-AES256-GCM-SHA384 means that you have to use an ECDSA server certificate. Which you don't have if you used the same one as in the working example...
Try ECDHE-RSA-AES256-GCM-SHA384 instead...

Joachim

Psychesnet · May 7, 2021, 8:33am

Hi jring,

Please check this. Thanks.

root@OpenWrt:/etc/nginx# openssl s_server -accept 4433 -key nginx.key -cert ngin
x.cer -debug -msg -ciphersuites ECDHE-RSA-AES256-GCM-SHA384
Error with command: "-ciphersuites ECDHE-RSA-AES256-GCM-SHA384"
547602675680:error:1426E0B9:SSL routines:ciphersuite_cb:no cipher match:ssl/ssl_ciph.c:1294:
root@OpenWrt:/etc/nginx# openssl ciphers -V | grep ECDHE-RSA-AES256-GCM-SHA384
          0xC0,0x30 - ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD

jring · May 7, 2021, 9:11am

Hi,

since ECDHE-RSA-AES256-GCM-SHA384 (and ECDHE-ECDSA-AES256-GCM-SHA384) is a TLS1.2 cipher, you need to use -cipher instead of -ciphersuites.

And of course leave out the -tls1_3 for the client.

Joachim

Psychesnet · May 10, 2021, 12:30am

Hi Jring,

Thanks. You are correct according to the man page of openssl. ^^

-cipher val
This allows the list of TLSv1.2 and below ciphersuites used by the server to be modified. 

-ciphersuites val
This allows the list of TLSv1.3 ciphersuites used by the server to be modified.