Davidc502- wrt1200ac wrt1900acx wrt3200acm wrt32x builds


#1760

Six days ago this commit was merged that had changed around some things with devcrypto.

@cotequeiroz can you give any insight into what could be causing this:

Wed Feb 27 08:59:21 2019 kern.err kernel: [16077.021990] marvell-cesa f1090000.crypto: dma_pool_free cesa_cache, dma 0x1cff6040 already free

Could it be caused by an outdated OpenVPN version (2.4.6 was released pre-OpenSSL 1.1.1/TLS1.3)? I can see that 2.4.7 was release just on 21st Feb with better support for OpenSSL 1.1.1 (and probably 1.1.1b).


#1761

I'm investigating issues with digest support and forks/threads. I haven't confirmed this, but it seems like there's a problem with processes that copy a message digest context from one process (or thread) to another one. sshd is the one I'm currently looking into. It seems to be creating two instances of the /dev/crypto engine, and then copying an MD context from one instance to the other one, causing an ENOENT error, although I haven't seen those kernel error messages yet, but I haven't run openvpn. Try to apply https://patchwork.ozlabs.org/patch/1049156/ (the patch adds, among other goodies, the configuration options I'll use next) and then disable digests in /etc/openssl.cnf: Add this line before any section (lines with brackets []):

openssl_conf=openssl_def

Then, add the following sections anywhere past the last of the unsectioned commands (I add this to the very end of the file)--notice that you don't need the CIPHERS line to enable all ciphers, I've added it here for reference:

[openssl_def]
engines=engine_section

[engine_section]
devcrypto=devcrypto_section

[devcrypto_section]
CIPHERS=ALL
DIGESTS=NONE

Please tell me if the errors stop. Digests are not efficient anyways, at least not with the wrt3200acm (tested with openssl speed, only with a block size of 16384, they are a bit faster than software). Perhaps with VPN traffic on jumbo frames they may be worth a try, but for most TLS applications, it is very slow. You can use different configuration files, btw, setting the OPENSSL_CONF environment variable with the configuration filename.
I would disable the ECB ciphers as well, since they are used mostly in rng code, in small blocks (I don't recall if it is 16 or 64 bytes), where the cost of the ioctl (context switch) is high. I run mine with

CIPHERS=DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC

Note that, without the patch above, the engine is used for algorithms implemented in the kernel without hardware acceleration, which slows them down not matter what.
I will try to fix the copy issue, or else disable digests by default (probably best anyway).


#1762

That commit backs out patches that were added in the original 1.1.1a push, but had since been up-streamed; or, they were and are still there. I would suggest that a response from the author is more likely if you pose your question in PR1547 where the cryptodev enhancement was kicked around to debug things. But by way of a data point, I have not seen this message on either a mamba or rango with openvpn in play.


#1763

I am the author of the commits, btw. I didn't point to PR1547 because I haven't propagated the digest fix to AF_ALG yet, even though the changes in the patch I mentioned are there already (you may use https://github.com/openwrt/openwrt/pull/1547/commits/4731795a675c999093b5a8d30cecd25610a5a6f8 if you wish; or even the full PR--parallel build is good, just don't rely on the AF_ALG engine yet, or at least disable digests).


#1764

Yes, understood. It is that was the thread before the title change. I was not suggesting to pull anything from there, just that I felt a response from you was more likely, and perhaps more apropos, there than here.


#1765

FYI, openvpn is unconfigured and disabled on my system and I'm infrequently seeing the errors (about 20 of them in the last 10hrs). Although that's likely not the only package on the system making use of openssl.

edit they seem to come two or three at time, never more than 5 seconds apart :thinking:


#1766

My comment regarding openvpn was in response to the speculation that it might be the cause of the issue as stated in the post to which I was responding.

I guess I should also state that I have a current @cotequeiroz cryptodev patch from patchworks in my image.


#1767

Please try to run (to increase devcrypto verbosity):

sysctl ioctl.cryptodev_verbosity=1

And see if there are any cryptodev messages close to the other error. Messages about failure to load or bad ciphers/hash/transforms, especially ciphers 4 & 12/blowfish/rmd160, are part of algorithm detection, and are normal.


#1768

Okay, did that, and there's nothing in the logs similar to what you suggested. Although I have found that the errors are easily reproducible by accessing LuCi via https.


#1769

Good observation... I didn't notice, but haven't had any of those errors until I logged into LuCi just now.


#1770

I'm still running my own builds based off of the 18.06 branch so I won't be able to apply these patches given that I'm still on 1.0.2r. Firstly, I just wanted to help direct the users of @davidc502's build facing SSL errors to you, the principal author of the recent OpenSSL patches. Secondly, to get more users to test and get your fixes/enhancements in before Hauke's planned branch of OpenWRT 19.03 between the 7th & 17th March.

I do agree that hardware digests are slower. It is corroborated by this.


#1771

Dear Dave,
I first say - Hello. On this last Build r9506 - OpenVpn would not start for me. Most significantly, I could not get the OpenVpn interface to properly function. The interface - I tried both uci and Luci - and it said there was an " error " or something to that effect. I fell back to r9287 and everything worked as it should. I use OpenVpn as a NORDVPN client.
I really have not seen anyone raise this issue in this thread, so I just figured that I would relate to you my finding regarding this issue.
Anyway, thanks for all the great work you do. If I am doing anything wrong - I hope that someone will let me know.
I setup my VPN connection as described in these posts:

and here :slight_smile:

PIA OPENVPN on OpenWrt / Lede

For NORDVPN I use this guide with Interface and Firewall Creation from Guides listed above - a sort of hybrid setup :slight_smile:

http://stitchroads.blogspot.com/2018/08/how-to-setup-nordvpn-openvpn-on.html

Peace and well Wishes,

directnupe


#1772

I am also getting these crypto erros with r9506 on my wrt1900acs v2. Additionally, my VPN connection is so slow that I can barely use it...

Is there a proposed fix yet?


#1773

I use Torguard, but I mainly just use the client when I need it. However, I do have my router setup for OpenVPN to be used with Tourguard for testing. I haven't used it in a while, but I just checked and it fired up without issue. Currently I have a 600/400mbps connection, and with OpenVPN, I'm getting about 80/80mbps which seems pretty typical for me with tourguard.

Let me know if you would like to see my configuration, and maybe that will help you with yours.

config openvpn 'TorGuard_AES128CBC_SHA256'
	option client '1'
	option resolv_retry 'infinite'
	option nobind '1'
	option persist_key '1'
	option persist_tun '1'
	option ca '/etc/openvpn/torguard/ca.crt'
	option ns_cert_type 'server'
	option tls_auth '/etc/openvpn/torguard/ta.key 1'
	option cipher 'AES-128-CBC'
	option verb '3'
	option fast_io '1'
	option auth_user_pass '/etc/openvpn/auth-user-pass'
	option auth 'SHA256'
	option reneg_sec '0'
	list remote 'chi.central.usa.torguardvpnaccess.com 1912'
	option sndbuf '393216'
	option rcvbuf '393216'
	option log '/tmp/openvpn.log'
	option proto 'udp'
	option compress 'lzo'
	option dev 'tun'
	option engine 'dynamic'

#1774

I'm finished investigating a bug involving openssh, that resulted in this: engines and forks are not getting along openssl/openssl#8430. TLDR so far: engine support (& hw crypto) is not plug & play; it won't work with all applications.

Expanding the supported algorithms increases the chances of catching an incompatibility. So, we need to disable the ones causing trouble & slowdown. My recommendation is to leave only the CBC ciphers enabled, and disable everything else.

I'm going to tackle the kernel errors now. As soon as I have news, I'll post them here.


#1775

@davidc502, @WiteWulf
I need some info here. I'm using plain openwrt here, and I'm unable to reproduce the kernel error by using luci. I've tried it with either uhttpd, and nginx. I'm not familiar with Dave's build, so bear with my noobness. Are there any significant changes from plain openwrt that I should be aware of, patches to apply? Or perhaps ciphersuite/browser to use?


#1776

Out of curiosity, has anyone recently compared mbed TLS and OpenSSL as far as performance when using OpenVPN? I did it a while back on MIPS and found mbed TLS to be faster but MIPS doesn't have hardware acceleration.


#1777

@cotequeiroz

LuCi Rosy is custom, so that may play into the problem? I can check tonight where I will remove Rosy and try Material or base LuCi.

Or maybe you can use the Rosy sources to check Rosy out on your build? There are 2 github sources...Will check to see which one I am pointed to for the build.

Currently no other patches except with a wifi hack I do for AMSDU, but shouldn't be associated.


#1778

Yes, this is happening no matter the theme chosen. As for Rosy, I used to have a custom feed, but was no longer needed and I commented it out.

So, I'm thinking this must have something to do with the packages chosen.

EDIT

So, I may have to go back with a clean slate (new .conf), and see how a new build acts.


#1779

Okay, a little more testing here:

  • Safari 12.0.2 on macOS 10.14.2 - no errors

  • Chrome 72.0.3626.121 on macOS 10.14.2 - errors

  • Safari on iOS 12.1.4 (iPhone 7) - no errors

  • Chrome 72.0.3626.101 on iOS 12.14 (iPhone 7) - errors

  • Firefox 65.0.1 on macOS 10.14.2 - errors (also complains about SEC_ERROR_UNKNOWN_ISSUER for my letsencrypt cert)

  • Brave 0.60.48 on macOS 10.14.2 - errors

  • Chrome (unknown version) on FireOS 6.3.0.1 (Fire HD8) - errors

I'm running luci on uhttpd with the standard settings in David's build and a letsencrypt SSL certificate. The router is also dual-stacked with a Hurricane Electric 6in4 tunnel and uhttpd is bound to both IPv4 and IPv6.

Hope that helps, please don't hesitate to ask if there's anything else I can do.

edit just realised I had a Windows box to hand, too:

  • Edge 42.17134.1.0 on Windows 10 - errors
  • Internet Explorer 11.590.1734.0 on Windows 10 - no errors