Letsencrypt acme certs (via dns api) won't renew/work with uhttpd

This was working for a long time but now uhttpd fails to load cert/key:

root@apu:~# /etc/init.d/uhttpd start
root@apu:~# logread -e uhttpd
Tue Sep 10 16:22:57 2019 daemon.err uhttpd[10366]: Failed to load certificate/key files

I have acme, acme-dns and luci-app-acme installed, it was working before, but now fails with Challenge error: {"type":"urn:acme:error:malformed","detail":"Expired authorization","status": 404}

acme debug log:

root@apu:~# /etc/init.d/acme restart
root@apu:~# logread -e acme
Tue Sep 10 16:23:14 2019 daemon.info acme: Running ACME for apu.lan.wrtpoona.in
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: acme: Running ACME for apu.lan.wrtpoona.in
Tue Sep 10 16:23:14 2019 daemon.info acme: Found previous cert config. Issuing renew.
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: acme: Found previous cert config. Issuing renew.
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: Lets find script dir.
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _SCRIPT_='/usr/lib/acme/acme.sh'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _script='/usr/lib/acme/acme.sh'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _script_home='/usr/lib/acme'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: Using config home:/etc/acme
Tue Sep 10 16:23:14 2019 daemon.info run-acme[10393]: https://github.com/Neilpang/acme.sh
Tue Sep 10 16:23:14 2019 daemon.info run-acme[10393]: v2.7.8
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: Using config home:/etc/acme
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: ACME_DIRECTORY='https://acme-v01.api.letsencrypt.org/directory'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: DOMAIN_PATH='/etc/acme/apu.lan.wrtpoona.in'
Tue Sep 10 16:23:14 2019 daemon.info run-acme[10393]: Renew: 'apu.lan.wrtpoona.in'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: Le_API='https://acme-v01.api.letsencrypt.org/directory'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: Using config home:/etc/acme
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: ACME_DIRECTORY='https://acme-v01.api.letsencrypt.org/directory'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _main_domain='apu.lan.wrtpoona.in'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _alt_domains='no'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: Using ACME_DIRECTORY: https://acme-v01.api.letsencrypt.org/directory
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _init api for server: https://acme-v01.api.letsencrypt.org/directory
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: GET
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: url='https://acme-v01.api.letsencrypt.org/directory'
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: timeout=
Tue Sep 10 16:23:14 2019 daemon.err run-acme[10393]: _CURL='curl -L --silent --dump-header /etc/acme/http.header  -g '
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ret='0'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_KEY_CHANGE='https://acme-v01.api.letsencrypt.org/acme/key-change'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_NEW_AUTHZ='https://acme-v01.api.letsencrypt.org/acme/new-authz'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_NEW_ORDER='https://acme-v01.api.letsencrypt.org/acme/new-cert'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_NEW_ACCOUNT='https://acme-v01.api.letsencrypt.org/acme/new-reg'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_REVOKE_CERT='https://acme-v01.api.letsencrypt.org/acme/revoke-cert'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_NEW_NONCE
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ACME_VERSION
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: Le_NextRenewTime='1565327446'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _on_before_issue
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _chk_main_domain='apu.lan.wrtpoona.in'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _chk_alt_domains
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: Le_LocalAddress
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: d='apu.lan.wrtpoona.in'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: Check for domain='apu.lan.wrtpoona.in'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _currentRoot='dns_he'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: d
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _saved_account_key_hash is not changed, skip register account.
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: Read key length:2048
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _createcsr
Tue Sep 10 16:23:15 2019 daemon.info run-acme[10393]: Single domain='apu.lan.wrtpoona.in'
Tue Sep 10 16:23:15 2019 daemon.info run-acme[10393]: Getting domain auth token for each domain
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: ok, let's start to verify
Tue Sep 10 16:23:15 2019 daemon.info run-acme[10393]: Verifying:apu.lan.wrtpoona.in
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: d='apu.lan.wrtpoona.in'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: keyauthorization='-xxxx'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: uri='https://acme-v01.api.letsencrypt.org/acme/challenge/xxxx'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _currentRoot='dns_he'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: url='https://acme-v01.api.letsencrypt.org/acme/challenge/xxxx'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: payload='{"resource": "challenge", "keyAuthorization": "-xxxx"}'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: RSA key
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: GET
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: url='https://acme-v01.api.letsencrypt.org/directory'
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: timeout=
Tue Sep 10 16:23:15 2019 daemon.err run-acme[10393]: _CURL='curl -L --silent --dump-header /etc/acme/http.header  -g '
Tue Sep 10 16:23:16 2019 daemon.err run-acme[10393]: ret='0'
Tue Sep 10 16:23:16 2019 daemon.err run-acme[10393]: POST
Tue Sep 10 16:23:16 2019 daemon.err run-acme[10393]: _post_url='https://acme-v01.api.letsencrypt.org/acme/challenge/xxxx'
Tue Sep 10 16:23:16 2019 daemon.err run-acme[10393]: _CURL='curl -L --silent --dump-header /etc/acme/http.header  -g '
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: _ret='0'
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: code='404'
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: apu.lan.wrtpoona.in:Challenge error: {"type":"urn:acme:error:malformed","detail":"Expired authorization","status": 404}
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: Skip for removelevel:
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: pid
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: No need to restore nginx, skip.
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: _clearupdns
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: skip dns.
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: _on_issue_err
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: Please add '--debug' or '--log' to check more details.
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: See: https://github.com/Neilpang/acme.sh/wiki/How-to-debug-acme.sh
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: url='https://acme-v01.api.letsencrypt.org/acme/challenge/xxxx'
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: payload='{"resource": "challenge", "keyAuthorization": "-BSPaOcPECC846o5coAJfS33jxdLzg3h-8RfJEjeG6U.gOcOju5yxBXkolYzVRnmn3Ao73bEv01_SbIA6n18094"}'
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: POST
Tue Sep 10 16:23:17 2019 daemon.err run-acme[10393]: _post_url='https://acme-v01.api.letsencrypt.org/acme/challenge/xxxx'
Tue Sep 10 16:23:18 2019 daemon.err run-acme[10393]: _CURL='curl -L --silent --dump-header /etc/acme/http.header  -g '
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: _ret='0'
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: code='404'
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: socat doesn't exists.
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: Diagnosis versions:
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: openssl:openssl
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: OpenSSL 1.0.2s  28 May 2019
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: apache:
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: apache doesn't exists.
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: nginx:
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: nginx doesn't exists.
Tue Sep 10 16:23:19 2019 daemon.err run-acme[10393]: socat:

acme and uhttpd config:

root@apu:~# cat /etc/config/acme

config acme
	option state_dir '/etc/acme'
	option account_email 'xxxx@mail.com'
	option debug '1'

config cert 'example'
	option keylength '2048'
	option update_uhttpd '1'
	option webroot '/www'
	option dns 'dns_he'
	list credentials 'HE_Username="xxxxx"'
	list credentials 'HE_Password="xxxxx"'
	option use_staging '1'
	option enabled '0'
	list domains 'apu.lan.wrtpoona.in'

config cert 'APU'
	option enabled '1'
	option keylength '2048'
	option update_uhttpd '1'
	option webroot '/www'
	option use_staging '0'
	list credentials 'HE_Username="xxxx"'
	list credentials 'HE_Password="xxxxx"'
	option dns 'dns_he'
	list domains 'apu.lan.wrtpoona.in'

root@apu:~# cat /etc/config/uhttpd

config uhttpd 'main'
	list listen_http '0.0.0.0:80'
	list listen_http '[::]:80'
	list listen_https '0.0.0.0:443'
	list listen_https '[::]:443'
	option home '/www'
	option max_requests '3'
	option max_connections '100'
	option cgi_prefix '/cgi-bin'
	list lua_prefix '/cgi-bin/luci=/usr/lib/lua/luci/sgi/uhttpd.lua'
	option script_timeout '60'
	option network_timeout '30'
	option http_keepalive '20'
	option tcp_keepalive '1'
	option cert '/etc/acme/apu.lan.wrtpoona.in/apu.lan.wrtpoona.in.cer'
	option key '/etc/acme/apu.lan.wrtpoona.in/apu.lan.wrtpoona.in.key'
	option redirect_https '0'
	option rfc1918_filter '0'

config cert 'defaults'
	option days '730'
	option bits '2048'
	option country 'ZZ'
	option state 'Somewhere'
	option location 'Unknown'
	option commonname 'OpenWrt'

Letsencrypt (using same methodology as on working system) today informed me that V1 protocol no longer supported, need V2

latest acme.sh: https://github.com/Neilpang/acme.sh

still haven't got it working, but pretty sure V2 is part of issue.

I upgraded to 18.06.05 and it's working.