Letsencrypt renew stuck

Installing and Using OpenWrt
1

I'm using 23.05.2 and 23.5.0 both not renewing a letsencrypt certificate. It just gets stuck. Then dnsapi host is actually never being contacted, I checked with tcpdump.
I'm using powerdns dns api.

Here is the config:

config acme
        option debug '1'
        option account_email 'jochen@example.com'

config cert 'maincert'
        option keylength 'ec-384'
        list domains 'apukeller.example.com'
        option validation_method 'dns'
        option dns 'dns_pdns'
        list credentials 'PDNS_ServerId=localhost'
        list credentials 'PDNS_Ttl=60'
        list credentials 'PDNS_Url="https://dnsapi.example.com"'
        list credentials 'PDNS_Token="verysecret"'
        option enabled '1'
        option use_staging '0'
        option update_uhttpd '1'

Thx for helping. The same configuration used to work until recently. Other Linux hosts using dehydrated still work. The task just gets stuck for at least like 5 minutes.

Here's the debug log:

root@apukeller:~# /etc/init.d/acme restart
Command failed: Not found
acme-acmesh: Running ACME for apukeller.example.com
acme-acmesh: /usr/lib/acme/client/acme.sh --debug --renew --home /etc/acme -d apukeller.example.com
Lets find script dir.
_SCRIPT_='/usr/lib/acme/client/acme.sh'
_script='/usr/lib/acme/client/acme.sh'
_script_home='/usr/lib/acme/client'
Using config home:/etc/acme
https://github.com/acmesh-official/acme.sh
v3.0.6
Running cmd: renew
_renewServer
Using config home:/etc/acme
default_acme_server
ACME_DIRECTORY='https://acme.zerossl.com/v2/DV90'
DOMAIN_PATH='/etc/acme/apukeller.example.com'
Renew: 'apukeller.example.com'
Le_API='https://acme-v02.api.letsencrypt.org/directory'
Renew to Le_API=https://acme-v02.api.letsencrypt.org/directory
Using config home:/etc/acme
ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
_main_domain='apukeller.example.com'
_alt_domains='no'
Le_NextRenewTime='1697493632'
Using ACME_DIRECTORY: https://acme-v02.api.letsencrypt.org/directory
_init api for server: https://acme-v02.api.letsencrypt.org/directory
GET
url='https://acme-v02.api.letsencrypt.org/directory'
timeout=
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
ret='0'
ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
ACME_NEW_AUTHZ
ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf'
ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
Using CA: https://acme-v02.api.letsencrypt.org/directory
_on_before_issue
_chk_main_domain='apukeller.example.com'
_chk_alt_domains
Le_LocalAddress
d='apukeller.example.com'
Check for domain='apukeller.example.com'
_currentRoot='dns_pdns'
d
_saved_account_key_hash is not changed, skip register account.
Read key length:4096
_createcsr
Single domain='apukeller.example.com'
Getting domain auth token for each domain
d
url='https://acme-v02.api.letsencrypt.org/acme/new-order'
payload='{"identifiers": [{"type":"dns","value":"apukeller.example.com"}]}'
RSA key
HEAD
_post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g  -I  '
_ret='0'
POST
_post_url='https://acme-v02.api.letsencrypt.org/acme/new-order'
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
_ret='0'
code='201'
Le_LinkOrder='https://acme-v02.api.letsencrypt.org/acme/order/1072828657/222918725406'
Le_OrderFinalize='https://acme-v02.api.letsencrypt.org/acme/finalize/1072828657/222918725406'
url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/284817054046'
payload
POST
_post_url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/284817054046'
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
_ret='0'
code='200'
d='apukeller.example.com'
Getting webroot for domain='apukeller.example.com'
_w='dns_pdns'
_currentRoot='dns_pdns'
entry='"type":"dns-01","status":"pending","url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA","token":"Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI"'
token='Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI'
uri='https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA'
keyauthorization='Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI.4JPgXSJympl9pU_3o6LrwNtopFBCHxS9i2GnWGdJBCw'
dvlist='apukeller.example.com#Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI.4JPgXSJympl9pU_3o6LrwNtopFBCHxS9i2GnWGdJBCw#https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA#dns-01#dns_pdns'
d
vlist='apukeller.example.com#Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI.4JPgXSJympl9pU_3o6LrwNtopFBCHxS9i2GnWGdJBCw#https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA#dns-01#dns_pdns,'
d='apukeller.example.com'
_d_alias
txtdomain='_acme-challenge.apukeller.example.com'
txt='k1nj8GcYBIMjxBQGgndSZwWMT9ZT4hwKDpJY5RmUSsA'
d_api='/usr/lib/acme/client/dnsapi/dns_pdns.sh'
Found domain api file: /usr/lib/acme/client/dnsapi/dns_pdns.sh
Adding txt value: k1nj8GcYBIMjxBQGgndSZwWMT9ZT4hwKDpJY5RmUSsA for domain:  _acme-challenge.apukeller.example.com
Detect root zone
GET
url='https://dnsrx1.example.com/api/v1/servers/localhost/zones'
timeout=
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
^C
root@apukeller:~# /etc/init.d/acme restart
Command failed: Not found
acme-acmesh: Running ACME for apukeller.example.com
acme-acmesh: /usr/lib/acme/client/acme.sh --debug --ecc -d apukeller.example.com --keylength ec-384 --accountemail jochen@example.com --server letsencrypt --dns dns_pdns --issue --home /etc/acme
Selected server: https://acme-v02.api.letsencrypt.org/directory
Lets find script dir.
_SCRIPT_='/usr/lib/acme/client/acme.sh'
_script='/usr/lib/acme/client/acme.sh'
_script_home='/usr/lib/acme/client'
Using config home:/etc/acme
https://github.com/acmesh-official/acme.sh
v3.0.6
Using server: https://acme-v02.api.letsencrypt.org/directory
Running cmd: issue
_main_domain='apukeller.example.com'
_alt_domains='no'
Using config home:/etc/acme
ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
DOMAIN_PATH='/etc/acme/apukeller.example.com_ecc'
Using ACME_DIRECTORY: https://acme-v02.api.letsencrypt.org/directory
_init api for server: https://acme-v02.api.letsencrypt.org/directory
GET
url='https://acme-v02.api.letsencrypt.org/directory'
timeout=
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
ret='0'
ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
ACME_NEW_AUTHZ
ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf'
ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
Using CA: https://acme-v02.api.letsencrypt.org/directory
_on_before_issue
_chk_main_domain='apukeller.example.com'
_chk_alt_domains
Run pre hook:'/usr/lib/acme/notify prepare'
Le_LocalAddress
d='apukeller.example.com'
Check for domain='apukeller.example.com'
_currentRoot='dns_pdns'
d
_saved_account_key_hash is not changed, skip register account.
Read key length:2048
Creating domain key
Using config home:/etc/acme
ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
Use length 384
Using ec name: secp384r1
The domain key is here: /etc/acme/apukeller.example.com_ecc/apukeller.example.com.key
_createcsr
Single domain='apukeller.example.com'
Getting domain auth token for each domain
d
url='https://acme-v02.api.letsencrypt.org/acme/new-order'
payload='{"identifiers": [{"type":"dns","value":"apukeller.example.com"}]}'
RSA key
HEAD
_post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g  -I  '
_ret='0'
POST
_post_url='https://acme-v02.api.letsencrypt.org/acme/new-order'
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
_ret='0'
code='201'
Le_LinkOrder='https://acme-v02.api.letsencrypt.org/acme/order/1072828657/222919289886'
Le_OrderFinalize='https://acme-v02.api.letsencrypt.org/acme/finalize/1072828657/222919289886'
url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/284817054046'
payload
POST
_post_url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/284817054046'
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
_ret='0'
code='200'
d='apukeller.example.com'
Getting webroot for domain='apukeller.example.com'
_w='dns_pdns'
_currentRoot='dns_pdns'
entry='"type":"dns-01","status":"pending","url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA","token":"Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI"'
token='Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI'
uri='https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA'
keyauthorization='Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI.4JPgXSJympl9pU_3o6LrwNtopFBCHxS9i2GnWGdJBCw'
dvlist='apukeller.example.com#Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI.4JPgXSJympl9pU_3o6LrwNtopFBCHxS9i2GnWGdJBCw#https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA#dns-01#dns_pdns'
d
vlist='apukeller.example.com#Fwh-_MAOzBSgHBlvKh3n0G7IQBUF1rF_9dwlJGw-WYI.4JPgXSJympl9pU_3o6LrwNtopFBCHxS9i2GnWGdJBCw#https://acme-v02.api.letsencrypt.org/acme/chall-v3/284817054046/ZV0HtA#dns-01#dns_pdns,'
d='apukeller.example.com'
_d_alias
txtdomain='_acme-challenge.apukeller.example.com'
txt='k1nj8GcYBIMjxBQGgndSZwWMT9ZT4hwKDpJY5RmUSsA'
d_api='/usr/lib/acme/client/dnsapi/dns_pdns.sh'
Found domain api file: /usr/lib/acme/client/dnsapi/dns_pdns.sh
Adding txt value: k1nj8GcYBIMjxBQGgndSZwWMT9ZT4hwKDpJY5RmUSsA for domain:  _acme-challenge.apukeller.example.com
Detect root zone
GET
url='https://dnsrx1.example.com/api/v1/servers/localhost/zones'
timeout=
_CURL='curl --silent --dump-header /etc/acme/http.header  -L  -g '
2

It seems the problem was that the api endpoint changed over time. Even though the value in /etc/config/acme was correct, the dnsapi endpoint value in the acmesh config in /etc/acme/account.conf was still the old one and thus failed.
Seems like a bug to me.