Opkg certificate problems after upgrade to 21.02rc4

Hi,

I updated a unifi AC mesh from 19.07 to 21.02rc4 like this:

sysupgrade -f /tmp/mybackup.tar.gz -n -p openwrt.image

Basically the update went well. After reboot, my config was still there and all the stuff with the interface updated also seem to have worked.

Yet opkg is broken now. When I run opkg update I get this:

root@unifiac:~# opkg update
Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ath79/generic/packages/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ath79/generic/packages/Packages.gz

Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/base/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/base/Packages.gz

Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/luci/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/luci/Packages.gz

Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/packages/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/packages/Packages.gz

Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/routing/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/routing/Packages.gz

Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/telephony/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/telephony/Packages.gz

Collected errors:
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ath79/generic/packages/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/base/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/luci/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/packages/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/routing/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/telephony/Packages.gz, wget returned 5.

At first I thought it was a problem with my upstream interface only having IPv6 but that was not the case.
Then I tried to wget one of the URLs and I get presented a certificate error.

root@unifiac:~# wget https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ath79/generic/packages/Packages.gz
Downloading 'https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ath79/generic/packages/Packages.gz'
Connecting to 2a01:4f8:251:321::2:443
Connection error: Invalid SSL certificate

I ran opkg update --no-check-certificate and then it works.
I reinstalled ca-bundle, but doesn't change a thing.
Is it possible that somehow the CA cert for letsencrypt is missing or something?

the download server certificate seems to be OK https://downloads.openwrt.org/releases/21.02.0-rc4/

So it's an issue on your side.

What do you mean?
I did not create the OpenWrt image, so how is this on my side?

I mean that there is something wrong in your device, the server's https certificate is fine since I can connect to it without getting a warning from my web browser.

I don't know if with "CA cert for letsencrypt is missing or something" you meant the server certificate or the certificate in your router. All OpenWrt servers use letsencrypt to get free https certificates afaik

I can confirm your observations but they do not help me.
A client needs to be able to determine if the server certificate is valid or not.

To be able to do that one needs the CA cert of the issuer, in that case of letsencrypt.
Browser embed those certificates AFAIK.
honestly don't know how OpenWrt does it but I expected it to be the ca-bundle, which btw only contains
those two files:

/etc/ssl/cert.pem
/etc/ssl/certs/ca-certificates.crt

Both seem to be the same file and originate from https://www.accv.es/ I don't know what that is.
Maybe someone with more insight can shed light into the dark plz.

in OpenWrt there are actually two packages about certificates. One dumps a bunch of certificates as single files and one is a bundle of many certs in a single file.

Some applications want one, some want the other, I don't know, see the package definition for the certs packages https://github.com/openwrt/openwrt/blob/master/package/system/ca-certificates/Makefile

they are ca-certificates and ca-bundle

So maybe try with the other, or with both

No. I will create an issue on the bugtracker.

Edit:
The x86 image does not suffer from this problem... I will try to reflash the image to double check before I open an issue.

1 Like

Did you check the current date & time on your OpenWRT box?
Certificates have an expiration date and also an issue date. When the system time is not set correctly, which can be the case in a fresh OpenWRT installation where WAN is not set up or no NTP server is reachable, the certificate validation can easily fail when the system date is before the issue date of the certificate.
That happened to me on my Raspberry Pi 4 during set-up phase where I had LAN-only.

1 Like

I checked it and the time is in sync perfectly, but thanks for the hint :slight_smile:

This is a weird topic. I've been rebuilding x86 images (image builder, not buildroot) over and over again and suddenly I have an iteration of images that also suffer this very specific problem.
Dunno what's going on here.

1 Like

Read today's threads about letsencrypt root certificate expiration today, September 30th.

1 Like

I'm stuck here too. How can I fix it?

I used the opkg option --no-check-certificate

1 Like