Software update fails on 1 of my devices

bartje · February 13, 2023, 8:48am

Hello,

I have 3 accesspoints and on 2 of them there are no issues when I click,
"Update list..." in the "Software" menu.
The 3th one however raises an error.

Executing package manager

Downloading https://downloads.openwrt.org/releases/22.03.3/targets/ath79/generic/packages/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.3/targets/ath79/generic/packages/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/base/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/base/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/luci/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/luci/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/packages/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/packages/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/routing/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/routing/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/telephony/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/telephony/Packages.gz

Errors

Collected errors:
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.3/targets/ath79/generic/packages/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/base/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/luci/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/packages/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/routing/Packages.gz, wget returned 5.
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/telephony/Packages.gz, wget returned 5.

The opkg update command failed with code 6.

which is strange since the config is the same as for the other 2,
diagnostics(ping/tracert/nslookup) doesn't find any issues.
I tried to reboot but the issue remains.

How can I find the root cause of this? It's not that I really need this software update right now, but I think it might be related to some network I experience sometimes.

badulesia · February 13, 2023, 8:52am

Did you setup a gateway ? Without it the AP can't access internet for itself.

frollic · February 13, 2023, 8:59am

this wouldn't work if there wasn't one.

trendy · February 13, 2023, 9:04am

From wget exit codes:
5 SSL verification failure
What is the output of opkg list-installed | grep cert ?

bartje · February 13, 2023, 9:09am

it's empty

root@OpenWrt4:~# opkg list-installed | grep cert
root@OpenWrt4:~#

but that also is the case on the devices where "update list" is working

root@OpenWrt5:~# opkg list-installed | grep cert
root@OpenWrt5:~#

I could still be that I'm missing some CA's, don't know if I need to update them manually.

trendy · February 13, 2023, 9:39am

What about opkg list-installed | grep wget ?

bartje · February 13, 2023, 9:52am

same result

root@OpenWrt4:~# opkg list-installed | grep wget
root@OpenWrt4:~#

root@OpenWrt5:~# opkg list-installed | grep wget
root@OpenWrt5:~#

trendy · February 13, 2023, 9:54am

And are they all the same devices? What is the output of ubus call system board ?

bartje · February 13, 2023, 10:11am

yes the are
TP-Link Deco M4R v2
but for the v2, the documentation points to the v1 firmware

root@OpenWrt4:~# ubus call system board
{
        "kernel": "5.10.161",
        "hostname": "OpenWrt4",
        "system": "Qualcomm Atheros QCA956X ver 1 rev 0",
        "model": "TP-Link Deco M4R v1",
        "board_name": "tplink,deco-m4r-v1",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "22.03.3",
                "revision": "r20028-43d71ad93e",
                "target": "ath79/generic",
                "description": "OpenWrt 22.03.3 r20028-43d71ad93e"
        }
}
root@OpenWrt4:~#

root@OpenWrt5:~# ubus call system board
{
        "kernel": "5.10.161",
        "hostname": "OpenWrt5",
        "system": "Qualcomm Atheros QCA956X ver 1 rev 0",
        "model": "TP-Link Deco M4R v1",
        "board_name": "tplink,deco-m4r-v1",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "22.03.3",
                "revision": "r20028-43d71ad93e",
                "target": "ath79/generic",
                "description": "OpenWrt 22.03.3 r20028-43d71ad93e"
        }
}
root@OpenWrt5:~#

trendy · February 13, 2023, 10:17am

That's rather odd. We could troubleshoot deeper, but it will be faster to take a backup and reset to defaults. It's my understanding that the devices run as dumb APs, so nothing complicated to restore.

bartje · February 13, 2023, 11:45am

that doesn't help,
you are right about the ssl part btw

root@OpenWrt4:~# wget https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/routing/Packages.gz
Downloading 'https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/routing/Packages.gz'
Connecting to 168.119.138.211:443
Connection error: Invalid SSL certificate
root@OpenWrt4:~#

but here is the interesting part.
on OpenWrt5 this download succeeded (log from OpenWrt6)

root@OpenWrt6:~# wget https://downloads.openwrt.org/releases/22.03.3/packages/mi
ps_24kc/routing/Packages.gz
Downloading 'https://downloads.openwrt.org/releases/22.03.3/packages/mips_24kc/routing/Packages.gz'
Connecting to 168.119.138.211:443
Writing to 'Packages.gz'
Packages.gz          100% |*******************************| 12244   0:00:00 ETA
Download completed (12244 bytes)
root@OpenWrt6:~#

but after resrtarting OpenWrt5 it failed the same way

I checked the certificate on downloads.openwrt.org and that one looks like a solid let's encrypt one.
So no idea why after a restart the certificate is suddenly rejected.

trendy · February 13, 2023, 12:48pm

Wrong time on the dumbAP makes it think the SSL is not valid.

bartje · February 13, 2023, 1:06pm

thanks, that's it,
now I'm wondering why ntp doesn't work as expected but problem solved

frollic · February 13, 2023, 1:33pm

if you're using FQDNs for the NTP, non-working DNS(es), would cause the NTP fail to sync.

bartje · February 13, 2023, 2:01pm

I see,
A warning for that would be nice,
I use my local ntp server based on host,
I changed it to IP and all is working.
I prefer host because that way I can modify ip addresses only by changing dhcp server.

efahl · February 13, 2023, 4:52pm

I "cheat" by intercepting all NTP queries on my main router, so everything on the LAN is synced to it within less than a millisecond.

nft add rule inet fw4 dstnat_lan   udp dport 123   counter redirect   comment "NTP: Handle all NTP requests locally."

bartje · February 13, 2023, 7:46pm

that is an interesting one,
it would mean I don't have to specify ntp per device anymore,
in this case would I just replace "dstnat_lan" with my ntp server ip?
I'm not that familiar with Linux firewall rules.

frollic · February 13, 2023, 7:50pm

You do, but you catch all the outgoing calls, and redirect them to the LAN NTP.
I'm not sure it would have solved your specific issue though, if there are no DNS
IP, there would be no call to catch, because the name look up would fail in the
1st place.

efahl · February 13, 2023, 11:18pm

Correct, I don't bother changing any of the hosts or IPs on my servers, workstations, TVs, phones etc any more, as everything gets intercepted by the router. You can basically try to get time from anything that has an IP addressl, and it "works". Here's from my Ubuntu server, with a nonsense IP:

$ ntpdate -q 1.2.3.4
server 1.2.3.4, stratum 2, offset -0.000747, delay 0.02858
13 Feb 15:01:17 ntpdate[1540594]: adjust time server 1.2.3.4 offset -0.000747 sec

And I was being dumb showing that redirect as a nft rule, this can easily be accomplished in LuCI. Go to Network -> Firewall, on the Port Forwards tab and add a new rule that looks like this:

Make sure you've got the NTP server running on your router, first, though! (Above ntpdate -q <router-IP> is what I use before creating the rule.)

bartje · February 14, 2023, 12:27pm

I checked but I think I'm doing something wrong,
I installed ntpd on my router
then I changed the ntp server candidates back to avoid an infinite loop

this part actually works.
and then I created the rule

for some reason I'm missing restrict from address family,
but the rule is created and enabled

but when I manipulate the time on my router to be wrong, windows is still retrieving the correct time from time.windows.com