Second the "pretty damn wierd"... my guess would have been an OOM, or a segfault of some kind, but possibly also memory-related. Maybe worth running a top or shell loop that dumps /proc/meminfo (or /proc/vmstat or install vmstat and run that with a delay) in one SSH session while running the upgrade in another.
I'm unable to find a way to resolve this error message. Could someone please advise?
owut check
owut - OpenWrt Upgrade Tool 2025.01.29~bced54ad-r1 (/usr/bin/owut)
ERROR: uclient error code=1
This could be due to the server being down or inaccessible, check
https://sysupgrade.openwrt.org/json/v1/overview.json
ERROR: Response status 0 while downloading
https://sysupgrade.openwrt.org/json/v1/overview.json
I think the build bots are currently building 24.10.0 again, which means that you might not be able to download your target just yet. You can track the progress here if you're that way inclined you can wait and try in a day or so.
This might be more of a ASU server issue than an owut issue, not sure where I should post this, so please direct me where to post this if this is the wrong place.
There seem to be issues actually building the images, even when the installed configuration is valid. I wanted to consolidate everything into the image (instead of having packages added and removed in the overlay, I use the full dnsmasq and wpad for example) and this happened:
owut - OpenWrt Upgrade Tool 2025.01.29~bced54ad-r1 (/usr/bin/owut)
ASU-Server https://sysupgrade.openwrt.org
Upstream https://downloads.openwrt.org
Target mediatek/filogic
Profile glinet_gl-mt6000
Package-arch aarch64_cortex-a53
Root-FS-type squashfs
Sys-type sysupgrade
Version-from 24.10.0 r28427-6df0e3d02a (kernel 6.6.73)
Version-to 24.10.0 r28427-6df0e3d02a (kernel 6.6.73)
Build-FS-type squashfs
Build-at 2025-02-03T23:09:37Z (~83 hours ago)
Image-prefix openwrt-24.10.0-mediatek-filogic-glinet_gl-mt6000
Image-URL https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic
Image-file openwrt-24.10.0-mediatek-filogic-glinet_gl-mt6000-squashfs-sysupgrade.bin
Installed 260 packages
Top-level 107 packages
Default 39 packages
User-installed 78 packages (top-level only)
Package version changes:
luci 24.297.79519~bcd13b9 25.037.68331~42f464c
luci-app-attendedsysupgrade 25.035.62793~4f7a183 25.037.68331~42f464c
luci-app-firewall 25.014.06885~bd39c63 25.037.68331~42f464c
luci-app-package-manager 24.341.02353~276fef8 25.037.68331~42f464c
luci-app-sqm 25.035.62793~4f7a183 25.037.68331~42f464c
luci-app-statistics 25.035.62793~4f7a183 25.037.68331~42f464c
luci-base 25.014.55016~7046a1c 25.037.68331~42f464c
luci-lib-uqr 25.035.62793~4f7a183 25.037.68331~42f464c
luci-light 23.024.33244~34dee82 25.037.68331~42f464c
luci-mod-admin-full 19.253.48496~3f93650 25.037.68331~42f464c
luci-mod-network 25.021.30718~36b6107 25.037.68331~42f464c
luci-mod-status 25.009.65022~20bf9a4 25.037.68331~42f464c
luci-mod-system 24.336.49068~ef16778 25.037.68331~42f464c
luci-proto-ipv6 24.350.69236~8d13ea4 25.037.68331~42f464c
luci-proto-ppp 25.009.71487~a88d4af 25.037.68331~42f464c
luci-proto-wireguard 25.035.62793~4f7a183 25.037.68331~42f464c
luci-ssl 24.328.43591~9e70c9a 25.037.68331~42f464c
luci-theme-bootstrap 24.357.59015~779ca14 25.037.68331~42f464c
18 packages are out-of-date
Default package analysis:
Default Provided-by
dnsmasq dnsmasq-full
nftables nftables-json
ppp not installed
ppp-mod-pppoe not installed
wpad-basic-mbedtls not installed
There are currently package build failures for 24.10.0 aarch64_cortex-a53:
Feed: telephony
freetdm Fri Feb 7 06:47:15 2025 - not installed
Failures don't affect this device, details at
https://downloads.openwrt.org/releases/faillogs-24.10/aarch64_cortex-a53/
Request:
Version 24.10.0 r28427-6df0e3d02a (kernel 6.6.73)
Request hash:
6c6b46b7fbbd1ab78536a5cc59bc325983b98f2379a6d113fc8c9f7064f406f6
--
Status: queued - 0 ahead of you
Progress: 0s total = 0s in queue + 0s in build
--
Status: init
Progress: 1s total = 0s in queue + 1s in build
--
Status: container_setup
Progress: 3s total = 0s in queue + 3s in build
--
Status: validate_manifest
Progress: 17s total = 0s in queue + 17s in build
--
Status: Error: Impossible package selection
Progress: 20s total = 0s in queue + 20s in build
Build failed in 20s total = 0s in queue + 20s to build:
ASU server error =
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/rq/worker.py", line 1633, in perform_job
return_value = job.perform()
^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/rq/job.py", line 1331, in perform
self._result = self._execute()
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/rq/job.py", line 1365, in _execute
result = self.func(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/asu/build.py", line 241, in build
report_error(job, "Impossible package selection")
File "/app/asu/util.py", line 298, in report_error
raise RuntimeError(msg)
RuntimeError: Impossible package selection
ASU server stderr =
Generate local signing keys...
Generate local certificate...
Package list missing or not up-to-date, generating it.
Building package index...
Downloading http://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/packages/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_core
Downloading http://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/packages/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/base/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_base
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/base/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/kmods/6.6.73-1-d649d775435da5a8637f7a955a80d331/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_kmods
Downloading http://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/kmods/6.6.73-1-d649d775435da5a8637f7a955a80d331/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/luci/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_luci
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/luci/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/packages/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_packages
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/packages/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/routing/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_routing
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/routing/Packages.sig
Signature check failed.
Remove wrong Signature file.
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/telephony/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_telephony
Downloading http://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/telephony/Packages.sig
Signature check passed.
Downloading file:packages/Packages
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/imagebuilder
Downloading file:packages/Packages.sig
Signature check passed.
Collected errors:
* opkg_install_pkg: Checksum or size mismatch for package iftop. Either the opkg or the package index are corrupt. Try 'opkg update'.
* opkg_install_cmd: Cannot install package iftop.
* opkg_install_pkg: Checksum or size mismatch for package irqbalance. Either the opkg or the package index are corrupt. Try 'opkg update'.
* opkg_install_cmd: Cannot install package irqbalance.
* opkg_install_pkg: Checksum or size mismatch for package cgi-io. Either the opkg or the package index are corrupt. Try 'opkg update'.
* opkg_install_cmd: Cannot install package luci.
* opkg_install_pkg: Checksum or size mismatch for package cgi-io. Either the opkg or the package index are corrupt. Try 'opkg update'.
* opkg_install_cmd: Cannot install package luci-app-attendedsysupgrade.
* opkg_install_pkg: Checksum or size mismatch for package librrd1. Either the opkg or the package index are corrupt. Try 'opkg update'.
* opkg_install_cmd: Cannot install package luci-app-statistics.
make[2]: *** [Makefile:234: package_install] Error 255
make[1]: *** [Makefile:171: _call_manifest] Error 2
make: *** [Makefile:349: manifest] Error 2
ERROR: Build failed with status 500
The above errors are often due to the upgrade server lagging behind the
build server, first suggestion is to wait a while and try again.
Looks like there are some seriously broken packages on the servers, as those same packages install fine with opkg locally. Owut initially claims that "Failures don't affect this device" but that doen't seem to be quite true.
I'm surprised that the build servers don't use a consistent snapshot of the state, or that they upgrade just before a build? Furthermore: these packages don't seem to have been updated since the release of 24.10: I installed yesterday (and installed the packages then as well). I don't see any upgrades to the packages since then, so how long typically should one "wait a while" before trying again? It seems to be days at least?
1 Like
Yeah, like @vorpal says, this is an ASU server issue and I haven't dug into the uclient code deeply enough to detect that in owut to produce a more meaningful message...
If you try to access the site now, does it work?
That looks like the package index is mangled somehow. While the base OpenWrt OS has a fixed release point, the packages follow a rolling release strategy, so you can sometimes fall into the cracks between when a package's .ipk is uploaded and when the corresponding index gets uploaded. If the servers and/or network are congested, like when, oh I don't know, there's a new release and 1000 eager beavers are trying to download it, then it can take 10s of minutes; or maybe something times out and you have to wait for the next cycle.
That's sort of hard to answer, because "it depends". The ASU server keeps failed build status around for 10 minutes (i.e., if you submit an identical build request within that window, it will return cached results, after 10 minutes the cache is flushed and the build is retried, possibly failing again), so that's the very minimum possible.
If the failure was because you landed between the rsync of two files, then maybe things will heal right away and the 10m delay is enough. If the rsync times out or otherwise fails the upload, then it could indeed be days before that platform's package builds go through again and repair everything that's mismatched.
1 Like
I think I ran into a bug with owut. I have the following /etc/sysupgrade.conf:
/etc/.git
/etc/.gitignore
/root/.config
/root/.gitconfig
/root/bin
/root/.profile
/root/bin was preserved by owut, but the dotfiles where not.
I'm currently restoring them from backup, but that is annoying.
The files were preserved on normal sysupgrade from 23.05.5 to 24.10 (using a non-customised build)
Hi Eric, I’m still getting the error and it’s been ongoing for a while now. My attendedsysupgrade config is fine and the LuCI app works. I have two GL.iNet MT-6000 devices and both return the error.
Thanks!
Now that you mention it, I think that happened to me, too. I did an -rc7 to .0 upgrade and when I later edited a file my /root/.vimrc was gone. This is all sysupgrade's thing, but when I do sysupgrade -l it lists out exactly what I expect, including /root/.vimrc. Oh, and I see that .config got restore, so ???
I'll look at sysupgrade and see if it's doing something funny on restore. I'm pretty sure it's saving everything correctly since I used my Monday morning backup to recover the .vimrc file and it's based on sysupgrade -b blah && scp blah nas:.
1 Like
Is it coming back instantly, like 0-2 seconds for the "build" phase? If that's the case, ASU server is still coughing up the cached failure (which should time out in 10 minutes, so it would be bad if it did).
I'm looking at where iftop comes from, and all the package and index file dates look ok, so it would seem they are corrupted internally.
https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/packages/
What does opkg update && opkg download iftop show on your device?
I just tried a download (faking it from an x86, but build request should be identical), and it succeeded. (The final "error" is just saying I shouldn't try to install a glinet img on my x86.)
$ owut download --device mediatek/filogic:glinet_gl-mt6000 -r "$(cat x86-kmods)" -a iftop
ASU-Server https://sysupgrade.openwrt.org
Upstream https://downloads.openwrt.org
Target mediatek/filogic
Profile glinet_gl-mt6000
Package-arch aarch64_cortex-a53
Root-FS-type squashfs
Sys-type sysupgrade
Version-from 24.10.0 r28427-6df0e3d02a (kernel 6.6.73)
Version-to 24.10.0 r28427-6df0e3d02a (kernel 6.6.73)
Build-FS-type squashfs
Build-at 2025-02-03T23:09:37Z (~97 hours ago)
Image-prefix openwrt-24.10.0-mediatek-filogic-glinet_gl-mt6000
Image-URL https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic
Image-file openwrt-24.10.0-mediatek-filogic-glinet_gl-mt6000-squashfs-sysupgrade.bin
Installed 281 packages
Top-level 85 packages
Default 39 packages
User-installed 65 packages (top-level only)
Request:
Version 24.10.0 r28427-6df0e3d02a (kernel 6.6.73)
ROOTFS_PARTSIZE set to 512 MB
Request hash:
61530683d1374ecbbe6afb2022696d342b7dd61dd88a2ca62993b3e1637ce7a7
--
Status: queued - 0 ahead of you
Progress: 0s total = 0s in queue + 0s in build
--
Status: container_setup
Progress: 1s total = 0s in queue + 1s in build
--
Status: validate_manifest
Progress: 15s total = 0s in queue + 15s in build
--
Status: building_image
Progress: 42s total = 0s in queue + 42s in build
--
Status: done
Progress: 46s total = 0s in queue + 46s in build
Build succeeded in 46s total = 0s in queue + 46s to build:
version_number = 24.10.0
version_code = r28427-6df0e3d02a (requested r28427-6df0e3d02a)
kernel_version = 6.6.73
rootfs_size_mb = 512
init-script = no-init-script
Image source: https://sysupgrade.openwrt.org/store/c44f5fc26bd1794aa6dd2c3b9fc17e58f2616f7245b4776ff713312982b25789/openwrt-24.10.0-9b95f46c723f-mediatek-filogic-glinet_gl-mt6000-squashfs-sysupgrade.bin
Image saved : /tmp/firmware.bin
Manifest : /tmp/firmware-manifest.json
Verifying : /tmp/firmware.bin (23563092 bytes) against /tmp/firmware.sha256sums
Saved sha256 matches
ERROR: sysupgrade validation failed:
stderr =
Fri Feb 7 15:51:38 PST 2025 upgrade: Device generic not supported by this image
Fri Feb 7 15:51:38 PST 2025 upgrade: Supported devices: glinet,gl-mt6000
Oh, I see that I have rootfs size set to 512 on that device, so this is probably not good to install anywhere. 
But at least it built ok.
It returns the error instantly.
Here is the output from those commands:
opkg update && opkg download iftop
Downloading https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_core
Downloading https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/packages/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/base/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_base
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/base/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/kmods/6.6.73-1-d649d775435da5a8637f7a955a80d331/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_kmods
Downloading https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/kmods/6.6.73-1-d649d775435da5a8637f7a955a80d331/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_luci
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/luci/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_packages
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/packages/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_routing
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/routing/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_telephony
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/telephony/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/24.10.0/packages/aarch64_cortex-a53/packages/iftop_2018.10.03~77901c8c-r2_aarch64_cortex-a53.ipk
Downloaded iftop as ./iftop_2018.10.03~77901c8c-r2_aarch64_cortex-a53.ipk.
Thoughts?
Okay so I add a custom uci-defaults script via command-line:
owut download -I /rom/etc/uci-defaults/99-asu-defaults
The script is a working script, proven on many occasions.
Image created, I make sure the script is part of the image before manually sys-uprading, not keeping settings. (I do this to create a clean backup, and check if there are any settings that may have been introduced/removed between the last clean backup).
Situation lately is after reboot, the script doesn't get utilised.
The script still exists within /etc/uci-defaults/,
exists in /rom/etc/uci-defaults/,
and no /tmp/setup.log exits.
Is there something else here to check?
Do you know the proper location to report bugs in sysupgrade?
@efahl. Just wanted to say that I used the owut upgrade -S 1024 command and all settings were preserved after flashing. Thank you!
1 Like
It appears that all the ssl libraries are there, as opkg needs those to do https. The packages appear to be in the correct place and are included in the package indexes. The only thing I can think of is that the ASU server has a stale or misconfigured image builder, but that can't be true if one of your other devices with that same target actually built.
Can you try again and post the output from owut download -v and I'll pick through that and see if I can make anything out.
If the script still exists in that directory after the reboot, that almost always indicates that it encountered an error during execution. Check the script and see if you can find anything amiss, and ensure that when it terminates properly it does exit 0 or equivalent.
1 Like
Since that's part of base files, the openwrt main issues is where that should go. Make sure to prefix the issue title so as to get the right eyes on it "sysupgrade: blah blah".
1 Like
Well, that's unexpected. This was the first time with that rootfs size setting?
Hahaha. And here I thought you were trying to surprise us. Now I'm worried if it wasn't intended.
Yes. I backed up my configuration like a good boy, upgraded using OWUT and when I logged in to restore my configuration I couldn't get pass the login screen. That's when it dawned on me that the config was still intact.