AUC appears to have cached build failure due to server-side issue

Installing and Using OpenWrt
1

Hey folks, I'm trying to use the auc command (specifically auc -B 23.05.3) on my Linksys E8450 router running openwrt 23.05.4 (ie: I'm trying to downgrade to 23.05.3 from 23.05.4. This is because I'm trying to figure out if a usb issue I'm having is a regression).

The auc command unfortunately errored out with:

Signature check passed.
Downloading https://downloads.openwrt.org/releases/23.05.3/packages/aarch64_cortex-a53/telephony/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/openwrt_telephony
Downloading https://downloads.openwrt.org/releases/23.05.3/packages/aarch64_cortex-a53/telephony/Packages.sig
Signature check passed.
Downloading file:packages/Packages
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-mediatek/../../../../builder/dl/imagebuilder
Downloading file:packages/Packages.sig
Signature check passed.
Collected errors:
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/23.05.3/packages/aarch64_cortex-a53/base/odhcp6c_2023-05-12-bcd28363-20_aarch64_cortex-a53.ipk, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

 * opkg_install_pkg: Failed to download odhcp6c. Perhaps you need to run 'opkg update'?
 * opkg_install_cmd: Cannot install package odhcp6c.
 * opkg_install_pkg: Checksum or size mismatch for package odhcp6c. Either the opkg or the package index are corrupt. Try 'opkg update'.
 * opkg_install_cmd: Cannot install package odhcp6c.
make[2]: *** [Makefile:189: package_install] Error 255
make[1]: *** [Makefile:144: _call_image] Error 2
make: *** [Makefile:262: image] Error 2
Error: Error while building firmware. See stdout/stderr
Bad message (74)

This appears to be something that is going wrong on the auc builder itself, and unfortunately it appears that this failure is cached (while the first attempt took some time to return an error, subsequent attempts return it instantly).

Questions:

  1. Am I correct in believing this is an attended sysupgrade server issue?
  2. If it is, does anyone know if the attended sysupgrade server side issue here is known and/or being resolved?
  3. If the attended sysupgrade server-side issue is resolved at some point, will the cache entry for this error result be removed or otherwise time out?

Thanks.

2
  1. not derivable from your output
  2. no if no
  3. no if no
    Please post complete auc output and ubus call system board
3

complete auc output: https://gist.github.com/codyps/3cfb2e6554b14cf0103cc7386971fca9 (was too long to post inline)

root@crust:~# ubus call system board
{
        "kernel": "5.15.162",
        "hostname": "crust.einic.org",
        "system": "ARMv8 Processor rev 4",
        "model": "Linksys E8450 (UBI)",
        "board_name": "linksys,e8450-ubi",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "23.05.4",
                "revision": "r24012-d8dd03c46f",
                "target": "mediatek/mt7622",
                "description": "OpenWrt 23.05.4 r24012-d8dd03c46f"
        }
}
root@crust:~#

The reason I suspected something off with the ASU server is that the package it appears to fail to download (https://downloads.openwrt.org/releases/23.05.3/packages/aarch64_cortex-a53/base/odhcp6c_2023-05-12-bcd28363-20_aarch64_cortex-a53.ipk) was downloadable without issue for me manually, and https://openwrt.org/faq/failed_to_download_wget_returned_4 implies this particular error is some networking issue or perhaps a /tmp space issue (ie: on the server in this case, because that is where the wget is being executed). Presumably wget could print us some better logs, but they're either silenced or redirected somewhere.

4

I'd try in 24h, yes it sometimes caches failed builds.
Can you state problem with 23.05.4 -> it may be resolved in 23.05-SNAPSHOT already and no revert needed.

5

Correct.

Yes, this is a known issue and is being worked on.

Yes, but you won't want to wait for it; cached builds are flushed after a week.

If you use https://firmware-selector.openwrt.org/ for the build, instead of auc, you can trick the ASU server into bypassing the cache by adding a no-op values in the "uci-defaults" (an empty comment works the first time, just keep adding a "#" on each attempt).

To get your package list use

 eval $(ubus call rpc-sys packagelist '{"all":false}' | jsonfilter -e 'pkgs=$.packages') && echo $pkgs

and paste that output into the package list on the firmware selector, after the default list.