Owut: OpenWrt Upgrade Tool

No, but I have dug through luci-app-attendedsysupgrade quite thoroughly while building this thing and have some ideas to make it a little more robust.

My general view is that anyone doing fancy stuff stuff like increasing rootfs partition size, or adding custom uci-defaults, is typically going to be a command line user anyway, so having that sort of thing in LuCI is not worth it in terms of cost (implementation, maintenance, documentation, code size/drive space).

owut's feature set is really aimed at people who want to experiment a lot with "router stuff", but don't want to spend time doing builds. They'd rather focus on, for example, automating their VLAN scripts or configuring DNS or whatever, and thus have lots of packages and configs to manage, making upgrades a pain.

On the other hand, I've added some checks in owut explicitly to address issues we often see here on the forums, like "why can't I install package X, it was there yesterday?" and "what's going on when I see 'requested r23456 but got r12345'?" Those sorts of checks are of benefit to everyone, so I am thinking of ways to pull them out and package them as services for any router-based ASU client to use...

3 Likes

Well, I guess Daniel answered that one.

3 Likes

works flawlessly one one router, fails on another:

root@wifiap1:~# owut check --verbose
owut - OpenWrt Upgrade Tool 2024.06.24~1b222bdc-r1 (/usr/bin/owut)
ERROR: uclient error code=1
Response status: 0
ERROR: Could not download 'https://sysupgrade.openwrt.org/json/v1/overview.json', terminating.
root@wifiap:~# curl -I https://sysupgrade.openwrt.org/json/v1/overview.json
HTTP/2 200 
accept-ranges: bytes
access-control-allow-headers: *
access-control-allow-methods: POST, GET, OPTIONS
access-control-allow-origin: *
alt-svc: h3=":443"; ma=2592000
content-type: application/json
etag: "d2a2end9u3r8dlu"
last-modified: Wed, 26 Jun 2024 15:47:36 GMT
server: Caddy
vary: Accept-Encoding
content-length: 17634
date: Wed, 26 Jun 2024 15:48:38 GMT

Unfortunately the error message is non-descriptive, curl will fetch the file without issues.

Could you try uclient-fetch https://sysupgrade.openwrt.org/json/v1/overview.json and see if that works?

I've been having issues this morning using uclient-fetch to get various files, it just hangs, where curl gets the file just fine on the same device... Since owut and uclient-fetch rely on the same libuclient.so underneath, trying that alternate path might tell us something.

I also have problems ,in this case a RAX120v2. It might be a server error as it also errors out on the firmware selector (??)

root@OpenWrt:~# owut download
Board-name     netgear_rax120v2
Target         qualcommax/ipq807x
Root-FS-type   squashfs
Sys-type       sysupgrade
Package-arch   aarch64_cortex-a53
Version-from   SNAPSHOT r26742-d648ee4c58 (kernel 6.6.34)
Version-to     SNAPSHOT r26792-646ebbd32c (kernel 6.6.35)
Build-FS-type  squashfs
Build-at       2024-06-25T19:42:54Z
Image-prefix   openwrt-qualcommax-ipq807x-netgear_rax120v2
Image-file     openwrt-qualcommax-ipq807x-netgear_rax120v2-squashfs-sysupgrade.bin
Image-URL      https://mirror-03.infra.openwrt.org/snapshots/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-netgear_rax120v2-squashfs-sysupgrade.bin
Installed      286 packages
Top-level      131 packages
Default         47 packages
User-installed  93 packages (top-level only)

Package version changes:
  base-files                       1606~d648ee4c58                            1606~646ebbd32c
  kmod-ath                         6.6.34.6.6.15-r2                           6.6.35.6.6.15-r2
  kmod-ath11k                      6.6.34.6.6.15-r2                           6.6.35.6.6.15-r2
  kmod-ath11k-ahb                  6.6.34.6.6.15-r2                           6.6.35.6.6.15-r2
  kmod-cfg80211                    6.6.34.6.6.15-r2                           6.6.35.6.6.15-r2
  kmod-crypto-acompress            6.6.34-r1                                  6.6.35-r1
  kmod-crypto-aead                 6.6.34-r1                                  6.6.35-r1
  kmod-crypto-ccm                  6.6.34-r1                                  6.6.35-r1
  kmod-crypto-cmac                 6.6.34-r1                                  6.6.35-r1
  kmod-crypto-crc32c               6.6.34-r1                                  6.6.35-r1
  kmod-crypto-ctr                  6.6.34-r1                                  6.6.35-r1
  kmod-crypto-gcm                  6.6.34-r1                                  6.6.35-r1
  kmod-crypto-geniv                6.6.34-r1                                  6.6.35-r1
  kmod-crypto-gf128                6.6.34-r1                                  6.6.35-r1
  kmod-crypto-ghash                6.6.34-r1                                  6.6.35-r1
  kmod-crypto-hash                 6.6.34-r1                                  6.6.35-r1
  kmod-crypto-hmac                 6.6.34-r1                                  6.6.35-r1
  kmod-crypto-manager              6.6.34-r1                                  6.6.35-r1
  kmod-crypto-michael-mic          6.6.34-r1                                  6.6.35-r1
  kmod-crypto-null                 6.6.34-r1                                  6.6.35-r1
  kmod-crypto-rng                  6.6.34-r1                                  6.6.35-r1
  kmod-crypto-seqiv                6.6.34-r1                                  6.6.35-r1
  kmod-crypto-sha3                 6.6.34-r1                                  6.6.35-r1
  kmod-crypto-sha512               6.6.34-r1                                  6.6.35-r1
  kmod-fs-autofs4                  6.6.34-r1                                  6.6.35-r1
  kmod-fs-exfat                    6.6.34-r1                                  6.6.35-r1
  kmod-fs-ext4                     6.6.34-r1                                  6.6.35-r1
  kmod-fs-ntfs3                    6.6.34-r1                                  6.6.35-r1
  kmod-fuse                        6.6.34-r1                                  6.6.35-r1
  kmod-gpio-button-hotplug         6.6.34-r3                                  6.6.35-r3
  kmod-gpio-nxp-74hc164            6.6.34-r1                                  6.6.35-r1
  kmod-hwmon-core                  6.6.34-r1                                  6.6.35-r1
  kmod-input-core                  6.6.34-r1                                  6.6.35-r1
  kmod-leds-gpio                   6.6.34-r1                                  6.6.35-r1
  kmod-lib-crc-ccitt               6.6.34-r1                                  6.6.35-r1
  kmod-lib-crc16                   6.6.34-r1                                  6.6.35-r1
  kmod-lib-crc32c                  6.6.34-r1                                  6.6.35-r1
  kmod-lib-lzo                     6.6.34-r1                                  6.6.35-r1
  kmod-libphy                      6.6.34-r1                                  6.6.35-r1
  kmod-mac80211                    6.6.34.6.6.15-r2                           6.6.35.6.6.15-r2
  kmod-nf-conntrack                6.6.34-r1                                  6.6.35-r1
  kmod-nf-conntrack6               6.6.34-r1                                  6.6.35-r1
  kmod-nf-flow                     6.6.34-r1                                  6.6.35-r1
  kmod-nf-log                      6.6.34-r1                                  6.6.35-r1
  kmod-nf-log6                     6.6.34-r1                                  6.6.35-r1
  kmod-nf-nat                      6.6.34-r1                                  6.6.35-r1
  kmod-nf-reject                   6.6.34-r1                                  6.6.35-r1
  kmod-nf-reject6                  6.6.34-r1                                  6.6.35-r1
  kmod-nfnetlink                   6.6.34-r1                                  6.6.35-r1
  kmod-nft-core                    6.6.34-r1                                  6.6.35-r1
  kmod-nft-fib                     6.6.34-r1                                  6.6.35-r1
  kmod-nft-nat                     6.6.34-r1                                  6.6.35-r1
  kmod-nft-offload                 6.6.34-r1                                  6.6.35-r1
  kmod-nls-base                    6.6.34-r1                                  6.6.35-r1
  kmod-phy-aquantia                6.6.34-r1                                  6.6.35-r1
  kmod-ppp                         6.6.34-r1                                  6.6.35-r1
  kmod-pppoe                       6.6.34-r1                                  6.6.35-r1
  kmod-pppox                       6.6.34-r1                                  6.6.35-r1
  kmod-qca-nss-dp                  6.6.34.2024.04.16~5bf8b91e-r1              6.6.35.2024.04.16~5bf8b91e-r1
  kmod-qca-ssdk                    6.6.34.2024.04.17~3d060f7a-r2              6.6.35.2024.06.13~c451136b-r3
  kmod-qrtr                        6.6.34-r1                                  6.6.35-r1
  kmod-qrtr-smd                    6.6.34-r1                                  6.6.35-r1
  kmod-scsi-core                   6.6.34-r1                                  6.6.35-r1
  kmod-slhc                        6.6.34-r1                                  6.6.35-r1
  kmod-sound-core                  6.6.34-r1                                  6.6.35-r1
  kmod-spi-bitbang                 6.6.34-r1                                  6.6.35-r1
  kmod-spi-gpio                    6.6.34-r1                                  6.6.35-r1
  kmod-thermal                     6.6.34-r1                                  6.6.35-r1
  kmod-usb-core                    6.6.34-r1                                  6.6.35-r1
  kmod-usb-dwc3                    6.6.34-r1                                  6.6.35-r1
  kmod-usb-dwc3-qcom               6.6.34-r1                                  6.6.35-r1
  kmod-usb-storage                 6.6.34-r1                                  6.6.35-r1
  kmod-usb-storage-extras          6.6.34-r1                                  6.6.35-r1
  kmod-usb-storage-uas             6.6.34-r1                                  6.6.35-r1
  kmod-usb-xhci-hcd                6.6.34-r1                                  6.6.35-r1
  kmod-usb3                        6.6.34-r1                                  6.6.35-r1
  libopenssl                       3.0.14-r1                                  3.0.14-r2
  libsane                          1.0.32-r4                                  1.3.1-r1
  luci-i18n-adblock-es             24.163.67634~860b321                       24.176.44345~05e4b54
  luci-i18n-attendedsysupgrade-es  24.171.69191~c06c6d1                       24.176.44345~05e4b54
  luci-i18n-base-es                24.171.69191~c06c6d1                       24.176.44345~05e4b54
  luci-i18n-firewall-es            24.167.25894~264e465                       24.176.44345~05e4b54
  luci-i18n-samba4-es              24.163.67634~860b321                       24.176.44345~05e4b54
  luci-mod-status                  24.141.29261~e7f2e6b                       24.176.44408~db6234c
  sane-snapscan                    1.0.32-r4                                  1.3.1-r1
  ubi-utils                        2.1.6-r2                                   2.1.6-r3
86 packages are out-of-date.

Default package analysis:
  Default              Provided-by
  kmod-hwmon-g762      not installed, possibly replaced by another package
  nftables             not installed, possibly replaced by another package
  wpad-basic-mbedtls   not installed, possibly replaced by another package

There are currently package build failures for SNAPSHOT aarch64_cortex-a53:
  Feed: packages
    basicstation               Tue Jun 25 09:22:35 2024 - not installed
    elektra                    Tue Jun 25 11:11:55 2024 - not installed
    freeradius3                Tue Jun 25 11:12:48 2024 - not installed
    kadnode                    Tue Jun 25 08:28:55 2024 - not installed
    libuhttpd                  Tue Jun 25 09:35:05 2024 - not installed
    micropython                Tue Jun 25 08:32:52 2024 - not installed
    schroot                    Tue Jun 25 11:22:51 2024 - not installed
    shadowsocks-libev          Tue Jun 25 10:15:32 2024 - not installed
    umurmur                    Tue Jun 25 09:45:17 2024 - not installed
  Feed: telephony
    freeswitch                 Tue Jun 25 11:05:44 2024 - not installed
    freeswitch-mod-bcg729      Tue Jun 25 11:23:15 2024 - not installed
    freetdm                    Tue Jun 25 11:23:18 2024 - not installed
Failures don't affect this device, details at
  https://mirror-03.infra.openwrt.org/snapshots/faillogs/aarch64_cortex-a53/

Build: Requesting version r26792-646ebbd32c (kernel 6.6.35)

Requesting build ----------------------
Hash:   d030386f7cd46115a6be623a9c182a6b
Status: 202
Detail: queued - 0 ahead of you

Progress   1 (  12s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   2 (  23s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   3 (  34s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   4 (  46s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   5 (  57s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   6 (  68s) -----------------------------
Status: 202
Detail: started - building_image

Progress   7 (  80s) -----------------------------
Status: 202
Detail: started - building_image

Progress   8 (  91s) -----------------------------
Status: 202
Detail: started - building_image

Progress   9 ( 102s) -----------------------------
Status: 202
Detail: started - building_image

Progress  10 ( 114s) -----------------------------
Status: 202
Detail: started - building_image

Progress  11 ( 125s) -----------------------------
Status: 202
Detail: started - building_image

Progress  12 ( 136s) -----------------------------
Status: 202
Detail: started - building_image

Progress  13 ( 148s) -----------------------------
Status: 202
Detail: started - building_image

Progress  14 ( 159s) -----------------------------
Status: 500
Detail: Error: Error while building firmware. See stdout/stderr
error  =
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/rq/worker.py", line 1430, in perform_job
    rv = job.perform()
         ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/rq/job.py", line 1280, in perform
    self._result = self._execute()
                   ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/rq/job.py", line 1317, in _execute
    result = self.func(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/asu/build.py", line 233, in build
    report_error(job, "Error while building firmware. See stdout/stderr")
  File "/app/asu/common.py", line 319, in report_error
    raise
RuntimeError: No active exception to reraise

stdout =
Checking 'true'... ok.
Checking 'false'... ok.
Checking 'working-make'... ok.
Checking 'case-sensitive-fs'... ok.
Checking 'proper-umask'... ok.
Checking 'perl-data-dumper'... ok.
Checking 'perl-findbin'... ok.
Checking 'perl-file-copy'... ok.
Checking 'perl-file-compare'... ok.
Checking 'perl-thread-queue'... ok.
Checking 'perl-ipc-cmd'... ok.
Checking 'tar'... ok.
Checking 'find'... ok.
Checking 'bash'... ok.
Checking 'xargs'... ok.
Checking 'patch'... ok.
Checking 'diff'... ok.
Checking 'cp'... ok.
Checking 'seq'... ok.
Checking 'awk'... ok.
Checking 'grep'... ok.
Checking 'egrep'... ok.
Checking 'getopt'... ok.
Checking 'realpath'... ok.
Checking 'stat'... ok.
Checking 'gzip'... ok.
Checking 'unzip'... ok.
Checking 'bzip2'... ok.
Checking 'wget'... ok.
Checking 'install'... ok.
Checking 'perl'... ok.
Checking 'python'... ok.
Checking 'python3'... ok.
Checking 'python3-distutils'... ok.
Checking 'python3-stdlib'... ok.
Checking 'file'... ok.
Checking 'which'... ok.
Checking 'ldconfig-stub'... ok.
Building images for qualcommax - Netgear RAX120v2
Packages: UDPspeeder adguardhome ath11k-firmware-ipq8074 attr auc avahi-dbus-daemon base-files block-mount busybox ca-bundle dnsmasq dropbear e2fsprogs firewall4 fuse3-utils getrandom glib2 ipq-wifi-netgear_rax120v2 jansson kmod-ath kmod-ath11k kmod-ath11k-ahb kmod-crypto-acompress kmod-crypto-aead kmod-crypto-ccm kmod-crypto-cmac kmod-crypto-crc32c kmod-crypto-gcm kmod-crypto-michael-mic kmod-fs-autofs4 kmod-fs-exfat kmod-fs-ext4 kmod-fs-ntfs3 kmod-gpio-button-hotplug kmod-gpio-nxp-74hc164 kmod-hwmon-core kmod-leds-gpio kmod-lib-crc-ccitt kmod-lib-crc32c kmod-lib-lzo kmod-libphy kmod-nf-conntrack kmod-nf-conntrack6 kmod-nf-flow kmod-nf-log kmod-nf-log6 kmod-nf-nat kmod-nf-reject kmod-nf-reject6 kmod-nfnetlink kmod-nft-core kmod-nft-fib kmod-nft-nat kmod-nft-offload kmod-nls-base kmod-phy-aquantia kmod-ppp kmod-pppoe kmod-pppox kmod-qca-nss-dp kmod-qrtr kmod-qrtr-smd kmod-scsi-core kmod-slhc kmod-spi-gpio kmod-thermal kmod-usb-core kmod-usb-dwc3 kmod-usb-dwc3-qcom kmod-usb-storage-extras kmod-usb-storage-uas kmod-usb3 libcomerr libe2p libext2fs libffmpeg-full libjson-script libmbedtls libmnl libnftnl libpcap libpthread librt libsmartcols libss libuclient libustream-mbedtls logd losetup luci luci-app-adblock luci-app-dawn luci-app-minidlna luci-app-samba4 luci-compat luci-i18n-adblock-es luci-i18n-attendedsysupgrade-es luci-i18n-base-es luci-i18n-firewall-es luci-i18n-minidlna-es luci-i18n-opkg-es luci-i18n-samba4-es luci-i18n-usteer-es luci-lib-json luci-proto-relay luci-ssl luci-theme-material mtd ntfs-3g-utils odhcp6c odhcpd-ipv6only openssh-sftp-server opkg owut ppp ppp-mod-pppoe procd procd-seccomp procd-ujail samba4-admin sane-snapscan sshfs tcpdump uboot-envtools uci uclient-fetch urandom-seed urngd vsftpd wpad-mbedtls wsdd2 ath11k-firmware-ipq8074 base-files busybox ca-bundle dnsmasq dropbear e2fsprogs firewall4 ipq-wifi-netgear_rax120v2 kernel kmod-ath11k-ahb kmod-fs-ext4 kmod-gpio-button-hotplug kmod-gpio-nxp-74hc164 kmod-leds-gpio kmod-nft-offload kmod-phy-aquantia kmod-qca-nss-dp kmod-spi-gpio kmod-usb-dwc3 kmod-usb-dwc3-qcom kmod-usb3 libustream-mbedtls logd losetup mtd odhcp6c odhcpd-ipv6only opkg ppp ppp-mod-pppoe procd procd-seccomp procd-ujail uboot-envtools uci uclient-fetch urandom-seed urngd


Installing packages...
Installing libc (1.2.5-r4) to root...
Installing libgcc1 (13.3.0-r4) to root...
/**Downloading and installing packages**/
Configuring ipq-wifi-netgear_rax120v2.
Configuring kmod-scsi-core.
Configuring kmod-nls-base.
Configuring kmod-usb-core.
Configuring kmod-usb-storage.
Configuring terminfo.
Configuring libubox20240329.
Configuring libubus20231128.
Configuring libuci20130104.
Configuring libjson-c5.
Configuring libblobmsg-json20240329.
Configuring rpcd.
Configuring rpcd-mod-file.
Configuring libnl-tiny1.
Configuring rpcd-mod-luci.
Configuring libucode20230711.
Configuring rpcd-mod-ucode.
Configuring cgi-io.
Configuring ucode.
Configuring ucode-mod-fs.
Configuring ucode-mod-uci.
Configuring ucode-mod-ubus.
Configuring ucode-mod-math.
Configuring ucode-mod-html.
Configuring liblucihttp0.
Configuring liblucihttp-ucode.
Configuring luci-base.
Configuring luci-theme-material.
Configuring libpthread.
Configuring librt.
Configuring libatomic1.
Configuring libusb-1.0-0.
Configuring libsane.
Configuring sane-snapscan.
Configuring libiwinfo-data.
Configuring libiwinfo20230701.
Configuring iwinfo.
Configuring openssh-sftp-server.
Configuring libopenssl3.
Configuring libpam.
Configuring jshn.
Configuring jsonfilter.
Configuring coreutils.
Configuring coreutils-sort.
Configuring ca-bundle.
Configuring libuclient20201210.
Configuring uclient-fetch.
Configuring opkg.
Configuring adblock.
Configuring luci-app-adblock.
Configuring liblua5.1.5.
Configuring lua.
Configuring luci-lib-nixio.
Configuring luci-lib-ip.
Configuring luci-lib-jsonc.
Configuring liblucihttp-lua.
Configuring luci-lib-base.
Configuring kmod-fuse.
Configuring ntfs-3g.
Configuring libgpg-error.
Configuring libgcrypt.
Configuring libuuid1.
Configuring ntfs-3g-utils.
Configuring libbz2-1.0.
Configuring libexpat.
Configuring libdbus.
Configuring dbus.
Configuring libavahi-dbus-support.
Configuring libdaemon.
Configuring avahi-dbus-daemon.
Configuring kmod-crypto-gf128.
Configuring luci-app-opkg.
Configuring kmod-usb-storage-uas.
Configuring libopus.
Configuring ubusd.
Configuring ubus.
Configuring iw.
Configuring kmod-crypto-hash.
Configuring kmod-crypto-null.
Configuring kmod-crypto-aead.
Configuring kmod-crypto-manager.
Configuring kmod-input-core.
Configuring kmod-sound-core.
Configuring alsa-lib.
Configuring busybox.
Configuring kmod-crypto-hmac.
Configuring kmod-crypto-sha512.
Configuring kmod-crypto-sha3.
Configuring kmod-crypto-rng.
Configuring kmod-crypto-geniv.
Configuring kmod-crypto-seqiv.
Configuring kmod-crypto-ctr.
Configuring kmod-nfnetlink.
Configuring kmod-nf-reject.
Configuring kmod-nf-reject6.
Configuring kmod-nf-conntrack.
Configuring kmod-nf-conntrack6.
Configuring kmod-nf-nat.
Configuring kmod-nf-log.
Configuring kmod-nf-log6.
Configuring kmod-crypto-crc32c.
Configuring kmod-lib-crc32c.
Configuring kmod-nft-core.
Configuring kmod-nft-fib.
Configuring libudebug.
Configuring netifd.
Configuring ucode-mod-nl80211.
Configuring wifi-scripts.
Configuring wireless-regdb.
Configuring kmod-cfg80211.
Configuring kmod-crypto-cmac.
Configuring kmod-crypto-ccm.
Configuring kmod-crypto-ghash.
Configuring kmod-crypto-gcm.
Configuring hostapd-common.
Configuring kmod-mac80211.
Configuring kmod-ath.
Configuring kmod-crypto-michael-mic.
Configuring kmod-hwmon-core.
Configuring kmod-thermal.
Configuring kmod-ath11k.
Configuring libubus-lua.
Configuring umdns.
Configuring zlib.
Configuring libffi.
Configuring libattr.
Configuring libpcre2.
Configuring glib2.
Configuring libtasn1.
Configuring kmod-usb-dwc3.
Configuring luci-mod-system.
Configuring kmod-nf-flow.
Configuring kmod-lib-crc-ccitt.
Configuring ucode-mod-uloop.
Configuring libmbedtls21.
Configuring px5g-mbedtls.
Configuring getrandom.
Configuring libtirpc.
Configuring libncurses6.
Configuring libreadline8.
Configuring libpopt0.
Configuring libcap.
Configuring libgmp10.
Configuring libnettle8.
Configuring libgnutls.
Configuring liburing.
Configuring attr.
Configuring libavahi-client.
Configuring samba4-libs.
Configuring samba4-server.
Configuring luci-app-samba4.
Configuring luci-i18n-samba4-es.
Configuring kmod-libphy.
Configuring kmod-phy-aquantia.
Configuring luci-theme-bootstrap.
Configuring kmod-slhc.
Configuring kmod-ppp.
Configuring kmod-pppox.
Configuring kmod-pppoe.
Configuring ucode-mod-lua.
Configuring luci-lua-runtime.
Configuring procd-ujail.
Configuring usign.
Configuring openwrt-keyring.
Configuring ubi-utils.
Configuring ubox.
Configuring fstools.
Configuring fwtool.
Configuring base-files.
Configuring kmod-lib-crc16.
Configuring ath11k-firmware-ipq8074.
Configuring relayd.
Configuring libexif.
Configuring libustream-mbedtls20201210.
Configuring kmod-usb-dwc3-qcom.
Configuring libsmartcols1.
Configuring kmod-nft-nat.
Configuring kmod-nft-offload.
Configuring libmnl0.
Configuring libnftnl11.
Configuring jansson4.
Configuring nftables-json.
Configuring firewall4.
Configuring uboot-envtools.
Configuring dnsmasq.
Configuring kmod-usb-xhci-hcd.
Configuring libjson-script20240329.
Configuring procd.
Configuring libflac.
Configuring kmod-crypto-acompress.
Configuring kmod-spi-bitbang.
Configuring kmod-spi-gpio.
Configuring rpcd-mod-iwinfo.
Configuring luci-mod-status.
Configuring kmod-qca-ssdk.
Configuring usteer.
Configuring luci-app-usteer.
Configuring block-mount.
Configuring libogg0.
Configuring libvorbis.
Configuring kmod-usb3.
Configuring kmod-fs-exfat.
Configuring luci-app-firewall.
Configuring libblkid1.
Configuring libsqlite3-0.
Configuring libpcap1.
Configuring tcpdump.
Configuring odhcp6c.
Configuring libfuse3-3.
Configuring fuse3-utils.
Configuring sshfs.
Configuring fdk-aac.
Configuring kmod-gpio-nxp-74hc164.
Configuring uci.
Configuring kmod-fs-ext4.
Configuring dawn.
Configuring losetup.
Configuring luci-proto-ipv6.
Configuring luci-mod-network.
Configuring luci-mod-admin-full.
Configuring luci-proto-ppp.
Configuring rpcd-mod-rrdns.
Configuring uhttpd.
Configuring uhttpd-mod-ubus.
Configuring luci-light.
Configuring luci-ssl.
Configuring dropbear.
Configuring luci-app-dawn.
Configuring luci-compat.
Configuring adguardhome.
Configuring mtd.
Configuring odhcpd-ipv6only.
Configuring procd-seccomp.
Configuring kmod-qrtr.
Configuring kmod-qrtr-smd.
Configuring urandom-seed.
Configuring wsdd2.
Configuring libcomerr0.
Configuring ppp.
Configuring kmod-leds-gpio.
Configuring kmod-gpio-button-hotplug.
Configuring luci-i18n-firewall-es.
Configuring logd.
Configuring kmod-qca-nss-dp.
Configuring ucode-mod-uclient.
Configuring kmod-fs-ntfs3.
Configuring rpcd-mod-rpcsys.
Configuring attendedsysupgrade-common.
Configuring luci-app-attendedsysupgrade.
Configuring libss2.
Configuring libext2fs2.
Configuring libe2p2.
Configuring e2fsprogs.
Configuring luci-i18n-adblock-es.
Configuring libjpeg-turbo.
Configuring ucode-mod-rtnl.
Configuring wpad-mbedtls.
Configuring libv4l.
Configuring lame-lib.
Configuring libffmpeg-full.
Configuring libid3tag.
Configuring minidlna.
Configuring libstdcpp6.
Configuring UDPspeeder.
Configuring luci-app-minidlna.
Configuring luci-proto-relay.
Configuring auc.
Configuring kmod-fs-autofs4.
Configuring luci-i18n-usteer-es.
Configuring luci-lib-json.
Configuring luci.
Configuring samba4-admin.
Configuring owut.
Configuring kmod-ath11k-ahb.
Configuring kmod-lib-lzo.
Configuring luci-i18n-opkg-es.
Configuring luci-i18n-base-es.
Configuring kmod-usb-storage-extras.
Configuring vsftpd.
Configuring luci-i18n-attendedsysupgrade-es.
Configuring urngd.
Configuring ppp-mod-pppoe.
Configuring luci-i18n-minidlna-es.

Finalizing root filesystem...
Create '/builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/etc/filesystems'.
Add 'ntfs-3g' to known filesystems.
Enabling adblock
Enabling adguardhome
Enabling avahi-daemon
Enabling boot
Enabling bootcount
Enabling cron
Enabling dawn
Enabling dbus
Enabling dnsmasq
Enabling done
Enabling dropbear
Enabling firewall
Enabling fstab
Enabling gpio_switch
Enabling led
Enabling log
Enabling minidlna
Enabling network
Enabling odhcpd
Enabling packet_steering
Enabling radius
Enabling relayd
Enabling rpcd
Enabling samba4
Enabling smp_affinity
Enabling sysctl
Enabling sysfixtime
Enabling sysntpd
Enabling system
Enabling ucitrack
Enabling udpspeeder
Enabling uhttpd
Enabling umdns
Enabling umount
Enabling urandom_seed
Enabling urngd
Enabling usteer
Enabling vsftpd
Enabling wpad
Enabling wsdd2

Building images...
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on /builder/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/root.squashfs, block size 262144.
[=============================================================\] 2667/2667 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
        compressed data, compressed metadata, compressed fragments,
        no xattrs, compressed ids
        duplicates are removed
Filesystem size 36355.05 Kbytes (35.50 Mbytes)
        24.16% of uncompressed filesystem size (150462.28 Kbytes)
Inode table size 20980 bytes (20.49 Kbytes)
        21.57% of uncompressed inode table size (97272 bytes)
Directory table size 27620 bytes (26.97 Kbytes)
        42.56% of uncompressed directory table size (64895 bytes)
Number of duplicate files found 312
Number of inodes 2874
Number of files 2288
Number of fragments 208
Number of symbolic links 397
Number of device nodes 1
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 188
Number of hard-links 0
Number of ids (unique uids + gids) 1
Number of uids 1
        unknown (0)
Number of gids 1
        unknown (0)
Image Name:   OpenWrt fake filesystem
Created:      Tue Jun 25 19:42:54 2024
Image Type:   AArch64 Linux Filesystem Image (uncompressed)
Data Size:    0 Bytes = 0.00 KiB = 0.00 MiB
Load Address: 00000000
Entry Point:  00000000
sysupgrade-netgear_rax120v2/
sysupgrade-netgear_rax120v2/CONTROL
sysupgrade-netgear_rax120v2/kernel
sysupgrade-netgear_rax120v2/root
[kernel]
mode=ubi
vol_id=0
vol_type=dynamic
vol_name=kernel
image=/builder/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/netgear_rax120v2-uImage.itb
[rootfs]
mode=ubi
vol_id=1
vol_type=dynamic
vol_name=rootfs
image=/builder/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/root.squashfs
vol_size=37228544
[rootfs_data]
mode=ubi
vol_id=2
vol_type=dynamic
vol_name=rootfs_data
vol_size=1MiB
vol_flags=autoresize
ubinize: volume size was not specified in section "kernel", assume minimum to fit image "/builder/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/netgear_rax120v2-uImage.itb"5364596 bytes (5.1 MiB)

stderr =
Generate local signing keys...
Generate local certificate...
Package list missing or not up-to-date, generating it.

Building package index...
Downloading https://downloads.openwrt.org/snapshots/targets/qualcommax/ipq807x/packages/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_core
Downloading https://downloads.openwrt.org/snapshots/targets/qualcommax/ipq807x/packages/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/base/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_base
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/base/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/snapshots/targets/qualcommax/ipq807x/kmods/6.6.35-1-f37c2fdbdb753b455ef8cac06b859604/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_kmods
Downloading https://downloads.openwrt.org/snapshots/targets/qualcommax/ipq807x/kmods/6.6.35-1-f37c2fdbdb753b455ef8cac06b859604/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/luci/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_luci
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/luci/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/packages/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_packages
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/packages/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/routing/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_routing
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/routing/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/telephony/Packages.gz
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/openwrt_telephony
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/telephony/Packages.sig
Signature check passed.
Downloading file:packages/Packages
Updated list of available packages in /builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/../../../../builder/dl/imagebuilder
Downloading file:packages/Packages.sig
Signature check passed.
/builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/etc/init.d/usteer: line 10: /lib/functions/network.sh: No such file or directory
/builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/etc/init.d/usteer: line 11: /usr/share/libubox/jshn.sh: No such file or directory
/builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/etc/init.d/usteer: line 12: /lib/functions.sh: No such file or directory
./etc/init.d/usteer: line 10: /lib/functions/network.sh: No such file or directory
./etc/init.d/usteer: line 11: /usr/share/libubox/jshn.sh: No such file or directory
./etc/init.d/usteer: line 12: /lib/functions.sh: No such file or directory
Pseudo file "dev" exists in source filesystem "/builder/build_dir/target-aarch64_cortex-a53_musl/root-qualcommax/dev".
Ignoring, exclude it (-e/-ef) to override.
10477+1 records in
10477+1 records out
5364596 bytes (5.4 MB, 5.1 MiB) copied, 0.0501193 s, 107 MB/s
0+1 records in
1+0 records out
5373888 bytes (5.4 MB, 5.1 MiB) copied, 0.00743924 s, 722 MB/s
36355+1 records in
36356+0 records out
37228544 bytes (37 MB, 36 MiB) copied, 0.151445 s, 246 MB/s
/builder/staging_dir/host/bin/tar: Option --mtime: Treating date '@1719344574' as 2024-06-25 19:42:54
cat: OpenWrt: No such file or directory
cat: OpenWrt: No such file or directory
cp: cannot stat '/builder/d030386f7cd46115a6be623a9c182a6b/openwrt-6f67fa131d74-qualcommax-ipq807x-netgear_rax120v2-initramfs-uImage.itb': No such file or directory
make[3]: *** [Makefile:43: /builder/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/tmp/openwrt-6f67fa131d74-qualcommax-ipq807x-netgear_rax120v2-squashfs-web-ui-factory.img] Error 1
make[2]: *** [Makefile:247: build_image] Error 2
make[1]: *** [Makefile:153: _call_image] Error 2
make: *** [Makefile:310: image] Error 2

ERROR: Build failed with status 500
The above errors are often due to the upgrade server lagging behind the
build server, first suggestion is to wait a while and try again.

Looks like it!

I have no issue moving to the new Owut option, but I'm curious what happens with the old asu/auc upgrade path:

  • will asu/auc fail when it cannot find it's own package in the process of requesting a custom image?
  • or will it succeed and install a new image that simply lacks asu/auc once the packages have been removed from the package lists?
  • or will it know to install Owrt?

Yes, that's the one I expect right now. auc will request a build containing auc and the ASU server will fail with 'invalid package selection' or whatever its error is.

I've been looking at the package_changes that the ASU server provides, which allows such things as "replace fw3 with fw4 if you are crossing the r18611 boundary". I'm going to see if I can get a PR into the ASU server so the auc -> owut change happens automatically in the same way (and since auc implements the package change functionality, it should work for upgrades using it).

EDIT
PR submitted https://github.com/openwrt/asu/pull/868/files

2 Likes

I replicated your build (but without any extra packages) and got the same thing. Then tried with x86 and it worked fine, so...

Regrettably, I can't figure out how to make this one go away. The imagebuilder on the ASU server is broken somehow for that specific device, so the best I can do is that final message:

The above errors are often due to the upgrade server lagging behind the
build server, first suggestion is to wait a while and try again.

Once the buildbots create a new imagebuilder (you will see the Version-to SNAPSHOT r26792-646ebbd32c (kernel 6.6.35) "r-number" bump up beyond 26792), then it should (SHOULD!) work. Fingers crossed.

EDIT
I'm going to see if I can detect that specific class of error in the response logs and add my final paragraph-ish to the owut output... https://github.com/efahl/owut/issues/14

1 Like

uclient-fetch works.

root@wifiap1:~# uclient-fetch https://sysupgrade.openwrt.org/json/v1/overview.json
Downloading 'https://sysupgrade.openwrt.org/json/v1/overview.json'
Connecting to 206.81.21.114:443
Writing to 'overview.json'
overview.json        100% |*******************************| 17634   0:00:00 ETA
Download completed (17634 bytes)
root@wifiap-conroe1:~# owut check --verbose
owut - OpenWrt Upgrade Tool 2024.06.24~1b222bdc-r1 (/usr/bin/owut)
ERROR: uclient error code=1
Response status: 0
ERROR: Could not download 'https://sysupgrade.openwrt.org/json/v1/overview.json', terminating.
root@wifiap1:~#

Hmm. I'm pretty sure that error code 1 is UCLIENT_ERROR_CONNECT (https://github.com/openwrt/uclient/blob/master/uclient.h#L34), but it's being reported several calls past where we actually connect.

I should have thought of this earlier: could you run again with an extra verbose, see if we're getting back any headers?

$ owut check -v -v
...
Downloaded https://sysupgrade.openwrt.org/json/v1/overview.json to /tmp/owut-overview.json
Response headers = {
  "accept-ranges": "bytes",
  "access-control-allow-headers": "*",
  "access-control-allow-methods": "POST, GET, OPTIONS",
  "access-control-allow-origin": "*",
  "alt-svc": "h3=\":443\"; ma=2592000",
  "content-length": "17634",
  "content-type": "application/json",
  "etag": "\"d2a6it85fz7bdlu\"",
  "last-modified": "Wed, 26 Jun 2024 19:01:07 GMT",
  "server": "Caddy",
  "vary": "Accept-Encoding",
  "date": "Wed, 26 Jun 2024 19:04:41 GMT",
  "status": 200
}

No change in output or result.

root@wifiap-conroe1:~# owut check -v -v
owut - OpenWrt Upgrade Tool 2024.06.24~1b222bdc-r1 (/usr/bin/owut)
ERROR: uclient error code=1
Response status: 0
ERROR: Could not download 'https://sysupgrade.openwrt.org/json/v1/overview.json', terminating.

EDIT: ran a tcpdump. curl produces traffic to asu-01.infra.openwrt.org.443, owut unfortunately does not put anything on the line.

Oho, good catch! I'm guessing something with a redirect is failing in owut. I'll see if I can chase anything down that rabbit hole... (Strange that it only happens on one of your devices, seems like you'd get the same redirect to a regional CDN from the same network, doesn't it?)

@aparcar
Ok, I found it. The IPv6 address for sysupgrade.openwrt.org site is bad.

Using GNU wget -4 ... works fine, wget -6 ... hangs forever (well, I killed it after three minutes, so close enough to forever).

Also

$ nslookup sysupgrade.openwrt.org
Non-authoritative answer:
sysupgrade.openwrt.org  canonical name = asu-01.infra.openwrt.org
Name:   asu-01.infra.openwrt.org
Address: 206.81.21.114

Non-authoritative answer:
sysupgrade.openwrt.org  canonical name = asu-01.infra.openwrt.org
Name:   asu-01.infra.openwrt.org
Address: 2a03:b0c0:3:d0::1574:1

$ ping 206.81.21.114
PING 206.81.21.114 (206.81.21.114): 56 data bytes
64 bytes from 206.81.21.114: seq=0 ttl=47 time=175.577 ms
64 bytes from 206.81.21.114: seq=1 ttl=47 time=172.185 ms

$ ping 2a03:b0c0:3:d0::1574:1
PING 2a03:b0c0:3:d0::1574:1 (2a03:b0c0:3:d0::1574:1): 56 data bytes
< hangs forever >
1 Like

try a traceroute to the v6 address and see if it breaks early or close to the destination LAN. You might need a working trace from someone to compare to though.

1 Like

Thanks, Daniel is looking at it, too, over on the devel list. He did a traceroute on his end and it works fine for him, so someone has a bad entry in BGP or something...

1 Like

@cdes, what's your gross geography. Same as me in North America?

Could you (or anyone else who has the time) run a traceroute or tracepath to that address and report back? (tracepath output is prettier, but traceroute is just fine, too. Host doesn't matter either, any computer or router where you've got a trace* command.)

MAKE SURE TO SANITIZE those first few addresses, as they'll be your local devices.

tracepath -6 2a03:b0c0:3:d0::1574:1

EDIT
As of 2024-06-26 17:33 PDT the routes all seem to be working again...

 6:  las-b23-link.ip.twelve99.net                         16.024ms
 7:  rest-bb1-v6.ip.twelve99.net                          72.928ms
 8:  nyk-bb1-v6.ip.twelve99.net                           89.371ms
 9:  ldn-bb2-v6.ip.twelve99.net                          150.534ms asymm 11
10:  slou-b2-v6.ip.twelve99.net                          149.089ms
11:  digitalocean-ic-378006.ip.twelve99-cust.net         151.966ms
12:  2a03:b0c0:fffe::92                                  148.894ms
13:  no reply
14:  no reply
15:  no reply
16:  no reply
17:  asu-01.infra.openwrt.org                            170.037ms reached
     Resume: pmtu 1500 hops 17 back 17
1 Like

traceroute to 2a03:b0c0:3:d0::1574:1 (2a03:b0c0:3:d0::1574:1) , 5 relative hops max, 52 byte packets
1 asu-01.infra.openwrt.org (2a03:b0c0:3:d0::1574:1) 21.277 ms 23.473 ms 24.971 ms

I think this will Not help

1 Like

Can I still specify the script in /rom?

Yup, that works fine:

owut upgrade --init-script /rom/etc/uci-defaults/99-asu-defaults

And when you decide that the /rom version needs modification:

cp /rom/etc/uci-defaults/99-asu-defaults /tmp
vi /tmp/99-asu-defaults
... make your changes ...
owut upgrade --init-script /tmp/99-asu-defaults

And if you want to reduce typing and memory load, just set it as default:

uci set attendedsysupgrade.owut.init_script=/rom/etc/uci-defaults/99-asu-defaults
uci commit

Then, when you use just owut upgrade, it defaults to /rom, but if you say owut --init-script /some/where/else then that overrides the config and uses your CLI one.

1 Like

Just to be clear, it will only default to /rom after you've set the value in the config. I've edited my post above, as I thought it was sort of ambiguous...