Build hangs: possible apk issue?

I'm trying to build firmware for my R7800, and I'm running into an issue where the build just hangs once it reaches the package/install stage. Gnome System Monitor showed that a process named apk was maxing out a core, and I left it overnight to see whether it would finish building, but it remained as it was in the morning. I have pulled the last main build of OpenWrt, I've updated the feeds, and I've tried to do a distclean, but I'm still experiencing this issue. When I run make with the V=sc option things hang once it reaches this output:

Index has 200 packages (of which 200 are new)
find /var/home/michael/src/router_stuff/openwrt/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/root-ipq806x -type d | xargs -r chmod 0755
rm -rf /var/home/michael/src/router_stuff/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/root-ipq806x /var/home/michael/src/router_stuff/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/root.orig-ipq806x
mkdir -p /var/home/michael/src/router_stuff/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/root-ipq806x/tmp
IPKG_INSTROOT=/var/home/michael/src/router_stuff/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/root-ipq806x /var/home/michael/src/router_stuff/openwrt/staging_dir/host/bin/fakeroot /var/home/michael/src/router_stuff/openwrt/staging_dir/host/bin/apk --root /var/home/michael/src/router_stuff/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/root-ipq806x --keys-dir /var/home/michael/src/router_stuff/openwrt --no-logfile --preserve-env add --no-cache --initdb --no-scripts --arch arm_cortex-a15_neon-vfpv4 \
	--repositories-file /dev/null --repository file:///var/home/michael/src/router_stuff/openwrt/staging_dir/packages/ipq806x/packages.adb \
	$(cat /var/home/michael/src/router_stuff/openwrt/tmp/apk_install_list) \
		"base-files=1656~355866fc11" \
		"libc=1.2.5-r4" \
		"kernel=6.6.87~2f3123518de27688e910b40f200ec22e-r1"

I guess based on this, and the fact that a process named apk keeps maxing out a core, that I'm experiencing an issue with apk. Here's the diffconfig for my build:

diffconfig
CONFIG_TARGET_ipq806x=y
CONFIG_TARGET_ipq806x_generic=y
CONFIG_TARGET_ipq806x_generic_DEVICE_netgear_r7800=y
CONFIG_ATH10K_LEDS=y
CONFIG_HTOP_LMSENSORS=y
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_PREFER_CHACHA_OVER_GCM=y
CONFIG_OPENSSL_WITH_ASM=y
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
CONFIG_OPENSSL_WITH_CMS=y
CONFIG_OPENSSL_WITH_DEPRECATED=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_IDEA=y
CONFIG_OPENSSL_WITH_MDC2=y
CONFIG_OPENSSL_WITH_PSK=y
CONFIG_OPENSSL_WITH_SEED=y
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_TLS13=y
CONFIG_OPENSSL_WITH_WHIRLPOOL=y
CONFIG_PACKAGE_ath10k-firmware-qca9984=y
# CONFIG_PACKAGE_ath10k-firmware-qca9984-ct is not set
CONFIG_PACKAGE_bcp38=y
CONFIG_PACKAGE_cgi-io=y
CONFIG_PACKAGE_hostapd-utils=y
CONFIG_PACKAGE_htop=y
CONFIG_PACKAGE_ip-tiny=y
CONFIG_PACKAGE_iptables-mod-ipopt=y
CONFIG_PACKAGE_iptables-nft=y
CONFIG_PACKAGE_kmod-ath10k=y
# CONFIG_PACKAGE_kmod-ath10k-ct is not set
CONFIG_PACKAGE_kmod-crypto-kpp=y
CONFIG_PACKAGE_kmod-crypto-lib-chacha20=y
CONFIG_PACKAGE_kmod-crypto-lib-chacha20poly1305=y
CONFIG_PACKAGE_kmod-crypto-lib-curve25519=y
CONFIG_PACKAGE_kmod-crypto-lib-poly1305=y
# CONFIG_PACKAGE_kmod-hwmon-core is not set
CONFIG_PACKAGE_kmod-ifb=y
CONFIG_PACKAGE_kmod-ipt-core=y
CONFIG_PACKAGE_kmod-ipt-ipopt=y
CONFIG_PACKAGE_kmod-nf-ipt=y
CONFIG_PACKAGE_kmod-nft-compat=y
CONFIG_PACKAGE_kmod-sched-cake=y
CONFIG_PACKAGE_kmod-sched-core=y
CONFIG_PACKAGE_kmod-udptunnel4=y
CONFIG_PACKAGE_kmod-udptunnel6=y
CONFIG_PACKAGE_kmod-wireguard=y
CONFIG_PACKAGE_libatomic=y
CONFIG_PACKAGE_libiptext=y
CONFIG_PACKAGE_libiptext-nft=y
CONFIG_PACKAGE_libiptext6=y
CONFIG_PACKAGE_liblucihttp=y
CONFIG_PACKAGE_liblucihttp-ucode=y
CONFIG_PACKAGE_libncurses=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libopenssl-conf=y
CONFIG_PACKAGE_libopenssl-legacy=y
CONFIG_PACKAGE_libustream-mbedtls=m
CONFIG_PACKAGE_libustream-openssl=y
CONFIG_PACKAGE_libxtables=y
CONFIG_PACKAGE_luci-app-bcp38=y
CONFIG_PACKAGE_luci-app-firewall=y
CONFIG_PACKAGE_luci-app-package-manager=y
CONFIG_PACKAGE_luci-app-sqm=y
CONFIG_PACKAGE_luci-base=y
CONFIG_PACKAGE_luci-lib-uqr=y
CONFIG_PACKAGE_luci-light=y
CONFIG_PACKAGE_luci-mod-admin-full=y
CONFIG_PACKAGE_luci-mod-network=y
CONFIG_PACKAGE_luci-mod-status=y
CONFIG_PACKAGE_luci-mod-system=y
CONFIG_PACKAGE_luci-proto-ipv6=y
CONFIG_PACKAGE_luci-proto-ppp=y
CONFIG_PACKAGE_luci-proto-wireguard=y
CONFIG_PACKAGE_luci-ssl-openssl=y
CONFIG_PACKAGE_luci-theme-bootstrap=y
CONFIG_PACKAGE_openssl-util=y
CONFIG_PACKAGE_resolveip=y
CONFIG_PACKAGE_rpcd=y
CONFIG_PACKAGE_rpcd-mod-file=y
CONFIG_PACKAGE_rpcd-mod-iwinfo=y
CONFIG_PACKAGE_rpcd-mod-luci=y
CONFIG_PACKAGE_rpcd-mod-rrdns=y
CONFIG_PACKAGE_rpcd-mod-ucode=y
CONFIG_PACKAGE_sqm-scripts=y
CONFIG_PACKAGE_tc-tiny=y
CONFIG_PACKAGE_terminfo=y
CONFIG_PACKAGE_ucode-mod-html=y
CONFIG_PACKAGE_ucode-mod-math=y
CONFIG_PACKAGE_uhttpd=y
CONFIG_PACKAGE_uhttpd-mod-ubus=y
CONFIG_PACKAGE_wireguard-tools=y
# CONFIG_PACKAGE_wpad-basic-mbedtls is not set
CONFIG_PACKAGE_wpad-openssl=y
CONFIG_PACKAGE_xtables-nft=y
CONFIG_WPA_MSG_MIN_PRIORITY=0

Are you by chance using GCC15 on your host?

Cause, I am having the same issue if host APK is built with GCC15.

As soon as APK is built by GCC14 or older it works just fine.

So, I am having a go at bisecting the exact GCC commit that causes this but its rather hard to do as there have been other breakages during GCC15 development

2 Likes

Yup, I'm on Fedora 42 beta, so it's GCC15 over here. Thanks for looking into this!

1 Like

Just want to add that I downgraded to GCC14, and now I'm able to compile a build.

Oh, thanks for finding this. I was going to update my Fedora boxes to 42 soon, so I'll run into this, too...

1 Like

Yeah, I am on Fedora 42, I managed to fixup tooling, toolchain and packages in Fedora to compile with GCC15 (This is all merged now) but APK is still broken.

I am trying to bisect as to what in GCC broke it but the list is long

Well, I at least narrowed this down to an LTO issue, cause as soon as LTO is disabled then APK works just fine.

Upstream was able to figure out the issue, will be fixed via:

2 Likes

Saw that PR, upgraded my main workstation to Fedora 42, pulled all the new code and built without any issue. :+1:t3:

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.