Recent builds of OpenWrt do not boot on RPi5B

For Developers
1

I built r28739-69890e16b37d for RPi5B and found that I cannot boot. The boot process hangs up and does not let me see a login prompt nor bring up the network. The fan on the heatsink just runs at full throttle.

What I see on the screen:

...
block: attempting to load /tmp/overlay/upper/etc/config/fstab
block: unable to load configuration (fstab: Entry not found)
block: attempting to load /tmp/overlay/etc/config/fstab
block: unable to load configuration (fstab: Entry not found)
block: attempting to load /etc/config/fstab
block: unable to load configuration (fstab: Entry not found)
block: no usable configuration

Last-known-good build was r28670-0fd9d00cd6fc. I do not have the time to bisect now, so any educated guesses are welcomed.

config.buildinfo 
CONFIG_TARGET_bcm27xx=y
CONFIG_TARGET_bcm27xx_bcm2712=y
CONFIG_TARGET_bcm27xx_bcm2712_DEVICE_rpi-5=y
CONFIG_DEVEL=y
CONFIG_TOOLCHAINOPTS=y
CONFIG_BUSYBOX_CUSTOM=y
CONFIG_BIND_ENABLE_DOH=y
# CONFIG_BINUTILS_USE_VERSION_2_42 is not set
CONFIG_BINUTILS_USE_VERSION_2_43=y
CONFIG_BINUTILS_VERSION="2.43.1"
CONFIG_BINUTILS_VERSION_2_43=y
CONFIG_BUSYBOX_CONFIG_EXTRA_CFLAGS="-mcpu=cortex-a76 -O3 -pipe"
CONFIG_BUSYBOX_CONFIG_FEATURE_GETOPT_LONG=y
CONFIG_BUSYBOX_CONFIG_FEATURE_SEAMLESS_XZ=y
CONFIG_BUSYBOX_CONFIG_FEATURE_SORT_BIG=y
CONFIG_BUSYBOX_CONFIG_FEATURE_TAR_LONG_OPTIONS=y
CONFIG_BUSYBOX_CONFIG_GETOPT=y
CONFIG_BUSYBOX_CONFIG_MOUNTPOINT=y
CONFIG_BUSYBOX_CONFIG_UNXZ=y
CONFIG_BUSYBOX_CONFIG_XZ=y
# CONFIG_GCC_USE_VERSION_13 is not set
CONFIG_GCC_USE_VERSION_14=y
CONFIG_GCC_VERSION="14.2.0"
CONFIG_GCC_VERSION_14=y
CONFIG_GNUTLS_ALPN=y
CONFIG_GNUTLS_ANON=y
CONFIG_GNUTLS_DTLS_SRTP=y
CONFIG_GNUTLS_HEARTBEAT=y
CONFIG_GNUTLS_OCSP=y
CONFIG_GNUTLS_PSK=y
CONFIG_HTOP_LMSENSORS=y
CONFIG_KERNEL_ARM64_16K_PAGES=y
# CONFIG_KERNEL_ARM64_4K_PAGES is not set
CONFIG_KERNEL_ARM64_VA_BITS=47
CONFIG_KERNEL_ARM64_VA_BITS_47=y
# CONFIG_KERNEL_ARM64_VA_BITS_48 is not set
CONFIG_KERNEL_ARM_PMU=y
CONFIG_KERNEL_ARM_PMUV3=y
CONFIG_KERNEL_CFLAGS="-mcpu=cortex-a76 -O2 -pipe"
CONFIG_KERNEL_CGROUP_DEVICE=y
CONFIG_KERNEL_PERF_EVENTS=y
CONFIG_KERNEL_PROC_PAGE_MONITOR=y
CONFIG_KERNEL_PSI=y
CONFIG_KERNEL_RPI_AXIPERF=y
# CONFIG_LUA_ECO_MBEDTLS is not set
CONFIG_LUA_ECO_OPENSSL=y
CONFIG_LXC_BUSYBOX_OPTIONS=y
CONFIG_LXC_KERNEL_OPTIONS=y
CONFIG_LXC_NETWORKING=y
CONFIG_LXC_SECCOMP=y
CONFIG_OPENSSH_LIBFIDO2=y
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_OPTIMIZE_SPEED=y
CONFIG_OPENSSL_WITH_ASM=y
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
CONFIG_OPENSSL_WITH_CMS=y
CONFIG_OPENSSL_WITH_DEPRECATED=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_IDEA=y
CONFIG_OPENSSL_WITH_MDC2=y
CONFIG_OPENSSL_WITH_PSK=y
CONFIG_OPENSSL_WITH_SEED=y
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_TLS13=y
CONFIG_OPENSSL_WITH_WHIRLPOOL=y
CONFIG_PACKAGE_TAR_BZIP2=y
CONFIG_PACKAGE_TAR_GZIP=y
CONFIG_PACKAGE_TAR_POSIX_ACL=y
CONFIG_PACKAGE_TAR_XATTR=y
CONFIG_PACKAGE_TAR_XZ=y
CONFIG_PACKAGE_TAR_ZSTD=y
CONFIG_PACKAGE_acl=y
# CONFIG_PACKAGE_apk-mbedtls is not set
CONFIG_PACKAGE_apk-openssl=y
CONFIG_PACKAGE_attendedsysupgrade-common=y
CONFIG_PACKAGE_attr=y
CONFIG_PACKAGE_avahi-dbus-daemon=y
CONFIG_PACKAGE_avahi-utils=y
CONFIG_PACKAGE_bind-dig=y
CONFIG_PACKAGE_bind-libs=y
CONFIG_PACKAGE_blkid=y
CONFIG_PACKAGE_block-mount=y
CONFIG_PACKAGE_blockdev=y
CONFIG_PACKAGE_bzip2=y
CONFIG_PACKAGE_cgi-io=y
CONFIG_PACKAGE_coreutils=y
CONFIG_PACKAGE_coreutils-dd=y
CONFIG_PACKAGE_coreutils-df=y
CONFIG_PACKAGE_coreutils-du=y
CONFIG_PACKAGE_coreutils-kill=y
CONFIG_PACKAGE_coreutils-ls=y
CONFIG_PACKAGE_coreutils-numfmt=y
CONFIG_PACKAGE_dbus=y
CONFIG_PACKAGE_dosfstools=y
# CONFIG_PACKAGE_dropbear is not set
CONFIG_PACKAGE_ethtool=y
CONFIG_PACKAGE_fdisk=y
CONFIG_PACKAGE_flock=y
CONFIG_PACKAGE_fscrypt=y
CONFIG_PACKAGE_fstrim=y
CONFIG_PACKAGE_getopt=y
CONFIG_PACKAGE_grep=y
# CONFIG_PACKAGE_hostapd-common is not set
CONFIG_PACKAGE_htop=y
CONFIG_PACKAGE_ip-full=y
CONFIG_PACKAGE_iperf3=y
CONFIG_PACKAGE_iptables-mod-ipopt=y
CONFIG_PACKAGE_iptables-zz-legacy=y
CONFIG_PACKAGE_kmod-crypto-kpp=y
CONFIG_PACKAGE_kmod-crypto-lib-chacha20=y
CONFIG_PACKAGE_kmod-crypto-lib-chacha20poly1305=y
CONFIG_PACKAGE_kmod-crypto-lib-curve25519=y
CONFIG_PACKAGE_kmod-crypto-lib-poly1305=y
CONFIG_PACKAGE_kmod-crypto-sha256=y
CONFIG_PACKAGE_kmod-fs-ext4=y
CONFIG_PACKAGE_kmod-ifb=y
CONFIG_PACKAGE_kmod-ikconfig=y
CONFIG_PACKAGE_kmod-inet-diag=y
CONFIG_PACKAGE_kmod-ip6tables=y
CONFIG_PACKAGE_kmod-ip6tables-extra=y
CONFIG_PACKAGE_kmod-ipt-checksum=y
CONFIG_PACKAGE_kmod-ipt-conntrack=y
CONFIG_PACKAGE_kmod-ipt-conntrack-extra=y
CONFIG_PACKAGE_kmod-ipt-conntrack-label=y
CONFIG_PACKAGE_kmod-ipt-core=y
CONFIG_PACKAGE_kmod-ipt-extra=y
CONFIG_PACKAGE_kmod-ipt-ipopt=y
CONFIG_PACKAGE_kmod-ipt-raw=y
CONFIG_PACKAGE_kmod-lib-crc16=y
CONFIG_PACKAGE_kmod-libphy=y
CONFIG_PACKAGE_kmod-macvlan=y
CONFIG_PACKAGE_kmod-mii=y
CONFIG_PACKAGE_kmod-netlink-diag=y
CONFIG_PACKAGE_kmod-nf-conncount=y
CONFIG_PACKAGE_kmod-nf-conntrack-netlink=y
CONFIG_PACKAGE_kmod-nf-ipt=y
CONFIG_PACKAGE_kmod-nf-ipt6=y
CONFIG_PACKAGE_kmod-nfnetlink-queue=y
CONFIG_PACKAGE_kmod-nft-compat=y
CONFIG_PACKAGE_kmod-nft-queue=y
CONFIG_PACKAGE_kmod-packet-diag=y
CONFIG_PACKAGE_kmod-sched-cake=y
CONFIG_PACKAGE_kmod-sched-core=y
CONFIG_PACKAGE_kmod-scsi-core=y
# CONFIG_PACKAGE_kmod-sound-core is not set
CONFIG_PACKAGE_kmod-udptunnel4=y
CONFIG_PACKAGE_kmod-udptunnel6=y
CONFIG_PACKAGE_kmod-unix-diag=y
CONFIG_PACKAGE_kmod-usb-net=y
CONFIG_PACKAGE_kmod-usb-net-asix-ax88179=y
CONFIG_PACKAGE_kmod-usb-net-cdc-ether=y
CONFIG_PACKAGE_kmod-usb-net-cdc-ncm=y
CONFIG_PACKAGE_kmod-usb-net-rtl8150=y
CONFIG_PACKAGE_kmod-usb-net-rtl8152=y
CONFIG_PACKAGE_kmod-usb-storage=y
CONFIG_PACKAGE_kmod-veth=y
CONFIG_PACKAGE_kmod-wireguard=y
CONFIG_PACKAGE_less=y
CONFIG_PACKAGE_libacl=y
CONFIG_PACKAGE_libatomic=y
CONFIG_PACKAGE_libattr=y
CONFIG_PACKAGE_libavahi-client=y
CONFIG_PACKAGE_libavahi-dbus-support=y
CONFIG_PACKAGE_libbpf=y
CONFIG_PACKAGE_libbz2=y
CONFIG_PACKAGE_libcap=y
CONFIG_PACKAGE_libcbor=y
CONFIG_PACKAGE_libdaemon=y
CONFIG_PACKAGE_libdaq3=y
CONFIG_PACKAGE_libdbus=y
CONFIG_PACKAGE_libdnet=y
CONFIG_PACKAGE_libelf=y
CONFIG_PACKAGE_libevdev=y
CONFIG_PACKAGE_libexpat=y
# CONFIG_PACKAGE_libf2fs is not set
CONFIG_PACKAGE_libfdisk=y
CONFIG_PACKAGE_libfido2=y
CONFIG_PACKAGE_libgdbm=y
CONFIG_PACKAGE_libgmp=y
CONFIG_PACKAGE_libgnutls=y
CONFIG_PACKAGE_libhwloc=y
CONFIG_PACKAGE_libip4tc=y
CONFIG_PACKAGE_libip6tc=y
CONFIG_PACKAGE_libiperf3=y
CONFIG_PACKAGE_libiptext=y
CONFIG_PACKAGE_libiptext-nft=y
CONFIG_PACKAGE_libiptext6=y
CONFIG_PACKAGE_libiwinfo-lua=y
CONFIG_PACKAGE_libkmod=y
CONFIG_PACKAGE_liblua=y
CONFIG_PACKAGE_liblucihttp=y
CONFIG_PACKAGE_liblucihttp-lua=y
CONFIG_PACKAGE_liblucihttp-ucode=y
CONFIG_PACKAGE_liblxc=y
CONFIG_PACKAGE_liblzma=y
CONFIG_PACKAGE_libmbedtls=m
CONFIG_PACKAGE_libmount=y
CONFIG_PACKAGE_libncurses=y
CONFIG_PACKAGE_libnetfilter-queue=y
CONFIG_PACKAGE_libnettle=y
CONFIG_PACKAGE_libnfnetlink=y
CONFIG_PACKAGE_libnghttp2=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libpam=y
CONFIG_PACKAGE_libpcap=y
CONFIG_PACKAGE_libpci=y
CONFIG_PACKAGE_libpciaccess=y
CONFIG_PACKAGE_libpcre2=y
CONFIG_PACKAGE_libpopt=y
CONFIG_PACKAGE_libreadline=y
CONFIG_PACKAGE_libseccomp=y
CONFIG_PACKAGE_libsensors=y
CONFIG_PACKAGE_libsqlite3=y
CONFIG_PACKAGE_libstdcpp=y
CONFIG_PACKAGE_libsysfs=y
CONFIG_PACKAGE_libtasn1=y
CONFIG_PACKAGE_libtirpc=y
CONFIG_PACKAGE_libubus-lua=y
CONFIG_PACKAGE_libuci-lua=y
CONFIG_PACKAGE_libudev-zero=y
CONFIG_PACKAGE_libunwind=y
CONFIG_PACKAGE_liburcu=y
CONFIG_PACKAGE_liburing=y
CONFIG_PACKAGE_libusb-1.0=y
CONFIG_PACKAGE_libustream-mbedtls=m
CONFIG_PACKAGE_libustream-openssl=y
CONFIG_PACKAGE_libuv=y
CONFIG_PACKAGE_libxtables=y
CONFIG_PACKAGE_libzstd=y
CONFIG_PACKAGE_lm-sensors=y
CONFIG_PACKAGE_lsblk=y
CONFIG_PACKAGE_lscpu=y
CONFIG_PACKAGE_lsof=y
CONFIG_PACKAGE_lua=y
CONFIG_PACKAGE_luajit=y
CONFIG_PACKAGE_luci=y
CONFIG_PACKAGE_luci-app-attendedsysupgrade=y
CONFIG_PACKAGE_luci-app-firewall=y
CONFIG_PACKAGE_luci-app-lxc=y
CONFIG_PACKAGE_luci-app-nlbwmon=y
CONFIG_PACKAGE_luci-app-package-manager=y
CONFIG_PACKAGE_luci-app-pbr=y
CONFIG_PACKAGE_luci-app-samba4=y
CONFIG_PACKAGE_luci-app-sqm=y
CONFIG_PACKAGE_luci-base=y
CONFIG_PACKAGE_luci-compat=y
CONFIG_PACKAGE_luci-lib-base=y
CONFIG_PACKAGE_luci-lib-chartjs=y
CONFIG_PACKAGE_luci-lib-ip=y
CONFIG_PACKAGE_luci-lib-jsonc=y
CONFIG_PACKAGE_luci-lib-nixio=y
CONFIG_PACKAGE_luci-lib-uqr=y
CONFIG_PACKAGE_luci-light=y
CONFIG_PACKAGE_luci-lua-runtime=y
CONFIG_PACKAGE_luci-mod-admin-full=y
CONFIG_PACKAGE_luci-mod-network=y
CONFIG_PACKAGE_luci-mod-status=y
CONFIG_PACKAGE_luci-mod-system=y
CONFIG_PACKAGE_luci-proto-ipv6=y
CONFIG_PACKAGE_luci-proto-ppp=y
CONFIG_PACKAGE_luci-proto-wireguard=y
CONFIG_PACKAGE_luci-theme-bootstrap=y
CONFIG_PACKAGE_lxc=y
CONFIG_PACKAGE_lxc-attach=y
CONFIG_PACKAGE_lxc-auto=y
CONFIG_PACKAGE_lxc-autostart=y
CONFIG_PACKAGE_lxc-cgroup=y
CONFIG_PACKAGE_lxc-checkconfig=y
CONFIG_PACKAGE_lxc-common=y
CONFIG_PACKAGE_lxc-config=y
CONFIG_PACKAGE_lxc-configs=y
CONFIG_PACKAGE_lxc-console=y
CONFIG_PACKAGE_lxc-copy=y
CONFIG_PACKAGE_lxc-create=y
CONFIG_PACKAGE_lxc-destroy=y
CONFIG_PACKAGE_lxc-device=y
CONFIG_PACKAGE_lxc-execute=y
CONFIG_PACKAGE_lxc-freeze=y
CONFIG_PACKAGE_lxc-hooks=y
CONFIG_PACKAGE_lxc-info=y
CONFIG_PACKAGE_lxc-init=y
CONFIG_PACKAGE_lxc-ls=y
CONFIG_PACKAGE_lxc-snapshot=y
CONFIG_PACKAGE_lxc-start=y
CONFIG_PACKAGE_lxc-stop=y
CONFIG_PACKAGE_lxc-templates=y
CONFIG_PACKAGE_lxc-top=y
CONFIG_PACKAGE_lxc-usernsexec=y
# CONFIG_PACKAGE_mkf2fs is not set
CONFIG_PACKAGE_mount-utils=y
CONFIG_PACKAGE_ncdu=y
CONFIG_PACKAGE_nlbwmon=y
# CONFIG_PACKAGE_odhcp6c is not set
# CONFIG_PACKAGE_odhcpd-ipv6only is not set
CONFIG_PACKAGE_openssh-client=y
CONFIG_PACKAGE_openssh-keygen=y
CONFIG_PACKAGE_openssh-moduli=y
CONFIG_PACKAGE_openssh-server=y
CONFIG_PACKAGE_pbr=y
CONFIG_PACKAGE_pciids=y
CONFIG_PACKAGE_pciutils=y
CONFIG_PACKAGE_procps-ng=y
CONFIG_PACKAGE_procps-ng-free=y
CONFIG_PACKAGE_procps-ng-kill=y
CONFIG_PACKAGE_procps-ng-pgrep=y
CONFIG_PACKAGE_procps-ng-pkill=y
CONFIG_PACKAGE_procps-ng-ps=y
CONFIG_PACKAGE_procps-ng-top=y
CONFIG_PACKAGE_procps-ng-uptime=y
CONFIG_PACKAGE_procps-ng-watch=y
CONFIG_PACKAGE_psmisc=y
CONFIG_PACKAGE_r8152-firmware=y
CONFIG_PACKAGE_resolveip=y
CONFIG_PACKAGE_rng-tools=y
CONFIG_PACKAGE_rpcd=y
CONFIG_PACKAGE_rpcd-mod-file=y
CONFIG_PACKAGE_rpcd-mod-iwinfo=y
CONFIG_PACKAGE_rpcd-mod-luci=y
CONFIG_PACKAGE_rpcd-mod-lxc=y
CONFIG_PACKAGE_rpcd-mod-rpcsys=y
CONFIG_PACKAGE_rpcd-mod-rrdns=y
CONFIG_PACKAGE_rpcd-mod-ucode=y
CONFIG_PACKAGE_rsync=y
CONFIG_PACKAGE_rsyncd=y
CONFIG_PACKAGE_samba4-libs=y
CONFIG_PACKAGE_samba4-server=y
CONFIG_PACKAGE_smartmontools=y
CONFIG_PACKAGE_smartmontools-drivedb=y
CONFIG_PACKAGE_sqm-scripts=y
CONFIG_PACKAGE_sysfsutils=y
CONFIG_PACKAGE_tar=y
CONFIG_PACKAGE_tc-tiny=y
CONFIG_PACKAGE_tcpdump=y
CONFIG_PACKAGE_terminfo=y
CONFIG_PACKAGE_tree=y
CONFIG_PACKAGE_tune2fs=y
CONFIG_PACKAGE_ucode-mod-html=y
CONFIG_PACKAGE_ucode-mod-lua=y
CONFIG_PACKAGE_ucode-mod-math=y
# CONFIG_PACKAGE_ucode-mod-uloop is not set
CONFIG_PACKAGE_uhttpd=y
CONFIG_PACKAGE_uhttpd-mod-ubus=y
CONFIG_PACKAGE_usbids=y
CONFIG_PACKAGE_usbutils=y
CONFIG_PACKAGE_vim-fuller=y
CONFIG_PACKAGE_wget-ssl=y
CONFIG_PACKAGE_wireguard-tools=y
# CONFIG_PACKAGE_wpad-basic-mbedtls is not set
CONFIG_PACKAGE_wsdd2=y
CONFIG_PACKAGE_xtables-legacy=y
CONFIG_PACKAGE_xtables-nft=y
CONFIG_PACKAGE_xz=y
CONFIG_PACKAGE_xz-utils=y
CONFIG_PACKAGE_zsh=y
CONFIG_PACKAGE_zstd=y
CONFIG_RSYNC_acl=y
CONFIG_RSYNC_xattr=y
CONFIG_SAMBA4_SERVER_AVAHI=y
CONFIG_SAMBA4_SERVER_NETBIOS=y
CONFIG_SAMBA4_SERVER_VFS=y
CONFIG_SAMBA4_SERVER_WSDD2=y
CONFIG_SQLITE3_COLUMN_METADATA=y
CONFIG_SQLITE3_DYNAMIC_EXTENSIONS=y
CONFIG_SQLITE3_FTS3=y
CONFIG_SQLITE3_FTS4=y
CONFIG_SQLITE3_FTS5=y
CONFIG_SQLITE3_RTREE=y
CONFIG_TARGET_KERNEL_PARTSIZE=32
CONFIG_TARGET_OPTIMIZATION="-mcpu=cortex-a76 -O3 -pipe"
CONFIG_TARGET_OPTIONS=y
CONFIG_TARGET_ROOTFS_PARTSIZE=512
# CONFIG_TARGET_ROOTFS_SQUASHFS is not set
CONFIG_ZSTD_OPTIMIZE_O3=y
CONFIG_KERNEL_WERROR=y
2

Why would you want to disable Squash FS? I think it's essential for partition overlay function.

3

This is the same diffconfig I have been using for ages. I don’t specifically recall disabling it. But I have a memory of just using the ext4 rootfs partition option. Again, if I build 0fd9d00cd6fc, using this same diffconfig, it boots fine.

EDIT: As a sanity-check, I checked the config.buildinfo from the last-known-good build I created on 23-Jan-2025 which I mentioned above and it is identical to the one I posted above, ie it contains # CONFIG_TARGET_ROOTFS_SQUASHFS is not set

EDIT2: Yes, there are both options available (ext4 and squashfs), see: https://downloads.openwrt.org/snapshots/targets/bcm27xx/bcm2712/

I think the source of the boot failure has to be elsewhere.

4

It turns out that those lines about not finding fstab are also present when I boot into the last-known-good image.

...
[    7.710162] EXT4-fs (mmcblk0p2): re-mounted ff313567-e9f1-5a5d-9895-3ba130b4a864 r/w. Quota mode: disabled.
[    7.720329] mount_root: loading kmods from internal overlay
[    7.728416] kmodloader: loading kernel modules from //etc/modules-boot.d/*
[    7.735796] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
[    7.791752] block: attempting to load /tmp/overlay/upper/etc/config/fstab
[    7.800376] block: unable to load configuration (fstab: Entry not found)
[    7.807463] block: attempting to load /tmp/overlay/etc/config/fstab
[    7.814150] block: unable to load configuration (fstab: Entry not found)
[    7.821284] block: attempting to load /etc/config/fstab
[    7.827111] block: unable to load configuration (fstab: Entry not found)
[    7.834240] block: no usable configuration
[    7.838732] block: attempting to load /etc/config/fstab
[    7.844346] block: unable to load configuration (fstab: Entry not found)
[    7.851465] block: no usable configuration
[    7.879830] insmod: module is already loaded - fat
[    7.885382] insmod: module is already loaded - vfat
[    7.907527] urandom-seed: Seed file not found (/etc/urandom.seed)
[    7.920361] macb 1f00100000.ethernet: gem-ptp-timer ptp clock unregistered.
[    7.928607] procd: - early -
[    7.931892] procd: - watchdog -
[    8.444819] procd: - watchdog -
[    8.448958] procd: - ubus -
[    8.501948] procd: - init -
[    8.571379] kmodloader: loading kernel modules from /etc/modules.d/*
[    8.585015] hid: raw HID events driver (C) Jiri Kosina
[    8.591173] GACT probability on
[    8.594831] Mirror/redirect action on
[    8.599655] u32 classifier
[    8.602734]     input device check on
[    8.606764]     Actions configured
[    8.620192] brcmutil: loading out-of-tree module taints kernel.
...
(continues to boot and run)

Since the box freezes I can't easily get the dmesg log, but the last few lines before the freeze are:

procd: - early -
procd: - watchdog -
EXT4-fs (nvme0n1p1): mounted filesystem ff313567-e9f1-5a5d-9895-3ba130b4a864 ro with ordered data mode. Quota mode: disabled.
procd: - watchdog -
procd: - ubus -

Not sure where to go from here to debug further. I tried to enter failsafe mode at boot but the USB connected keyboard is not responding/is not initialized. I can unplug it and plug it back and and see output in the console about a new usb device so it isn't frozen as I thought.

5

So this is getting more confusing.

Test 1: I flashed today's (02-Feb-2025) snapshot image: rpi-5-ext4-sysupgrade.img.gz and booted into it without any problem. It is running r28739-69890e16b3.

Test 2: I used the snapshot's config.buildinfo as my .config and built an image myself (also r28739-69890e16b3). The resulting image did not boot. I totally rebuilt my build system but found that again, the image generated will not boot.

EDIT: Could this be something on my host OS (Arch Linux) that is incompatible with the OpenWrt build system? If I look at the host packages from the last time I built a working image (23-Jan-2025), these are the differences:

abseil-cpp: downgrading from version 20240722.1-1 to version 20240722.0-1
archlinux-keyring: downgrading from version 20250123-1 to version 20241203-1
bind: downgrading from version 9.20.5-1 to version 9.20.4-2
electron33: downgrading from version 33.3.2-1 to version 33.3.1-3
fakeroot: downgrading from version 1.37-1 to version 1.36.2-1
fmt: downgrading from version 11.1.3-1 to version 11.1.2-1
fontconfig: downgrading from version 2:2.16.0-2 to version 2:2.16.0-1
gegl: downgrading from version 0.4.52-2 to version 0.4.52-1
glibc: downgrading from version 2.40+r66+g7d4b6bcae91f-1 to version 2.40+r16+gaa533d58ff-2
grml-zsh-config: downgrading from version 0.19.14-1 to version 0.19.13-1
gst-plugins-bad-libs: downgrading from version 1.24.12-1 to version 1.24.11-2
gst-plugins-base: downgrading from version 1.24.12-1 to version 1.24.11-2
gst-plugins-base-libs: downgrading from version 1.24.12-1 to version 1.24.11-2
gstreamer: downgrading from version 1.24.12-1 to version 1.24.11-2
gtk3: downgrading from version 1:3.24.48-1 to version 1:3.24.43-4
gum: downgrading from version 0.15.2-1 to version 0.15.0-1
gvfs: downgrading from version 1.56.1-2 to version 1.56.1-1
gvfs-afc: downgrading from version 1.56.1-2 to version 1.56.1-1
gvfs-gphoto2: downgrading from version 1.56.1-2 to version 1.56.1-1
gvfs-nfs: downgrading from version 1.56.1-2 to version 1.56.1-1
gvim: downgrading from version 9.1.1065-1 to version 9.1.1043-1
libadwaita: downgrading from version 1:1.6.4-1 to version 1:1.6.3-1
libetonyek: downgrading from version 0.1.12-2 to version 0.1.12-1
libnfs: downgrading from version 6.0.2-3 to version 5.0.3-1
libpaper: downgrading from version 2.2.6-1 to version 2.2.5-1
libpng: downgrading from version 1.6.46-1 to version 1.6.45-1
libvlc: ignoring package downgrade (3.0.21-12 => 3.0.21-11)
libvorbis: downgrading from version 1.3.7-4 to version 1.3.7-3
libwps: downgrading from version 0.4.14-3 to version 0.4.14-2
man-pages: downgrading from version 6.10-1 to version 6.9.1-1
mesa: downgrading from version 1:24.3.4-1 to version 1:24.3.3-2
noto-fonts: downgrading from version 1:2025.02.01-1 to version 1:2025.01.01-1
openucx: downgrading from version 1.18.0-1 to version 1.17.0-3
perl-image-exiftool: downgrading from version 13.17-1 to version 13.14-1
procps-ng: downgrading from version 4.0.5-2 to version 4.0.5-1
python-pyqt6: downgrading from version 6.8.0-3 to version 6.8.0-2
qt6-5compat: downgrading from version 6.8.2-1 to version 6.8.1-1
qt6-base: downgrading from version 6.8.2-1 to version 6.8.1-1
qt6-declarative: downgrading from version 6.8.2-1 to version 6.8.1-3
qt6-shadertools: downgrading from version 6.8.2-1 to version 6.8.1-1
qt6-svg: downgrading from version 6.8.2-1 to version 6.8.1-1
qt6-translations: downgrading from version 6.8.2-1 to version 6.8.1-1
qt6-wayland: downgrading from version 6.8.2-1 to version 6.8.1-1
unzip: downgrading from version 6.0-22 to version 6.0-21
vim-runtime: downgrading from version 9.1.1065-1 to version 9.1.1043-1
vlc: ignoring package downgrade (3.0.21-12 => 3.0.21-11)
vte-common: downgrading from version 0.78.3-1 to version 0.78.2-1
vte3: downgrading from version 0.78.3-1 to version 0.78.2-1
webrtc-audio-processing-1: downgrading from version 1.3-4 to version 1.3-3
xz: downgrading from version 5.6.4-1 to version 5.6.3-1
zenity: downgrading from version 4.0.5-1 to version 4.0.3-1

I am downgrading them and rebuilding the build system/image as a test but am running out of ideas.

EDIT: the result of the image built from the downgraded host OS packages was the same: does not boot.

6

Default for bcm27xx is 64, I suspect that with 32 is on the limit.

7

I saw that too, and I removed the line entirely, but I get the same inability to boot. Also using the snapshot config, which doesn’t have that defined will not boot.

8

I read some github discussions about 16k pages has issues with openwrt.

9

I also removed all customization including the larger pages. Even an image built with the vanilla snapshot's config.buildinfo gives the non-booting behavior. Yet, I can flash the official snapshot image and it boots as expected.

10

OK, this very odd situation is fixed. The fix involved a fresh git clone of the openwrt repo. From that, I was able to build my customized diffconfig (with 16k pages, and all) and have a fully functioning image. I would still like to know why the old build root caused the problem, but am happy that I have a functional build system again.

11

Cleaning up the buildroot after larger toolchain changes (you had a glibc update in your package upgrade) can be necessary, it's not unlikely to observe miscompiled binaries with a mixture between old- and new build artefacts. Issues like these get even more pronounced by things like distcc or similar build artefact caching helpers.

12

The only thing I can think of was that I had to back up the partition that holds my openwrt build root and I accidentally did so to an old NTFS partition which of course jacked up all of the octal permissions since NTFS for whatever reason has all files with their executable bit set. When I copied it back I went through and deleted most things (toolchains, host files, etc.) and even did a git reset --hard origin/HEAD which should have brought everything back to a valid state but, if I'm right, did not :smiley:

13

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.