Error building VDI image on latest Arch Linux

Hi, I don't know if I should ask it here or in the Arch Linux forum. But I should just start somewhere.

My problem is I can't build OpenWrt after upgrading to linux 6.4 from 6.3. The error is as follows:

File "/home/adonesp/Projects/openwrt/scripts/json_add_image_info.py", line 40, in <module> hash_file = hashlib.sha256(file_path.read_bytes()).hexdigest() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/pathlib.py", line 1051, in read_bytes return f.read()

Anybody familiar or have experienced this error?

Here is fine, that build issue should not be related to the upgrade of the host kernel (but probably a python3 upgrade coming alongside); I'm not a python specialist so I have to defer the debugging to someone else.

1 Like

Yes it must be due to a new python3 version. The file that produced the error is openwrt/scripts/json_add_image_info.py. This belongs to the OpenWrt project and I believe this should be fixed as it will produce similar error in other linux distros once they upgrade to the new python 3 version.

I don't see the actual error message here, is something missing?

On Arch, you have to expect that something breaks after an upgrade. It could very well be some missing, outdated or aur-installed python library.

Also, did you try to compile a snapshot version of OpenWrt or an older version?

I think I had something similar a few weeks ago and either a complete rebuild or syncing my fork fixed it.

To be honest, I don't know what a snapshot version is. I just checkout the latest stable git tag v22.03.5 and build it. I followed the guide in https://openwrt.org/docs/guide-developer/toolchain/use-buildsystem

I am on Arch running the 6.4.1 and have no issues. Did you install openwrt-devel meta package? Is your system up-to-date otherwise?

I'll try installing openwrt-devel and report back. Yes, all my system packages are up to date. I was able to build any openwrt version prior to upgrading my system.

Then you will probably need to recompile the build system as your environment has changed when upgrading your Arch Linux packages. Run make distclean to clean the tree and try again.

Note: This removes all build artifacts, including your configuration!

1 Like

I tried installing openwrt-devel meta package and did make distclean but it still did not solve the problem. I also found out that it was able to build the openwrt-x86-64-generic-ext4-rootfs.img but not the .vdi file. I wanted to be able to run OpenWrt on virtualbox.

Here's a more details error log:

DEVICE_ID="generic" SOURCE_DATE_EPOCH=1682627295 FILE_NAME="openwrt-x86-64-generic-ext4-rootfs.img" FILE_DIR="/home/adonesp/Projects/xopenwrt/build_dir/target-x86_64_musl/linux-x86_64/tmp" FILE_TYPE=rootfs FILE_FILESYSTEM="ext4" DEVICE_IMG_PREFIX="openwrt-x86-64-generic" DEVICE_VENDOR="Generic" DEVICE_MODEL="x86/64" DEVICE_VARIANT="" DEVICE_ALT0_VENDOR="" DEVICE_ALT0_MODEL="" DEVICE_ALT0_VARIANT="" DEVICE_ALT1_VENDOR="" DEVICE_ALT1_MODEL="" DEVICE_ALT1_VARIANT="" DEVICE_ALT2_VENDOR="" DEVICE_ALT2_MODEL="" DEVICE_ALT2_VARIANT="" DEVICE_TITLE="Generic x86/64" DEVICE_PACKAGES="kmod-amazon-ena kmod-amd-xgbe kmod-bnx2 kmod-e1000e kmod-e1000 kmod-forcedeth kmod-fs-vfat kmod-igb kmod-igc kmod-ixgbe kmod-r8169 kmod-tg3" TARGET="x86" SUBTARGET="64" VERSION_NUMBER="22.03.5" VERSION_CODE="r20134-5f15225c1e" SUPPORTED_DEVICES="" /home/adonesp/Projects/xopenwrt/scripts/json_add_image_info.py /home/adonesp/Projects/xopenwrt/build_dir/target-x86_64_musl/json_info_files/openwrt-x86-64-generic-ext4-rootfs.img.json
Traceback (most recent call last):
  File "/home/adonesp/Projects/xopenwrt/scripts/json_add_image_info.py", line 40, in <module>
    hash_file = hashlib.sha256(file_path.read_bytes()).hexdigest()
                               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/pathlib.py", line 1051, in read_bytes
    return f.read()
           ^^^^^^^^
MemoryError
make[5]: *** [Makefile:157: /home/adonesp/Projects/xopenwrt/build_dir/target-x86_64_musl/json_info_files/openwrt-x86-64-generic-ext4-rootfs.img.json] Error 1
make[5]: Leaving directory '/home/adonesp/Projects/xopenwrt/target/linux/x86/image'
make[4]: *** [Makefile:21: install] Error 2
make[4]: Leaving directory '/home/adonesp/Projects/xopenwrt/target/linux/x86'
make[3]: *** [Makefile:11: install] Error 2
make[3]: Leaving directory '/home/adonesp/Projects/xopenwrt/target/linux'
time: target/linux/install#18.80#230.93#265.19
    ERROR: target/linux failed to build.
make[2]: *** [target/Makefile:30: target/linux/install] Error 1
make[2]: Leaving directory '/home/adonesp/Projects/xopenwrt'
make[1]: *** [target/Makefile:24: /home/adonesp/Projects/xopenwrt/staging_dir/target-x86_64_musl/stamp/.target_install] Error 2
make[1]: Leaving directory '/home/adonesp/Projects/xopenwrt'
make: *** [/home/adonesp/Projects/xopenwrt/include/toplevel.mk:230: world] Error 2

I have fixed the issue by loading the image into the memory in chunks rather than as a whole. It fixes the MemoryError I have encountered.

I have submitted a patch to OpenWrt. Hopefully it can be merged.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.