Can't build OpenWrt - compilation fails because of python(?)

UsernameHasBeenTaken · October 14, 2024, 11:17am

Hello!

I was curious about building OpenWrt by myself and tried to compile it.
Unfortunately, the process failed and I don't how to fix it.

The steps I took:

Installed Ubuntu 24.04.1 LTS WSL (Ubuntu for Windows).
Installed required packages (https://openwrt.org/docs/guide-developer/toolchain/install-buildsystem) + python3-pip: sudo apt install build-essential clang flex bison g++ gawk gcc-multilib g++-multilib gettext git libncurses5-dev libssl-dev python3-setuptools rsync swig unzip zlib1g-dev file wget python3-pip
Obtained OpenWrt sources: git clone https://github.com/openwrt/openwrt.git. Next steps were performed in the openwrt directory.
Updated feeds: ./scripts/feeds update -a
Installed packages: ./scripts/feeds install -a
Downloaded recommended config: wget https://downloads.openwrt.org/snapshots/targets/mediatek/filogic/config.buildinfo -O .config
Selected my device and Luci package via make menuconfig and saved changes.
Ran compile with logging on 15 threads make -j16 V=sc

After a while I got this error:

Error

Looking in links: /home/user/openwrt_build/openwrt/tmp/tmpiigxk8lf
Processing /home/user/openwrt_build/openwrt/tmp/tmpiigxk8lf/setuptools-65.5.0-py3-none-any.whl
Processing /home/user/openwrt_build/openwrt/tmp/tmpiigxk8lf/pip-23.2.1-py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-23.2.1 setuptools-65.5.0
make[4]: Leaving directory '/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7'

Applying ./patches-host-setuptools/0001-Adjust-library-header-paths-for-cross-compilation.patch using plaintext:
patching file setuptools/_distutils/command/build_ext.py
touch /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/.setuptools_installed_65.5.0-2
touch /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/.pip_installed_23.2.1-1
mkdir -p /home/user/openwrt_build/openwrt/staging_dir/hostpkg/stamp
touch /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/.built
touch /home/user/openwrt_build/openwrt/staging_dir/hostpkg/stamp/.python3_installed
find /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7 -mindepth 1 -maxdepth 1 -not '(' -type f -and -name '.*' -and -size 0 ')' -print0 | xargs -r -0 rm -rf
make[3]: Leaving directory '/home/user/openwrt_build/openwrt/feeds/packages/lang/python/python3'
time: package/feeds/packages/python3/host-compile#1720.27#104.00#367.52
make[2]: Leaving directory '/home/user/openwrt_build/openwrt'
make[1]: *** [package/Makefile:171: /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/user/openwrt_build/openwrt'
make: *** [/home/user/openwrt_build/openwrt/include/toplevel.mk:248: world] Error 2

I have no idea how to fix it.
I'd appreciate any advice! Thank you!

frollic · October 14, 2024, 11:28am

rerun with -j1 instead of 16, see if you can spot the actual error.

seems you're missing a couple of steps from https://openwrt.org/docs/guide-developer/toolchain/use-buildsystem

UsernameHasBeenTaken · October 14, 2024, 12:03pm

Thank you! I have started the build process with j1 parameter: make -j1 V=sc. It will take a while...

I have double-checked the steps:

I'd like to build the master branch, also the directory containing sources didn't exist before cloning. Thus git clone https://git.openwrt.org/openwrt/openwrt.git should be enough. I have deleted it and re-cloned the project just in case... git pull output said that there are nothing to pull (no changes).
For now I do not use any local feeds, just the default ones.
make defconfig also said that there are no changes.
Kernel configuration step is marked as optional.

Hostle · October 14, 2024, 12:06pm

why not select a target before make ? no need to build all, unless that is your intention. also make sure the build deps are up to date. do you have python3 installed ?

make menuconfig

UsernameHasBeenTaken · October 14, 2024, 12:27pm

After I downloaded the official build config (wget https://downloads.openwrt.org/snapshots/targets/mediatek/filogic/config.buildinfo -O .config) I already had the correct Traget and Subtarget selected in the make menuconfig. In addition I have selected the Target Profile (set it to my device name) and LuCI.
Both the failed build and the one that is running right now (with -j1) have correct Target, Subtarget and Target Profile set in the .config file.
Or do you mean choosing Target and Subtarget somewhere else?

I do have python3 installed: Python 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0] on linux. The sources have 3.11 included: could this be a problem?

Hostle · October 14, 2024, 12:34pm

nope, that sounds correct. I thought you were just running make world ... which will build all targets.

UsernameHasBeenTaken · October 14, 2024, 12:48pm

Thank you for error collecting suggestion. I will run with output redirection next time (don't want to interrupt the current build: it has been running for a while).

Hostle · October 14, 2024, 12:50pm

shouldn't be, unless its specified in the makefile to use a certain version. One trick I like to use is the -i flag. It will ignore all errors all build whatever can be built successfully. Usually this helps hone in on the offending file.

make V=s -i

lantis1008 · October 14, 2024, 1:11pm

Check if you have “python” in your environment or only “python3”
If you are missing “python”, install the package “python-is-python3”

UsernameHasBeenTaken · October 14, 2024, 1:40pm

No, I didn't have python in my environment - only python3. I have installed python-is-python3 according to your recommendation.
Hmm, I wonder if it will affect the build which has been started before the package installation and still running...

frollic · October 14, 2024, 1:44pm

if it's pointing to the same binary, it shouldn't.

slh · October 14, 2024, 3:32pm

While technically correct, this will build unnecessarily a lot of stuff you'll never need for your single device and its rather modest package set. Do yourself a favour, spend some time up front to decide what you want (this can be as simple as target, subtarget, device and luci-ssl + anything else you want to see included) - and then only build that. Doing so significantly reduces your build time and the scratch space during the build, without any side effects.

UsernameHasBeenTaken · October 15, 2024, 9:53am

I couldn't wait for the compilation to finish with -j1 parameter: it took much too long. I interrupted it, installed python-is-python3 package and re-run compilation. This time with stdout and strderr being redirected to files: make -j20 V=sc 1> stdout.log 2> stderr.log
Unfortunately, python-is-python3 didn't help and I got the same error.
Here are the tails of both files (I can post more if needed):

stdout

...
running install_scripts
copying build/scripts-3.11/2to3-3.11 -> /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin
copying build/scripts-3.11/pydoc3.11 -> /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin
copying build/scripts-3.11/idle3.11 -> /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/2to3-3.11 to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/pydoc3.11 to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/idle3.11 to 755
rm /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_sysconfigdata__linux_x86_64-linux-gnu.py
rm -r /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/__pycache__
if test ! -d /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig; then \
        echo "Creating directory /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig"; \
        /home/user/openwrt_build/openwrt/staging_dir/host/bin/install -c -d -m 755 /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig; \
fi
if test -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3 -o -h /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3; \
then rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3; \
else true; \
fi
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s python3.11 python3)
if test "3.11" != "3.11"; then \
        rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3.11-config; \
        (cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s python3.11-config python3.11-config); \
        rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig/python-3.11.pc; \
        (cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig; ln -s python-3.11.pc python-3.11.pc); \
        rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig/python-3.11-embed.pc; \
        (cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig; ln -s python-3.11-embed.pc python-3.11-embed.pc); \
fi
rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3-config
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s python3.11-config python3-config)
rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig/python3.pc
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig; ln -s python-3.11.pc python3.pc)
rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig/python3-embed.pc
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/pkgconfig; ln -s python-3.11-embed.pc python3-embed.pc)
rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/idle3
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s idle3.11 idle3)
rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/pydoc3
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s pydoc3.11 pydoc3)
rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/2to3
(cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s 2to3-3.11 2to3)
if test "x" != "x" ; then \
        rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3-32; \
        (cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s python3.11-32 python3-32) \
fi
if test "x" != "x" ; then \
        rm -f /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin/python3-intel64; \
        (cd /home/user/openwrt_build/openwrt/staging_dir/hostpkg/bin; ln -s python3.11-intel64 python3-intel64) \
fi
if test "xupgrade" != "xno"  ; then \
        case upgrade in \
                upgrade) ensurepip="--upgrade" ;; \
                install|*) ensurepip="" ;; \
        esac; \
         ./python -E -m ensurepip \
                $ensurepip --root=/ ; \
fi
Looking in links: /home/user/openwrt_build/openwrt/tmp/tmpr1edyyzz
Processing /home/user/openwrt_build/openwrt/tmp/tmpr1edyyzz/setuptools-65.5.0-py3-none-any.whl
Processing /home/user/openwrt_build/openwrt/tmp/tmpr1edyyzz/pip-23.2.1-py3-none-any.whl
Installing collected packages: setuptools, pip
Successfully installed pip-23.2.1 setuptools-65.5.0
make[4]: Leaving directory '/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7'

Applying ./patches-host-setuptools/0001-Adjust-library-header-paths-for-cross-compilation.patch using plaintext:
patching file setuptools/_distutils/command/build_ext.py
touch /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/.setuptools_installed_65.5.0-2
touch /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/site-packages/.pip_installed_23.2.1-1
mkdir -p /home/user/openwrt_build/openwrt/staging_dir/hostpkg/stamp
touch /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/.built
touch /home/user/openwrt_build/openwrt/staging_dir/hostpkg/stamp/.python3_installed
find /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7 -mindepth 1 -maxdepth 1 -not '(' -type f -and -name '.*' -and -size 0 ')' -print0 | xargs -r -0 rm -rf
make[3]: Leaving directory '/home/user/openwrt_build/openwrt/feeds/packages/lang/python/python3'
time: package/feeds/packages/python3/host-compile#922.81#54.08#261.31
make[2]: Leaving directory '/home/user/openwrt_build/openwrt'
make[1]: Leaving directory '/home/user/openwrt_build/openwrt'

stderr

        not a dynamic executable
In file included from /home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/readline.h:36,
                 from /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/readline.c:36:
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:35:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   35 | typedef int Function () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:36:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   36 | typedef void VFunction () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:37:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   37 | typedef char *CPFunction () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:38:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   38 | typedef char **CPPFunction () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/readline.h:410:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
  410 | extern int rl_message ();
      | ^~~~~~
*** WARNING: renaming "_curses" since importing it failed: /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/lib.linux-x86_64-3.11/_curses.cpython-311-x86_64-linux-gnu.so: undefined symbol: extended_color_content
*** WARNING: renaming "_curses_panel" since importing it failed: PyCapsule_Import could not import module "_curses"
renaming build/scripts-3.11/pydoc3 to build/scripts-3.11/pydoc3.11
renaming build/scripts-3.11/idle3 to build/scripts-3.11/idle3.11
renaming build/scripts-3.11/2to3 to build/scripts-3.11/2to3-3.11
WARNING: kmod-crypto-hw-hifn-795x is not available in the kernel config - generating empty package
WARNING: kmod-crypto-hw-padlock is not available in the kernel config - generating empty package
WARNING: kmod-hwmon-max6642 is not available in the kernel config - generating empty package
WARNING: kmod-i2c-pxa is not available in the kernel config - generating empty package
WARNING: kmod-ledtrig-gpio is not available in the kernel config - generating empty package
WARNING: kmod-asn1-encoder is not available in the kernel config - generating empty package
WARNING: kmod-lib-objagg is not available in the kernel config - generating empty package
WARNING: kmod-lib-parman is not available in the kernel config - generating empty package
WARNING: kmod-ag71xx is not available in the kernel config - generating empty package
WARNING: kmod-dm9000 is not available in the kernel config - generating empty package
WARNING: kmod-thermal is not available in the kernel config - generating empty package
WARNING: kmod-rtc-mv is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-core is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-realtek is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-cmedia is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-analog is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-idt is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-si3054 is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-cirrus is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-ca0110 is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-ca0132 is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-conexant is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-via is not available in the kernel config - generating empty package
WARNING: kmod-sound-hda-codec-hdmi is not available in the kernel config - generating empty package
WARNING: kmod-video-cpia2 is not available in the kernel config - generating empty package
WARNING: kmod-imx2-wdt is not available in the kernel config - generating empty package
WARNING: kmod-imx7-ulp-wdt is not available in the kernel config - generating empty package
make[3]: [Makefile:73: compile] Error 1 (ignored)
test test_embed failed
test test_xml_etree failed
test test_xml_etree_c failed
Python/initconfig.c:2283:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
 2283 | config_envvars_usage()
      | ^~~~~~~~~~~~~~~~~~~~
Python/initconfig.c:2289:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
 2289 | config_xoptions_usage()
      | ^~~~~~~~~~~~~~~~~~~~~
        not a dynamic executable
In file included from /home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/readline.h:36,
                 from /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/readline.c:36:
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:35:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   35 | typedef int Function () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:36:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   36 | typedef void VFunction () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:37:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   37 | typedef char *CPFunction () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/rltypedefs.h:38:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
   38 | typedef char **CPPFunction () __attribute__((deprecated));
      | ^~~~~~~
/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include/readline/readline.h:410:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
  410 | extern int rl_message ();
      | ^~~~~~
/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.c: In function 'PyInit__curses':
/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.c:4948:1: warning: '/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/temp.linux-x86_64-3.11/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.gcda' profile count data file not found [-Wmissing-profile]
 4948 | }
      | ^
*** WARNING: renaming "_curses" since importing it failed: /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/lib.linux-x86_64-3.11/_curses.cpython-311-x86_64-linux-gnu.so: undefined symbol: extended_color_content
*** WARNING: renaming "_curses_panel" since importing it failed: PyCapsule_Import could not import module "_curses"
renaming build/scripts-3.11/pydoc3 to build/scripts-3.11/pydoc3.11
renaming build/scripts-3.11/idle3 to build/scripts-3.11/idle3.11
renaming build/scripts-3.11/2to3 to build/scripts-3.11/2to3-3.11
        not a dynamic executable
/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.c: In function 'PyInit__curses':
/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.c:4948:1: warning: '/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/temp.linux-x86_64-3.11/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.gcda' profile count data file not found [-Wmissing-profile]
 4948 | }
      | ^
*** WARNING: renaming "_curses" since importing it failed: /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/lib.linux-x86_64-3.11/_curses.cpython-311-x86_64-linux-gnu.so: undefined symbol: extended_color_content
*** WARNING: renaming "_curses_panel" since importing it failed: PyCapsule_Import could not import module "_curses"
renaming build/scripts-3.11/pydoc3 to build/scripts-3.11/pydoc3.11
renaming build/scripts-3.11/idle3 to build/scripts-3.11/idle3.11
renaming build/scripts-3.11/2to3 to build/scripts-3.11/2to3-3.11
        not a dynamic executable
/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.c: In function 'PyInit__curses':
/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.c:4948:1: warning: '/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/temp.linux-x86_64-3.11/home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/Modules/_cursesmodule.gcda' profile count data file not found [-Wmissing-profile]
 4948 | }
      | ^
*** WARNING: renaming "_curses" since importing it failed: /home/user/openwrt_build/openwrt/build_dir/hostpkg/Python-3.11.7/build/lib.linux-x86_64-3.11/_curses.cpython-311-x86_64-linux-gnu.so: undefined symbol: extended_color_content
*** WARNING: renaming "_curses_panel" since importing it failed: PyCapsule_Import could not import module "_curses"
renaming build/scripts-3.11/pydoc3 to build/scripts-3.11/pydoc3.11
renaming build/scripts-3.11/idle3 to build/scripts-3.11/idle3.11
renaming build/scripts-3.11/2to3 to build/scripts-3.11/2to3-3.11
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_posixsubprocess.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/__pycache__/_sysconfigdata__linux_x86_64-linux-gnu.cpython-311.pyc to 644
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/grp.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/unicodedata.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_sha512.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_lsprof.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_multibytecodec.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_json.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_ssl.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/xxlimited.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_queue.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_asyncio.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_hashlib.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_random.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/binascii.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_csv.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_contextvars.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_posixshmem.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_socket.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_struct.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_uuid.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_typing.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_codecs_kr.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_sysconfigdata__linux_x86_64-linux-gnu.py to 644
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/select.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_xxsubinterpreters.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/spwd.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/math.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/resource.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_multiprocessing.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_zoneinfo.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_crypt.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/readline.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_codecs_iso2022.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_datetime.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/array.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/syslog.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_bz2.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/cmath.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/ossaudiodev.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_elementtree.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_lzma.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/audioop.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_decimal.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_statistics.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/termios.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_heapq.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_curses.cpython-311-x86_64-linux-gnu_failed.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_pickle.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_md5.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_codecs_jp.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_curses_panel.cpython-311-x86_64-linux-gnu_failed.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_sha256.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_sha1.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/pyexpat.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_codecs_hk.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_opcode.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/fcntl.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/mmap.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_codecs_tw.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/zlib.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_sha3.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_bisect.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/xxlimited_35.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/_codecs_cn.cpython-311-x86_64-linux-gnu.so to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/ to 755
changing mode of /home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib/python3.11/lib-dynload/__pycache__ to 755
make[1]: *** [package/Makefile:171: /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/stamp/.package_compile] Error 2
make: *** [/home/user/openwrt_build/openwrt/include/toplevel.mk:248: world] Error 2

I don't see anything obvious what could go wrong.

@slh , thank you, that is a good idea, but I don't know what critical packages I must includ in the image to work on the router. I mean, fw4 and busybox are obvious; I have a few packages that I need; but besides this: what else should I includ in the firmware to be functional on the router? That's why I am sticking to the recommended packages list.

frollic · October 15, 2024, 10:26am

out of curiosity, are any of the packages you're adding not in the openwrt repo ?

instead of compiling, perhaps you should try https://openwrt.org/docs/guide-user/additional-software/imagebuilder ?
assuming it's usable, an image is built in less than a min or two.

UsernameHasBeenTaken · October 15, 2024, 10:52am

For now I am not using any packages that are not in the official repo.
But I had plans to include one git repo with 3 packages that needed to be compiled after I had successfully build the image with the "default + LuCI" settings.

I have a working pipline for building these packages right in the github now, but I have to install them manually afterwards.
That's why I wanted to get all-in-one image, including everything I need.

RuralRoots · October 15, 2024, 10:57am

Just build from a minimal .config starting from a make distclean.

Create your initial .config by:

make nconfig    # enter your Target
                # and SubTarget
                # and Target Profile
                # escape out of nconfig and select <save>
make defconfig  # make a default config for the selected Target/Release

Anything critically needed to produce a base viable build for your your Target, SubTarget, Target Profile will be set up for the .config.

You might want to reduce the parallel build process by specify utilizing half of your cores initially. Once you get a good build, you should be able to add anything else you want next time around.

UsernameHasBeenTaken · October 15, 2024, 11:29am

Thank you!

Trying the following:

rm -rf openwrt/
git clone https://github.com/openwrt/openwrt.git
cd openwrt/
make distclean
./scripts/feeds update -a
./scripts/feeds install -a
make nconfig
# Selected Target, Subtarget, Target Profile (Device) -> Esc -> Save
make defconfig
make -j7 V=sc 1> ../stdout.log 2> ../stderr.log

I have 14 physical cores, 20 virtual - thus -j7

Will report the results

frollic · October 15, 2024, 12:11pm

I just ran the above, with x86_64 as target, no issues.

UsernameHasBeenTaken · October 15, 2024, 2:20pm

No luck... Got a new error:

stdout

touch /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/stamp/.util-linux_installed
rm -rf /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/.pkgdir/libblkid.installed /home/user/openwrt_build
/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/.pkgdir/libblkid
mkdir -p /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/.pkgdir/libblkid
install -d -m0755 /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/.pkgdir/libblkid/usr/lib
cp -fpR /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/ipkg-install/usr/lib/libblkid.so.* /home/user/open
wrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/.pkgdir/libblkid/usr/lib/
touch /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/util-linux-2.40.2/.pkgdir/libblkid.installed
mkdir -p /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/root-mediatek/stamp
echo '1' | cmp -s - /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/pkginfo/libblkid.version || { echo '1' > /home/user/op
enwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/pkginfo/libblkid.version;  }
SHELL= flock /home/user/openwrt_build/openwrt/tmp/.root-copy.flock -c 'cp -fpR /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_mu
sl/util-linux-2.40.2/.pkgdir/libblkid/. /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/root-mediatek/'
touch /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/root-mediatek/stamp/.libblkid_installed
echo "libblkid" >> /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/pkginfo/util-linux.default.install
make[3]: Leaving directory '/home/user/openwrt_build/openwrt/package/utils/util-linux'
time: package/utils/util-linux/compile#84.06#8.11#40.86
make[2]: Leaving directory '/home/user/openwrt_build/openwrt'
make[1]: Leaving directory '/home/user/openwrt_build/openwrt'

stderr

bash: -c: line 1: syntax error near unexpected token `('
bash: -c: line 1: `make --jobserver-auth=3,4  -C /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/u-boot-mt7988_bananapi_bpi-r4-emmc/u-boot-2024.10 CROSS_COMPILE=aarch64-openwrt-linux-musl-  PATH=/home/user/openwrt_build/openwrt/staging_dir/host/bin:/home/user/openwrt_build/openwrt/staging_dir/toolchain-aarch64_cortex-a53_gcc-13.3.0_musl/bin:/home/user/openwrt_build/openwrt/staging_dir/toolchain-aarch64_cortex-a53_gcc-13.3.0_musl/bin:/home/user/openwrt_build/openwrt/staging_dir/host/bin:/home/user/openwrt_build/openwrt/staging_dir/toolchain-aarch64_cortex-a53_gcc-13.3.0_musl/bin:/home/user/openwrt_build/openwrt/staging_dir/host/bin:/home/user/openwrt_build/openwrt/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program:Files/BellSoft/LibericaJDK-17/bin/:/mnt/c/Windows/system32:/mnt/c/Windows:/mnt/c/Windows/System32/Wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0/:/mnt/c/Windows/System32/OpenSSH/:/mnt/c/Users/user/AppData/Local/Programs/Microsoft:VS:Code/bin:/mnt/c/Users/user/AppData/Roaming/npm:/mnt/c/Users/user/AppData/Local/GitHubDesktop/bin:/snap/bin HOSTCC="/home/user/openwrt_build/openwrt/staging_dir/host/bin/gcc" HOSTCFLAGS="-O2 -I/home/user/openwrt_build/openwrt/staging_dir/host/include -I/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include -I/home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/host/include -I/home/user/openwrt_build/openwrt/staging_dir/host/include -I/home/user/openwrt_build/openwrt/staging_dir/hostpkg/include -I/home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/host/include -std=gnu11" HOSTLDFLAGS="-L/home/user/openwrt_build/openwrt/staging_dir/host/lib -L/home/user/openwrt_build/openwrt/staging_dir/hostpkg/lib -L/home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/host/lib" LOCALVERSION="-OpenWrt-r27767-0a2ed285e4" STAGING_PREFIX="/home/user/openwrt_build/openwrt/staging_dir/host" PKG_CONFIG_PATH="/home/user/openwrt_build/openwrt/staging_dir/host/lib/pkgconfig" PKG_CONFIG_LIBDIR="/home/user/openwrt_build/openwrt/staging_dir/host/lib/pkgconfig" PKG_CONFIG_EXTRAARGS="--static" V=1 u-boot.bin'
make[3]: *** [Makefile:888: /home/user/openwrt_build/openwrt/build_dir/target-aarch64_cortex-a53_musl/u-boot-mt7988_bananapi_bpi-r4-emmc/u-boot-2024.10/.built] Error 2
make[2]: *** [package/Makefile:177: package/boot/uboot-mediatek/compile] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [package/Makefile:171: /home/user/openwrt_build/openwrt/staging_dir/target-aarch64_cortex-a53_musl/stamp/.package_compile] Error 2
make: *** [/home/user/openwrt_build/openwrt/include/toplevel.mk:248: world] Error 2

I am trying to build a Mediatek - Filogic - Banana BPR-R4 image.
I am also starting to think that it was not a bright idea to use Ubuntu WSL (Ubuntu for Windows). I am going to run an Ubuntu Server in VirtualBox and compile the image there

RuralRoots · October 15, 2024, 3:03pm

Yes. Too many parallel processes likely from the make[2]: *** Waiting for unfinished jobs.... output, and the non-native build environment.

That’s the reason make -j1 V=sc is really the only way to get a definitive handle on what went on. Stdout/Stderr really don’t provide a lot of context.

Building on a VM has a better chance than WSL in my opinion (which to me is something akin to running WINE on Linux), and clearly a native build environment is best for building.

At least it’s some progress

Edit: Just in case it isn’t clear, make -j1 V=sc won’t re-compile anything that has already been built eg. The biggest time is taken with compiling your cross compile toolchain. Left alone, it won’t need to be rebuilt. This holds true for all completed processes.