OpenWrt Forum Archive

Topic: build error when compiling ubox

The content of this topic has been archived on 3 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Post #1
tytus64
13 Feb 2015, 17:40

I hit the following error when building latest openwrt (cloned just today):

make[6]: Entering directory `<mydir>/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
[ 14%] Building C object CMakeFiles/kmodloader.dir/kmodloader.c.o
<mydir>openwrt/build_dir/target-arm-linux/ubox-2014-10-06/kmodloader.c:35:25: fatal error: libubox/avl.h: No such file or directory
 #include <libubox/avl.h>

I work for Broadcom and I was asked to see if I can get running our new chip with openwrt (at this point just a proof-of-concept). I substituted the tooolchain with our own and pointed to our own kernel tree (all done through make menuconfig).  I did not change anything else in the openwrt config - all default packages.

Unfortunately I am not very familiar with cmake... Below is the full log of the ubox build:

make[3]: Entering directory `/home/promanus/projects/local/openwrt/package/system/ubox'
. /home/promanus/projects/local/openwrt/include/shell.sh; gzip -dc /home/promanus/projects/local/openwrt/dl/ubox-2014-10-06-0b274c16a3f9d235735a4b84215071e1e004caa9.tar.gz | tar -C /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/.. -xf - 
touch /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/.prepared_529efcce488818d51acdbd7fd631f93d
(cd /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06; CFLAGS="-Os -pipe -march=armv7-a -mtune=cortex-a9 -fomit-frame-pointer -fno-strict-aliasing -marm -fno-caller-saves -mfloat-abi=hard " CXXFLAGS="-Os -pipe -march=armv7-a -mtune=cortex-a9 -fomit-frame-pointer -fno-strict-aliasing -marm -fno-caller-saves -mfloat-abi=hard " LDFLAGS="-L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib -L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/lib -L/opt/toolchains/stbgcc-4.8-1.0//usr/lib -L/opt/toolchains/stbgcc-4.8-1.0//lib  " cmake -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_VERSION=1 -DCMAKE_SYSTEM_PROCESSOR=arm -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS_RELEASE="-DNDEBUG" -DCMAKE_CXX_FLAGS_RELEASE="-DNDEBUG" -DCMAKE_C_COMPILER="/opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-gcc" -DCMAKE_C_COMPILER_ARG1="" -DCMAKE_CXX_COMPILER="/opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-g++" -DCMAKE_CXX_COMPILER_ARG1="" -DCMAKE_EXE_LINKER_FLAGS:STRING="-L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib -L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/lib -L/opt/toolchains/stbgcc-4.8-1.0//usr/lib -L/opt/toolchains/stbgcc-4.8-1.0//lib " -DCMAKE_MODULE_LINKER_FLAGS:STRING="-L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib -L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/lib -L/opt/toolchains/stbgcc-4.8-1.0//usr/lib -L/opt/toolchains/stbgcc-4.8-1.0//lib " -DCMAKE_SHARED_LINKER_FLAGS:STRING="-L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib -L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/lib -L/opt/toolchains/stbgcc-4.8-1.0//usr/lib -L/opt/toolchains/stbgcc-4.8-1.0//lib " -DCMAKE_AR="/opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-ar" -DCMAKE_NM="/opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-nm" -DCMAKE_RANLIB="/opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-ranlib" -DCMAKE_FIND_ROOT_PATH="/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux;/home/promanus/projects/local/openwrt/staging_dir/toolchain-arm-linux" -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=BOTH -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY -DCMAKE_STRIP=: -DCMAKE_INSTALL_PREFIX=/usr -DDL_LIBRARY=/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux -DCMAKE_PREFIX_PATH=/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux -DCMAKE_SKIP_RPATH=TRUE  . )
-- The C compiler identification is GNU 4.8.4
-- Check for working C compiler: /opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-gcc
-- Check for working C compiler: /opt/toolchains/stbgcc-4.8-1.0/bin/arm-linux-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CXX_COMPILER
    CMAKE_CXX_COMPILER_ARG1
    CMAKE_CXX_FLAGS_RELEASE
    CMAKE_FIND_ROOT_PATH_MODE_INCLUDE
    CMAKE_MODULE_LINKER_FLAGS
    DL_LIBRARY


-- Build files have been written to: /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06
rm -f /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/.configured_
touch /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/.configured_
CFLAGS="-Os -pipe -march=armv7-a -mtune=cortex-a9 -fomit-frame-pointer -fno-strict-aliasing -marm -fno-caller-saves -mfloat-abi=hard  -I/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/include -I/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/include -I/opt/toolchains/stbgcc-4.8-1.0//usr/include -I/opt/toolchains/stbgcc-4.8-1.0//include " CXXFLAGS="-Os -pipe -march=armv7-a -mtune=cortex-a9 -fomit-frame-pointer -fno-strict-aliasing -marm -fno-caller-saves -mfloat-abi=hard  -I/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/include -I/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/include -I/opt/toolchains/stbgcc-4.8-1.0//usr/include -I/opt/toolchains/stbgcc-4.8-1.0//include " LDFLAGS="-L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib -L/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/lib -L/opt/toolchains/stbgcc-4.8-1.0//usr/lib -L/opt/toolchains/stbgcc-4.8-1.0//lib  " make -j1 -C /home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/. AR="arm-linux-ar" AS="arm-linux-gcc -c -Os -pipe -march=armv7-a -mtune=cortex-a9 -fomit-frame-pointer -fno-strict-aliasing -marm -fno-caller-saves -mfloat-abi=hard" LD=arm-linux-ld NM="arm-linux-nm" CC="arm-linux-gcc" GCC="arm-linux-gcc" CXX="arm-linux-g++" RANLIB="arm-linux-ranlib" STRIP=arm-linux-strip OBJCOPY=arm-linux-objcopy OBJDUMP=arm-linux-objdump SIZE=arm-linux-size CROSS="arm-linux-" ARCH="arm" ;
make[4]: Entering directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
make[5]: Entering directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
make[6]: Entering directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
Scanning dependencies of target kmodloader
make[6]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
make[6]: Entering directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
[ 14%] Building C object CMakeFiles/kmodloader.dir/kmodloader.c.o
/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/kmodloader.c:35:25: fatal error: libubox/avl.h: No such file or directory
 #include <libubox/avl.h>
                         ^
compilation terminated.
make[6]: *** [CMakeFiles/kmodloader.dir/kmodloader.c.o] Error 1
make[6]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
make[5]: *** [CMakeFiles/kmodloader.dir/all] Error 2
make[5]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
make[4]: *** [all] Error 2
make[4]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06'
make[3]: *** [/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/ubox-2014-10-06/.built] Error 2
make[3]: Leaving directory `/home/promanus/projects/local/openwrt/package/system/ubox'
make[2]: *** [package/system/ubox/compile] Error 2
make[2]: Leaving directory `/home/promanus/projects/local/openwrt'
make[1]: *** [/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/stamp/.package_compile] Error 2
make[1]: Leaving directory `/home/promanus/projects/local/openwrt'
make: *** [world] Error 2
Post #2
tytus64
13 Feb 2015, 23:04

What I eventually end up doing is this:

diff --git a/package/system/ubox/Makefile b/package/system/ubox/Makefile
index de35d58..887c24a 100644
--- a/package/system/ubox/Makefile
+++ b/package/system/ubox/Makefile
@@ -43,4 +43,6 @@ define Package/ubox/install
        ln -s /sbin/kmodloader $(1)/usr/sbin/modprobe
 endef
 
+TARGET_CFLAGS += -I$(STAGING_DIR)/usr/include
+
 $(eval $(call BuildPackage,ubox))

To my surprise I had to add the same change to several other makefiles:

$ git status
On branch 3390
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   include/site/arm-linux
    modified:   package/network/config/firewall/Makefile
    modified:   package/network/services/odhcpd/Makefile
    modified:   package/system/fstools/Makefile
    modified:   package/system/procd/Makefile
    modified:   package/system/ubox/Makefile
    modified:   package/system/ubus/Makefile
    modified:   package/utils/jsonfilter/Makefile

Not sure if I am doing something wrong here (configuration???). But the changes made do not seem to be related to the custom toolchain and custom kernel...

Anybody wants to comment?

Post #3
jow
13 Feb 2015, 23:14

The include path problems are almost certainly related to a custom toolchain. Whats missing in your case is the magic performed by scripts/patch-specs.sh which is called by OpenWrt in the final build stage of GCC.

That script basically extends the builtin gcc command spec to automatically add -I and -L directives relative to the $STAGING_DIR environment variable, e.g. "-I$STAGING_DIR/include -I$STAGING_DIR/usr/include".

This is done to increase the portability of the toolchain, so that there's no need to funnel -I and -L flags through the entire build, autoconf, etc.

I do not know how you integrated that external toolchain exactly but if its a standard GCC based one you should be able to apply the patch-specs.sh script manually on it to solve your search path issues.

Post #4
tytus64
16 Feb 2015, 17:03

Thanks. Running scripts/patch-specs.sh fixed all include problems. The only change I had to do is the following:

diff --git a/include/site/arm-linux b/include/site/arm-linux
index 80cc8db..40f1a64 100644
--- a/include/site/arm-linux
+++ b/include/site/arm-linux
@@ -26,3 +26,5 @@ ac_cv_sizeof_unsigned_long=4
 ac_cv_sizeof_unsigned_long_long=8
 ac_cv_sizeof_unsigned_short=2
 ac_cv_sizeof_void_p=4
+ac_cv_func_malloc_0_nonnull=yes
+ac_cv_func_realloc_0_nonnull=yes

This change was necessary to fix the following problem:

make[6]: Entering directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/libubox-2015-01-28'
Linking C executable jshn
/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib/libjson-c.so: undefined reference to `rpl_malloc'
/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/usr/lib/libjson-c.so: undefined reference to `rpl_realloc'
collect2: error: ld returned 1 exit status
make[6]: *** [jshn] Error 1
make[6]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/libubox-2015-01-28'
make[5]: *** [CMakeFiles/jshn.dir/all] Error 2
make[5]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/libubox-2015-01-28'
make[4]: *** [all] Error 2
make[4]: Leaving directory `/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/libubox-2015-01-28'
make[3]: *** [/home/promanus/projects/local/openwrt/build_dir/target-arm-linux/libubox-2015-01-28/.built] Error 2
make[3]: Leaving directory `/home/promanus/projects/local/openwrt/package/libs/libubox'
make[2]: *** [package/libs/libubox/compile] Error 2
make[2]: Leaving directory `/home/promanus/projects/local/openwrt'
make[1]: *** [/home/promanus/projects/local/openwrt/staging_dir/target-arm-linux/stamp/.package_compile] Error 2
make[1]: Leaving directory `/home/promanus/projects/local/openwrt'
make: *** [world] Error 2

I wonder if I missed something else or this is a legitimate bug?

(Last edited by tytus64 on 16 Feb 2015, 17:03)

Post #5
jow
17 Feb 2015, 13:31

I guess it is a kind of semi-legitimate bug. Its likely only encountered by your toolchain because it uses a target name triplet which is not used by any other OpenWrt target.

The discussion might have continued from here.