Snapshots fail to build because of llvm-bpf

For Developers
1

For some time, snapshots have failed to build on my Mint/Ubuntu devel environment. Ever since qosify was added back in Nov of last year. The build system is failing to use the internal llvm-bpf. Is there a way to get the build system to use its internal llvm-bpf?

2

I have built snapshots for my RPi4 on Ubuntu WSL which is installed on my Win10_Prox64. No problem so far. What error you're getting?

3

Just download and extract the llvm toolchain binary from the buildbot download repo, and set two config options in menuconfig.

See

4

Perfect, thanks. This should at least get the builds working.

I'm curious, do you know what I need to do to get BPF_TOOLCHAIN_BUILD_LLVM=y working so that it builds llvm like it builds the rest of the toolchain? Clearly this is working for some people, since this is the setting in the default .config for my platforms. Which means the auto snapshot build system must do this.

5

Probably do not want to do that, to say that it is slow would be a gross understatement. For another option install latest from upstream.

1 Like
6

I've tried with the prebuilt llvm-bpf twice now, and still have issues. As soon as it goes to use the prebuilt clang, it errors out:

make[4]: Entering directory '/home/kfitzner/devel/openwrt/build_dir/target-aarch64_cortex-a53_musl/xdp-tools-1.2.8'

lib

  libxdp
    CC       staticobjs/libxdp.o
    CC       staticobjs/xsk.o
    CLANG    xdp-dispatcher.o
In file included from xdp-dispatcher.c:3:
In file included from ../../headers/linux/bpf.h:11:
/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
#include <asm/types.h>
         ^~~~~~~~~~~~~

This is on an Ubuntu 22.04 machine. My build steps are as follows:

git clone https://git.openwrt.org/openwrt/openwrt.git
cd openwrt
git pull
git checkout master
./scripts/feeds update -a
./scripts/feeds install -a
wget https://downloads.openwrt.org/snapshots/targets/mediatek/mt7622/config.buildinfo -O .config
wget https://downloads.openwrt.org/snapshots/targets/mediatek/mt7622/llvm-bpf-14.0.6.Linux-x86_64.tar.xz
tar -xvaf llvm-bpf-14.0.6.Linux-x86_64.tar.xz
make defconfig
make menuconfig
# Turn off "Build the LLVM-BPF toolchain tarball (NEW)" and set
# Advanced configuration options (for developers)  ---> BPF Toolchian to
# "Use Prebuild LLVM Toolchain"
make download
make -j 6 world

Any advice on what I've missed? It must be a path issue inside the build system, but I don't even begin to know where to tweak that.

7 
# CONFIG_BPF_TOOLCHAIN_BUILD_LLVM is not set
CONFIG_BPF_TOOLCHAIN_HOST_PATH="/usr/lib/llvm-15"
# CONFIG_HAS_PREBUILT_LLVM_TOOLCHAIN is not set
CONFIG_USE_LLVM_HOST=y

what I have in .config, installation from the above upstream.

1 Like
8

I found I was getting the error regardless of what llvm I use (host or pre-built). It looks to me like there is a bug in the build system. I can't tell for sure, because even with "make V=sc world" the build system is still not displaying the command lines it is using in all cases. But looking at the error (shown in the log below) it looks like it is improperly using include files from the host system. I don't think it should be using /usr/include/linux/types.h.

make[4]: Entering directory '/home/kfitzner/devel/openwrt/build_dir/target-aarch64_cortex-a53_musl/xdp-tools-1.2.8'

lib

  libxdp
    CC       staticobjs/libxdp.o
    CC       staticobjs/xsk.o
    CLANG    xdp-dispatcher.o
In file included from xdp-dispatcher.c:3:
In file included from ../../headers/linux/bpf.h:11:
/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
#include <asm/types.h>
         ^~~~~~~~~~~~~
1 error generated.

I temporarily fixed the issue by sudo ln -s x86_64-linux-gnu/asm /usr/include/asm to give the openwrt build access to my system's asm/types.h, and the build succeeded. This is less than ideal, since it shouldn't be pulling in my x86_64 types.h.

9

I have also finally got builds to work using the included llvm-bpf (rather than host or pre-packaged). The instructions on the build system usage page are incorrect now.

Specifically, "make download" will fail for qosify if the build system is set to use the built-in llvm unless it is preceded with "make toolchain/install".

Before I change the wiki page for build system usage, can anypne verify whether or not this is a bug in the qosify makefile or intended?