Replicating release builds (userspace packages)

I've had no problem checking out sources and building the firmware image with target-specific packages, but when it comes to building the userspace packages I've hit a brick wall of frustration.

What I am looking for is an easy way to automatically build all userspace packages that are built for the releases. Are they listed somewhere? Is there a way to run a build process that would result in the very same set of userspace packages found in the releases?

Yes, I know about CONFIG_ALL, but that just doesn't work. Firstly, it will build absolutely everything including stuff that no sane person would ever want in an embedded device and that of course are nowhere to be found in the releases userspace packages directories. And secondly several packages (that again, are not built in the releases) will fail to compile.

Surely I could just go through all the userspace packages in the release and painstakingly select them in the configuration menu, but there must be an automated way to do this. Surely maintainers do not do this for each release.

Any help would be much appreciated.

Thanks.

This is explained in the build system documentation [OpenWrt Wiki] Build system usage
'Using official build config'

2 Likes

No it is not. By following those instructions only the firmware image and the target specific packages are built. None of the userspace packages are enabled in config.buildinfo. And as I said in my initial post, CONFIG_ALL=y will result in all userspace packages being compiled which is nuts. I just want to compile the userspace packages that are compiled and made available for download in the official releases.

Nobody knows how the release userspace package list is built?

If you want 'everything', but "not really everything, as that would be madness" (and I agree, that would be madness for personal builds), you will have to make your own config decisions. You've been advised about the official build config, and the buildbot setup is also publicly available - but I'm sad to inform you, those are intended to build the mad wide world of 'everything'.

1 Like

If the buildbot is intended to build everything, then why aren't all packages (as in CONFIG_ALL=y) in the release packages download dir?

In my case: https://downloads.openwrt.org/releases/23.05.3/packages/mipsel_24kc/

For example libpfring isn't there (and incidentally also fails to build). Just like rust.

I just want to be able to replicate the release file set for one target, including userspace packages (mind you, not the exact binaries which I understand might be difficult, just the same available packages).

Regarding buildbot, yes, I guess what I'm looking for must be there somewhere since it does indeed build only the packages that end up in the release, but I know next to nothing about buildbot so I was hoping someone could help with some directions if that's the route I should follow.

https://downloads.openwrt.org/releases/faillogs-23.05/mipsel_24kc/packages/libpfring/

Thank you. As I said I know next to nothing about how buildbot does its thing. Will fix that.

In any case, libpfring is just an example. There's plenty of userspace packages that become selected with CONFIG_ALL=y that just aren't there in the release packages directory.

To be clear, I'm kind of OK with just spending a good bunch of hours in menuconfig selecting the packages I need and creating a firmware image. Used to do that, but then moved to using the imagebuilder which is a far superior method. I just want to do that locally, replicating the list of userspace packages that are built in the release.

I tried to follow the offical build config referenced above. However, I'm presented with compilation errors for libpfring (for v23.05.3, TP-Link Archer C7v2). I'd like to check here if this is a known issue ?

Details:
As sanity check, I'd like to ask for feedback if I'm following the correct procedure (which worked well before on 22.x), compiling inside WSL v2 using Ubuntu 22.04.4 LTS:

$ history
sudo apt install build-essential clang flex bison g++ gawk gcc-multilib g++-multilib gettext git libncurses5-dev libssl-dev python3-setuptools rsync swig unzip zlib1g-dev file wget
git clone https://git.openwrt.org/openwrt/openwrt.git
cd openwrt/
git fetch -t
git tag
git checkout v23.05.3
wget https://downloads.openwrt.org/releases/23.05.3/targets/ath79/generic/config.buildinfo -O .config
./scripts/feeds update -a && ./scripts/feeds install -a
make defconfig
make world
make ./package/feeds/packages/libpfring/compile  -j1 V=s

This however shows the following (which is identical to the buildbot log above as shared by dave14305 ):

...
make[2]: Entering directory '/home/sdc/openwrt/feeds/packages/libs/libpfring'
rm -f /home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/.built
touch /home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/.built_check
make -C "/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/linux-5.15.150" KERNEL_DIR="/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/linux-5.15.150" ARCH="mips" CROSS_COMPILE="mips-openwrt-linux-musl-" M="/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel" EXTRA_CFLAGS=" -I/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel" modules
make[3]: Entering directory '/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/linux-5.15.150'
  CC [M]  /home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.o
/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.c: In function 'ring_bind':
/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.c:5608:19: error: invalid application of 'sizeof' to incomplete type 'char[]'
 5608 |   char name[sizeof(sa->sa_data)+1];
      |                   ^
/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.c:5620:35: error: invalid application of 'sizeof' to incomplete type 'char[]'
 5620 |   memcpy(name, sa->sa_data, sizeof(sa->sa_data));
      |                                   ^
/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.c:5608:8: error: unused variable 'name' [-Werror=unused-variable]
 5608 |   char name[sizeof(sa->sa_data)+1];
      |        ^~~~
/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.c:5628:1: error: control reaches end of non-void function [-Werror=return-type]
 5628 | }
      | ^
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:289: /home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel/pf_ring.o] Error 1
make[3]: *** [Makefile:1908: /home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/kernel] Error 2
make[3]: Leaving directory '/home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/linux-5.15.150'
make[2]: *** [Makefile:92: /home/sdc/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/PF_RING-8.0.0/.built] Error 2
make[2]: Leaving directory '/home/sdc/openwrt/feeds/packages/libs/libpfring'
time: package/feeds/packages/libpfring/compile#2.25#0.23#2.38
    ERROR: package/feeds/packages/libpfring failed to build.
make[1]: *** [package/Makefile:129: package/feeds/packages/libpfring/compile] Error 1
make[1]: Leaving directory '/home/sdc/openwrt'
make: *** [/home/sdc/openwrt/include/toplevel.mk:232: package/feeds/packages/libpfring/compile] Error 2

Unless you have a reason to, don't. Use a tailored configuration for your device(s) only, including everything you want to have in there (=y), but nothing beyond that. The official build config is building 'everything butand the kitchen sink', literally. Doing so just wastes you dozens of GB build space and hours of CPU crunching for no reward, aside from being presented with each and every build failure (which the official buildbots just ignore and silently drop from the package lists).

Packages may fail to build, for a variety of reasons - the real question would be, do you need that packages or do you care. If you do, you'll need to fix it.

Thank you - Is there a way to silently drop the packages that are causing errors when building locally too ?

My main objective is to have a release build equivalent that allows to simply install modules from the repository (for convenience). I tend to set up the build environment once, and thereafter just recompile the packages that I am targeting. The build space and time are only a minor concern to me.

--- Update ---
anomeome was quicker than me...

The build page [OpenWrt Wiki] Build system usage has this information under Ignore Build Errors....

Essentially, one sets it at the commandline:
IGNORE_ERRORS=1 make ...

this is exactly right - it turns out that the .vermagic is not affected by libpfring. Hence it can be safely removed from the package build list. I'd speculate that this is probably true for all packages.

Also for those that want to do this too - the libpfring is found under Network Support in the menuconfig. It took me a moment to locate it.

ignore stuff

1 Like

Id do have a reason: I want for my devices to use my own package repo, and for now I want the flexibility of having a package repo.

Sure, a tailored version would be optimal... if I was sure at the moment of the packages I want to include in it.

Also note that building the whole shebang might use a lot of CPU and storage, but you do it once and when it's done, using image builder gives you a tailored image super fast.

One thing you suggested that I had missed is IGNORE_ERRORS=1. I'll try that and see whether the result contains the same packages than the release, but I'm not optimistic. As I said, CONFIG_ALL=y selects even Rust that as far I can remember is successfully built but is not in the official release package repo.

This is being super frustrating. I just want to produce the same set of packages in the official release and no one seems to know how (besides "look at what buildbot does", which I'm building up to do).

I agree that one should expect the release build to not fail.
Like you, I also am not looking forward to go through trial-and-error cycles to find build issues interactively. Doing a full compile, even with a silent fail, that just runs unsupervised (e.g. overnight) is in my opinion far more end-user friendly...