How to use imagebuilder to build image same packages as buildbot?

comicfans · March 15, 2019, 8:40am

Hello, I can build image from full openwrt source(to customize kernel a little without module ABI break), but it spends lots of time (build host/target toolchain), and I'd like to try SDK / imagebuilder(which has toolchain prebuilt) but got confused:

SDK build packages only, didn't generate target image
ImageBuilder built target image, but only contains very bare bone packages, full openwrt source can read config file (which came from openwrt config.seed) to build image with same packages, but Image builder must manually add PACKAGE variable (I've read buildbot default packages from packages.manifest)

my question1 : does openwrt imagebuilder provide 'standard' way to build image with exactly same set of packages as config.seed ? (https://openwrt.org/docs/guide-user/additional-software/imagebuilder has a section Building the Image Generator with all packages inside, but that seems to build imagebuilder itself from full openwrt source)

my question2 : how does imagebuilder generate the image kernel ? is it built from source , or just extracted from prebuilt binary and repacked ? if it is built from source , does it follow the same build process just like full openwrt source ? (apply all patches and build)

Thanks for advise .

mhel · March 19, 2019, 3:04pm

Being a newbie myself, I can only answer no. 2 since I'm still struggling to build from source.
If you use the image builder you're basically repackaging your own firmware with prebuilt .ipk's
Atleast from I've what tried, I didn't have to rebuild the imagebuilder.
From the link in your post you can find this example :
make image PROFILE=rt-n14u
$ make image PACKAGES="pkg1 pkg2 pkg3 -pkg4 -pkg5 -pkg6"

That's basically what you need to have a custom firmware, that is in terms of what is included on
your build without using opkg install afterwards.
Ofcourse if you really need to customize then you have to build from source.

jeff · March 19, 2019, 3:17pm

With the caveat that I don't use the image builder, but build from source, the "changes from default" are typically described in config.seed. For example, for an ar71xx device on the "generic" target, http://downloads.openwrt.org/releases/18.06.2/targets/ar71xx/generic/config.seed, the "interesting" lines are

CONFIG_PACKAGE_libiwinfo-lua=y
CONFIG_PACKAGE_liblua=y
CONFIG_PACKAGE_liblucihttp=y
CONFIG_PACKAGE_liblucihttp-lua=y
CONFIG_PACKAGE_libubus-lua=y
CONFIG_PACKAGE_lua=y
CONFIG_PACKAGE_luci=y
CONFIG_PACKAGE_luci-app-firewall=y
CONFIG_PACKAGE_luci-base=y
CONFIG_PACKAGE_luci-lib-ip=y
CONFIG_PACKAGE_luci-lib-jsonc=y
CONFIG_PACKAGE_luci-lib-nixio=y
CONFIG_PACKAGE_luci-mod-admin-full=y
CONFIG_PACKAGE_luci-proto-ipv6=y
CONFIG_PACKAGE_luci-proto-ppp=y
CONFIG_PACKAGE_luci-theme-bootstrap=y
CONFIG_PACKAGE_rpcd=y
CONFIG_PACKAGE_rpcd-mod-rrdns=y
CONFIG_PACKAGE_uhttpd=y

Many of these are dependencies. I'm guessing that enabling a LuCI collection will bring them all in.

The full list of packages in the image it typically in the maniifest, http://downloads.openwrt.org/releases/18.06.2/targets/ar71xx/generic/openwrt-18.06.2-ar71xx-generic.manifest, for the same target. Note that many of these are "default" for the generic platform.

comicfans · March 20, 2019, 2:26am

Thanks for suggestion, I've tried imagebuilder to custom packages, but seems it also repack prebuilt kernel so I can not patch kernel and generate custom kernel image.

jeff · March 20, 2019, 2:28am

That seems correct. You likely need the full build system (and quilt, if you want to patch the kernel)

Building “all” packages “at home” is a reasonably futile endeavor. You’d need a buildbot setup to accomplish that.

comicfans · March 20, 2019, 2:50am

Thanks for suggestion, I can build full image by using my device config.seed.
and I have to type some bash such as 'cat generic.manifest | cut -d " " -f 1 ...' to convert this file into imagebuilder accepted arguments. I wonder if imagebuilder provide quick way such as
'make manifest=mydevice.manifest' to achieve this ?
anyway, if imagebuilder just repack prebuilt kernel instead of compiling kernel, I can't use it to custom kernel.

I really hope openwrt can extend SDK function to support build kernel and image. so I don't need to build whole toolchain to just custom kernel a little

jeff · March 20, 2019, 2:53am

There’s very little difference in host requirements between the two. Once you’ve got the toolchain built, with ccache enabled, image builds are only a few minutes each.

On a mid-range system, it takes me 16 minutes from a fresh clone to image, including building the toolchain.

comicfans · March 20, 2019, 3:10am

Emm... seems your mid-range system is much more powerful than mine

jeff · March 20, 2019, 3:17am

ccache makes a huge difference, once it’s primed!

comicfans · March 20, 2019, 3:44am

I only build openwrt for major release . ccache's cache already wiped ( previous build is about half a year ago)

okibcn · February 1, 2022, 5:54am

I have created a shell script to build any image exactly like the official one — same set of packages — but with updated versions.
In addition to that, you can add more packages, or even add some local packages. It uses the imagebuilder and takes the list of packages from official builds.

https://github.com/okibcn/OpenWrt-R6020/blob/main/build.sh

By the way, it also supports development snapshots builds.