When building a firmware image, is there a way to apply `scripts/diffconfig.sh` to the kernel's `.config`?

For Developers
1

I build my openwrt firmware images from source (clone the git repo and build them with make) (not image-builder).

Ive tried tweaking the kernel config a few times and, well, the way the openwrt build system is set up makes it really hard to get it right. From a good bit of trial and error, Ive figured out 2 ways that at least have a chance of building a kernel that isnt broken, but both sort of suck:

Option 1: start by running make [-j$(nproc) V=sc] prepare_kernel_conf kernel_menuconfig

This doesnt actually show you the kernel's configuration in the kernel menuconfig . menu. I think what is happening here is that you are being shown some generic "base" config, and the specific selections you choose are then recorded and just those selections get layered on top of the "real" kernel config when you build the kernel.

Trying to configure the kernel without knowing the kernel base configuration is error prone and time consuming.

Option 2: build a kernel, steal its .config, and then reconfigure and rebuild the kernel.

If you run a make [-j$(nproc) V=sc] prepare, the .configfor the just-built kernel is available atbuilddir/target_<GCC_ARCH>/linux-<OPENWRT_ARCH>/linux-<KERNEL_VERSION>/.configIf youi take that and use it to overwrite the file attargets/<OPENWRT_TARGET>/config-X.Yand then runmake target/____/clean kernel_menuconfig` it will show you the actual kernel configuration.

The problem here is that you fuul it the full kernel .config as input, and so it treats it as if you manually selected every single config option, which breaks ass the automatic co-dependencies between options.

example: if you select option A and option B is automatically pulled in as a dependent option. If you then remove option A, option B get removed as well.

The same happens if you select A (with B automatically pulled in), export to .config, load .config, and then remove A - option B is stil there, because when you loaded the .config you in effect manually selected it (and all the other options)

But, if you select option B manually and then select option A, removing option A doesnt remove option B. Since dependencies are 1 way, you often get "A depends on B, B doesnt explicitly depend on A but will be useless or even actively harmful is A is not also present".

As such, reconfiguring a kernel where it thinks you manually selected every opotion and the automatic dependency system doesnt change the rest of the config as required, it is hard to build a working kernel.

Hence, my question. The diffconfig script basically reduces the .config down to just the options that were explicitly selected (manually or via a config file). loading this reduced "diffconfig" instead of the full .config will only select/deselect options we manually tweaked, which would make it much easier to get a working custom kernel (using "method 2" above).

2

Almost, but not exactly.
It reduces it to

  • the packages & options that were explicitly selected,
  • AND their dependencies.
    So, diffconfig output requires a bit further pruning to remove the dependencies. (E.g. luci-app-sqm pulls in sqm-scripts, which pulls in xxx..., so only luci-app-sqm is needed, etc. )

Regarding kernel options, they are normally read from the generic common options (for all targets), and the target and subtarget specific options.

E.g. for ipq806x routers first generic from https://github.com/openwrt/openwrt/blob/master/target/linux/generic/config-6.1 and then target-specific from https://github.com/openwrt/openwrt/blob/master/target/linux/ipq806x/config-6.1

Your idea of overwriting that target config with the full built config seems over-extensive.

Git diff might be your best tool for noting differences.
At least when I do kernel_menuconfig changes, the changes do get applied to the git controlled files.
e.g.

perus@ub2304:/Openwrt/r7800-2203$ git diff | head -n 25
diff --git a/target/linux/ipq806x/config-5.10 b/target/linux/ipq806x/config-5.10
index 8cc83652e0..de02e115d7 100644
--- a/target/linux/ipq806x/config-5.10
+++ b/target/linux/ipq806x/config-5.10
@@ -47,6 +47,7 @@ CONFIG_ARM_QCOM_SPM_CPUIDLE=y
 CONFIG_ARM_THUMB=y
 CONFIG_ARM_UNWIND=y
 CONFIG_ARM_VIRT_EXT=y
+CONFIG_ASSOCIATIVE_ARRAY=y
 CONFIG_AT803X_PHY=y
 CONFIG_BINFMT_FLAT_ARGVP_ENVP_ON_STACK=y
 CONFIG_BLK_DEV_LOOP=y
@@ -99,11 +100,14 @@ CONFIG_CRC16=y
 # CONFIG_CRC32_SARWATE is not set
 CONFIG_CRC32_SLICEBY8=y
 CONFIG_CRYPTO_BLAKE2S=y
+CONFIG_CRYPTO_CBC=y
+CONFIG_CRYPTO_CTS=y
 CONFIG_CRYPTO_DEFLATE=y
 CONFIG_CRYPTO_DEV_QCOM_RNG=y
 CONFIG_CRYPTO_DRBG=y
 CONFIG_CRYPTO_DRBG_HMAC=y
 CONFIG_CRYPTO_DRBG_MENU=y
+CONFIG_CRYPTO_ECB=y
 CONFIG_CRYPTO_HASH_INFO=y
3

Oh. Well, in that case, yeah I suppose a simple diff would do it.

That said, somewhere there has to be information about just the options you selected, right? I'm pretty sure you can make a change in menuconfig, save it, close menuconfig, reopen menuconfig, and undo the change you made and any other config changes that automatically happened get reversed as well. Assuming Im correct about this it would only be possible if the information about the specific options you selected is stored somewhere...

It is absolutely over-extensive (hense this post looking for a better way). I used that solution not because it was a good solution, but because it was a the only solution I knew about.

And I know about the generic --> target --> subtarget layering. The problem is that running make menuconfig doesnt show you all these. In particular all the kernel config modifications that happen automatically based on what you choose during "menuconfig" arent there, which makes it realy hard to configure the kernel without accidentally screwing up some of these necessary but invisible automatic kernel config changes.

The only solution I found here (which is again not a good one, but I dont know of an alternative) was to build the kernel without running kernel_menuconfig, copy its .config into the targets/<...>/config-6.1 for that target, then run kernel_menuconfig. Doing this I could at least see what the actual full kernel menuconfig was, but had the problems described in my original post.

I suppose I could save this full config, then rever to the original targets/<...>/config-6.1 file, regenerate the .config, diff them and add the differences to the targets/<...>/config-6.1 file. But It would be nice is there was a more straightforward solution here.

Related side question: the wiki says that you can add a CONFIG_TARGET=[target|subtarget|env] flag to the kernel_menuconfig call. Ive always just sort of ignored it since it isnt really clear what this does, bt any chance that setting that (perhaps to env?) would cause the kernel_menuconfig to pull in all the current changes without having to go through all this?

4

Sorry, but that information is not stored.
The .config is just a simple flat file about option selections, but there is no info about the reason why an option has been toggled.

That is pretty much what the normal diffconfig does for the normal .config

Personally I keep a short .config recipe, originally pruned from diffconfig.
See

Regarding kernel config, you might look into my earlier suggestion about git diff, and using is output.

5

I think I've figured out a halfway decent way to more-or-less do what I was trying to do.

If you have any suggestions/modifications that would improve the experience feel free to suggest them.

# setup package feeds
./scripts/feeds update -a
./scripts/feeds install -a
./scripts/feeds install -a

# configure openwrt settings
make menuconfig

# build unmodified kernel 
make -j$(nproc) V=sc download check prepare

# save important paths
target_board="$(grep -F CONFIG_TARGET_BOARD <.config | sed -E 's/^[^"]*"//;s/".*$//')"
target_kconfig="$(ls target/linux/${target_board}/config-*)"
builddir_kconfig="$(ls build_dir/target*/linux-${target_board}*/linux-*/.config)"

# copy full default config of the (just built) default kernel
mv "${builddir_kconfig}" .config.kernel

# generate the incomplete .config (for the kernel that kernel_menuconfig shows) in the kernel build dir
# dont change anything in kernel_menuconfig - immediately save it (to .config) and exit. 
make kernel_menuconfig

# diff and add to target config to get target diffconfig of full default kernel config
diff .config.kernel "${builddir_kconfig}" | grep '<' | sed -E s/'^< '// >> target/linux/qualcommax/config-6.1

# remove incomplete .config out of kernel build dir
rm "${builddir_kconfig}" 

# regenerate .config (which gives the full actual kernel) and actually custom configure the kernel this time -- save as .config when done
make kernel_menuconfig

# diff and add to target config to get diffconfig of full custom kernel config
# note: diff arguments are flipped from previous diff command
diff "${builddir_kconfig}" .config.kernel | grep '<' | sed -E s/'^< '// >> target/linux/qualcommax/config-6.1

# save full .config and fix up target config. 
# dont change anything in kernel_menuconfig - immediately save it (to .config) and exit
mv "${builddir_kconfig}" .config.kernel
make kernel_menuconfig

# make kernel
make -j$(nproc) V=sc prepare

# make the rest of the build
make -j$(nproc) -k V=sc