Failed to compile multi types of images of one device when multi CPU cores are used to compile

For Developers
1

Hi All,

I'm adding a new device and want to generate multi types of images in different user cases(For example, one case is used in sysupgrade via GUI and another case is used in upgrade via uboot and so on). So I have something like below in the makefile.

define Device/mydevice
   DEVICE_VENDOR := vendor_name
   KERNEL := action k1 | action k2 | action k3 ...
   ......
   IMAGES :=a.bin
   IMAGES +=b.bin
   IMAGES +=c.bin
   IMAGE/a.bin = action a1 | action a2 | action a3 .....
   IMAGE/b.bin = action b1 | action b2 | action b3 .....
   IMAGE/a.bin = action c1 | action c2 | action c3 .....

The problem is b1/b2/b3 and c1/c2/c3 depends on a.bin. It means a1/a2/a3 must run before b1/b2/b3 and c1/c2/c3. It's OK if I use one CPU core to compile, but it may fail when multi cores(like -j 32) are used because b1/b2/b3 and c1/c2/c3 may run before a.bin is generated.

One working way is to replace k1/k2/k3 with a1/a2/a3 because it seems that KERNEL always run first. However, this way is not elegant and confusing even though I don't need a standalone kernel file.

How to solve this problem? Thanks.

Tony

2

Simple solution on this, never ever use more than one core if you want to maximize the chance for positive build result.

But this applies all the time, not only your multi-build settings.

3

Compiling with single core is too slow, which is not a good solution.

4

As long as you use more than one core you will always end up in the chance that different parts of the build get finished before other parts and that will end up in a failed build.

5

After tracing the git history, I found below patch related to my problem.

commit 43be5087a915727e7dcb3459e2221f094c70811b
Author: Christian Lamparter <chunkeey@gmail.com>
Date:   Mon May 1 17:40:16 2017 +0200

    build: make image target wait for initramfs

    The image production rules does not have the initramfs-image
    as a dependency. So, from make’s perspective initramfs
    creation can run independently/in parallel with the image
    generation code in the target's Makefile.

    This is a problem for devices that have to use the initramfs
    for the image creation and can lead to broken images.

    Signed-off-by: Christian Lamparter <chunkeey@gmail.com>

diff --git a/include/image.mk b/include/image.mk
index 771f5b2c76..64c224e55f 100644
--- a/include/image.mk
+++ b/include/image.mk
@@ -480,7 +480,7 @@ define Device/Build/image
   ifndef IB
     $$(ROOTFS/$(1)/$(3)): $(if $(TARGET_PER_DEVICE_ROOTFS),target-dir-$$(ROOTFS_ID/$(3)))
   endif
-  $(KDIR)/tmp/$(call IMAGE_NAME,$(1),$(2)): $$(KDIR_KERNEL_IMAGE) $$(ROOTFS/$(1)/$(3))
+  $(KDIR)/tmp/$(call IMAGE_NAME,$(1),$(2)): $$(KDIR_KERNEL_IMAGE) $$(ROOTFS/$(1)/$(3)) $$(if $$(CONFIG_TARGET_ROOTFS_INITRAMFS),$(KDIR)/tmp/$$(KERNEL_INITRAMFS_IMAGE))
        @rm -f $$@
        [ -f $$(word 1,$$^) -a -f $$(word 2,$$^) ]
        $$(call concat_cmd,$(if $(IMAGE/$(2)/$(1)),$(IMAGE/$(2)/$(1)),$(IMAGE/$(2))))

If I add something like "$(KDIR_TMP)/$(DEVICE_IMG_PREFIX)-$(FILESYSTEMS)-a.bin" in the same place, my issue is solved. I know this is not elegant either, but it's a good start to solve this problem.