18.06-snapshot image builder for mvebu -- conflict between busybox & coreutils-sort

For Developers
1

Can't build an image which includes coreutils-sort with the 18.06 snapshot image builder. Current 17.01.4 release image builder works fine and doesn't produce the error below. Also, 18.06 snapshot includes coreutils-timeout just fine.

make image PROFILE=linksys-wrt3200acm PACKAGES="coreutils-sort"
Collected errors:
 * check_data_file_clashes: Package busybox wants to install file [...]/mvebu_18.06_snapshot/build_dir/target-arm_cortex-a9+vfpv3_musl_eabi/root-mvebu/usr/bin/sort
	But that file is already provided by package  * coreutils-sort
 * opkg_install_cmd: Cannot install package busybox.
2

Remove sort in busybox's coreutils config, clean and recompile busybox.

EDIT: I missed that You use image builder, sorry! The only way is build image without coreutils-sort and forced install this package after flashing.

3

Thanks for your reply. This was posted in hopes that it'd reach core dev. Seems to be a bug in 18.06.

4

The dependency handling was made a bit more clear, and conflicts are apparently noticed better.

One typical solution would be to change the install path of the sort package to a different that has precedence on PATH. That is e.g. how ip gets handled. There is busybox ip and also iproute up at the same time.

But for practicals with image builder: download sort ipk, extract the executable and insert it into image as a custom file into /usr/sbin

5

I just tested with 17.01, and trying to install coreutils-sort into a live 17.01 fails with the same conflict message:

root@router2:~#opkg install coreutils-sort
Installing coreutils-sort (8.23-3) to root...
Downloading http://downloads.lede-project.org/releases/17.01-SNAPSHOT/packages/mips_24kc/packages/coreutils-sort_8.23-3_mips_24kc.ipk
Installing coreutils (8.23-3) to root...
Downloading http://downloads.lede-project.org/releases/17.01-SNAPSHOT/packages/mips_24kc/packages/coreutils_8.23-3_mips_24kc.ipk
Configuring coreutils.
Collected errors:
 * check_data_file_clashes: Package coreutils-sort wants to install file /usr/bin/sort
        But that file is already provided by package  * busybox
 * opkg_install_cmd: Cannot install package coreutils-sort.

So, if you succeeded with the 17.01 imagebuilder, the bug is actually there (as it appears to allow packages to install conflicting files) ...

6

@jow is the maintainer of coreutils, so might by worthwhile to get his input

The underlying reason is of course a conflict in the installation paths by busybox applets and coreutils-apps. The conflict has been there for ages.

Busybox mostly installs into /bin but some apps like most coreutils apps do get installed into /usr/bin. That is likely in order to mimic the "real" coreutils applications. Coreutils installs apps into /usr/bin

This conflict has already been fixed for some other apps like traceroute and passwd, which have been patched from /usr/bin to /bin in Openwrt busybox.

I browsed the source code and found out the following:

There are roughly three kinds of coreutils apps duplicated in busybox:

  • apps not installed by default in busybox. opkg installing them should go ok, as the user should have explicitly changed his busybox build config to generate a conflict.

    base64 chcon cksum comm expand factor false fold groups hostid install link logname nl nohup nproc od paste printenv printf realpath runcon sha1sum sha224sum sha384sum sha512sum shred shuf split stat stty sum tac timeout tty unlink users truncate unexpand who whoami

  • installed by default in busybox, but to a different path. No conflict:

    cat chgrp chmod chown cp date dd echo kill ln ls mkdir mknod mktemp mv nice pwd rm rmdir sleep sync touch true uname

  • installed by default in busybox to the same path /usr/bin or /usr/sbin. Conflict:

    basename cut dirname du env head id md5sum mkfifo readlink seq sort sha256sum tail tee test tr uniq uptime wc yes

    conflict in /usr/sbin: chroot

Only this last group is actually interesting, as those might be fixed in busybox to enable installing corresponding coreutils into a live system or image (that has been compiled with default busybox).

Considering the last group, I feel that many of them are so simple that users really rarely do install them. Thinking from performance perspective, @stangri's example of "sort" is probably the one that will get installed due to performance (memory utilisation) reasons. But are others worthwhile to fix?

The busybox fix itself would be simple. This is the fix for sort app:

--- a/coreutils/sort.c
+++ b/coreutils/sort.c
@@ -29,7 +29,7 @@
 //config:	The SuSv3 sort standard is available at:
 //config:	http://www.opengroup.org/onlinepubs/007904975/utilities/sort.html
 
-//applet:IF_SORT(APPLET_NOEXEC(sort, sort, BB_DIR_USR_BIN, BB_SUID_DROP, sort))
+//applet:IF_SORT(APPLET_NOEXEC(sort, sort, BB_DIR_BIN, BB_SUID_DROP, sort))
 
 //kbuild:lib-$(CONFIG_SORT) += sort.o
7

A few points:

17.01.4 image builders do not have the conflict neither between busybox and coreutils-sort nor between busybox and coreutils-timeout.

18.06 snapshot image builder for mvebu has a conflict between busybox and coreutils-sort, but not between busybox and coreutils-timeout.

8

Like I explained above, "timeout" is not expected to generate conflict with the default busybox even with 18.06 as it is not in default busybox.

As a live 17.01 opkg has conflict for "sort" but imagebuilder does not, it seems to me that there has been a bug in the 17.01 imagebuilder that has been fixed now in 18.06 and master, bringing the conflict into highlight.

The fix is to remove the conflict from the packages, not to hide it by allowing imagebuilder to misbehave :wink:

Two basic solutions:

  • change install paths in busybox to /bin (like most busybox apps already). Has been done earlier, so far the standard solution
  • change install paths in coreutils. /usr/sbin would be first in PATH, overriding /usr/bin
1 Like