[SOLVED] Broken build on Trunk, regarding curl (libzstd dependency)

For Developers
1

My recent custom build for MT7620A is broken, now:
(libcurl: Missing dependency zstdlib)

make[3]: Entering directory '/etc/openwrt/openwrt/package/network/utils/curl'
mkdir -p /etc/openwrt/openwrt/bin/targets/ramips/mt7620/packages /etc/openwrt/openwrt/build_dir/target-mipsel_24kc_musl/curl-7.72.0/ipkg-mipsel_       24kc/libcurl/CONTROL /etc/openwrt/openwrt/staging_dir/target-mipsel_24kc_musl/pkginfo
install -d -m0755 /etc/openwrt/openwrt/build_dir/target-mipsel_24kc_musl/curl-7.72.0/ipkg-mipsel_24kc/libcurl/usr/lib
cp -fpR /etc/openwrt/openwrt/build_dir/target-mipsel_24kc_musl/curl-7.72.0/ipkg-install/usr/lib/libcurl.so.* /etc/openwrt/openwrt/build_dir/targ       et-mipsel_24kc_musl/curl-7.72.0/ipkg-mipsel_24kc/libcurl/usr/lib/
find /etc/openwrt/openwrt/build_dir/target-mipsel_24kc_musl/curl-7.72.0/ipkg-mipsel_24kc/libcurl -name 'CVS' -o -name '.svn' -o -name '.#*' -o -       name '*~'| xargs -r rm -rf
Package libcurl is missing dependencies for the following libraries:
libzstd.so.1
Makefile:188: recipe for target '/etc/openwrt/openwrt/bin/packages/mipsel_24kc/base/libcurl4_7.72.0-1_mipsel_24kc.ipk' failed
make[3]: *** [/etc/openwrt/openwrt/bin/packages/mipsel_24kc/base/libcurl4_7.72.0-1_mipsel_24kc.ipk] Error 1
make[3]: Leaving directory '/etc/openwrt/openwrt/package/network/utils/curl'
time: package/network/utils/curl/compile#0.34#0.20#0.74
package/Makefile:111: recipe for target 'package/network/utils/curl/compile' failed
make[2]: *** [package/network/utils/curl/compile] Error 2
make[2]: Leaving directory '/etc/openwrt/openwrt'
package/Makefile:107: recipe for target '/etc/openwrt/openwrt/staging_dir/target-mipsel_24kc_musl/stamp/.package_compile' failed
make[1]: *** [/etc/openwrt/openwrt/staging_dir/target-mipsel_24kc_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/etc/openwrt/openwrt'
/etc/openwrt/openwrt/include/toplevel.mk:233: recipe for target 'world' failed
make: *** [world] Error 2
1 Like
2

Looks like you need to remove zstd from your build, or patch curl to disable zstd support in its config script. There was a curl version bump, and apparently curl now detects zstd (if it is in the build)

It you do the patch for curl, you might author a PR to fix curl in main OpenWrt.

3 Likes
3

Cc @dedeckeh @Pepe

4

I'm seeing this problem as well. I think I worked around it by removing the check code from configure and configure.ac in the downloaded package

5

I patched this by editing package/network/utils/curl/Makefile and duplicating everything for zlib but changing the duplicated lines to zstd (and the DEPENDS entry as well)

Then edit package/network/utils/curl/Config.in find the ZLIB entry and clone those four lines and again change zlib->zstd and ZLIB->ZSTD everywhere in the cloned lines

Then make oldconfig and you can answer the new question (I said Y since I have zstd enabled from tar and something else pulling it in anyway), and then it builds fine.

I have a patchfile if anyone wants it but the description of hacks should be just about as fast.

EDIT: here's the patch if you paste it into cat - | base64 -d > fixcurl.patch
review and apply from there

ZGlmZiAtLWdpdCBhL3BhY2thZ2UvbmV0d29yay91dGlscy9jdXJsL0NvbmZpZy5pbiBiL3BhY2th
Z2UvbmV0d29yay91dGlscy9jdXJsL0NvbmZpZy5pbgppbmRleCA5YWZlYjAwYmM3Li5hOGQ4MmEz
MjJmIDEwMDY0NAotLS0gYS9wYWNrYWdlL25ldHdvcmsvdXRpbHMvY3VybC9Db25maWcuaW4KKysr
IGIvcGFja2FnZS9uZXR3b3JrL3V0aWxzL2N1cmwvQ29uZmlnLmluCkBAIC0xMzMsNiArMTMzLDEw
IEBAIGNvbmZpZyBMSUJDVVJMX1pMSUIKIAlib29sICJFbmFibGUgemxpYiBzdXBwb3J0IgogCWRl
ZmF1bHQgbgogCitjb25maWcgTElCQ1VSTF9aU1RECisJYm9vbCAiRW5hYmxlIHpzdGQgc3VwcG9y
dCIKKwlkZWZhdWx0IG4KKwogY29uZmlnIExJQkNVUkxfVU5JWF9TT0NLRVRTCiAJYm9vbCAiRW5h
YmxlIHVuaXggZG9tYWluIHNvY2tldCBzdXBwb3J0IgogCWRlZmF1bHQgbgpkaWZmIC0tZ2l0IGEv
cGFja2FnZS9uZXR3b3JrL3V0aWxzL2N1cmwvTWFrZWZpbGUgYi9wYWNrYWdlL25ldHdvcmsvdXRp
bHMvY3VybC9NYWtlZmlsZQppbmRleCBmNTM2MjFhODU4Li40M2Q4NjYxZTYxIDEwMDY0NAotLS0g
YS9wYWNrYWdlL25ldHdvcmsvdXRpbHMvY3VybC9NYWtlZmlsZQorKysgYi9wYWNrYWdlL25ldHdv
cmsvdXRpbHMvY3VybC9NYWtlZmlsZQpAQCAtMzcsNiArMzcsNyBAQCBQS0dfQ09ORklHX0RFUEVO
RFM6PSBcCiAgIENPTkZJR19MSUJDVVJMX0xJQklETjIgXAogICBDT05GSUdfTElCQ1VSTF9TU0gy
IFwKICAgQ09ORklHX0xJQkNVUkxfWkxJQiBcCisgIENPTkZJR19MSUJDVVJMX1pTVEQgXAogICBc
CiAgIENPTkZJR19MSUJDVVJMX0RJQ1QgXAogICBDT05GSUdfTElCQ1VSTF9GSUxFIFwKQEAgLTkx
LDggKzkyLDggQEAgZGVmaW5lIFBhY2thZ2UvbGliY3VybAogICBTRUNUSU9OOj1saWJzCiAgIENB
VEVHT1JZOj1MaWJyYXJpZXMKICAgREVQRU5EUzo9ICtMSUJDVVJMX1dPTEZTU0w6bGlid29sZnNz
bCArTElCQ1VSTF9PUEVOU1NMOmxpYm9wZW5zc2wgK0xJQkNVUkxfR05VVExTOmxpYmdudXRscyAr
TElCQ1VSTF9NQkVEVExTOmxpYm1iZWR0bHMKLSAgREVQRU5EUyArPSArTElCQ1VSTF9aTElCOnps
aWIgK0xJQkNVUkxfVEhSRUFERURfUkVTT0xWRVI6bGlicHRocmVhZCArTElCQ1VSTF9MREFQOmxp
Ym9wZW5sZGFwICtMSUJDVVJMX0xJQklETjI6bGliaWRuMgotICBERVBFTkRTICs9ICtMSUJDVVJM
X1NTSDI6bGlic3NoMiArTElCQ1VSTF9OR0hUVFAyOmxpYm5naHR0cDIgK2NhLWJ1bmRsZQorICBE
RVBFTkRTICs9ICtMSUJDVVJMX1pMSUI6emxpYiArTElCQ1VSTF9aU1REOnpzdGQgK0xJQkNVUkxf
VEhSRUFERURfUkVTT0xWRVI6bGlicHRocmVhZCArTElCQ1VSTF9MREFQOmxpYm9wZW5sZGFwCisg
IERFUEVORFMgKz0gK0xJQkNVUkxfTElCSUROMjpsaWJpZG4yICtMSUJDVVJMX1NTSDI6bGlic3No
MiArTElCQ1VSTF9OR0hUVFAyOmxpYm5naHR0cDIgK2NhLWJ1bmRsZQogICBUSVRMRTo9QSBjbGll
bnQtc2lkZSBVUkwgdHJhbnNmZXIgbGlicmFyeQogICBNRU5VOj0xCiAgIEFCSV9WRVJTSU9OOj00
CkBAIC0xMzAsNiArMTMxLDcgQEAgQ09ORklHVVJFX0FSR1MgKz0gXAogCSQoaWYgJChDT05GSUdf
TElCQ1VSTF9MSUJJRE4yKSwtLXdpdGgtbGliaWRuMj0iJChTVEFHSU5HX0RJUikvdXNyIiwtLXdp
dGhvdXQtbGliaWRuMikgXAogCSQoaWYgJChDT05GSUdfTElCQ1VSTF9TU0gyKSwtLXdpdGgtbGli
c3NoMj0iJChTVEFHSU5HX0RJUikvdXNyIiwtLXdpdGhvdXQtbGlic3NoMikgXAogCSQoaWYgJChD
T05GSUdfTElCQ1VSTF9aTElCKSwtLXdpdGgtemxpYj0iJChTVEFHSU5HX0RJUikvdXNyIiwtLXdp
dGhvdXQtemxpYikgXAorCSQoaWYgJChDT05GSUdfTElCQ1VSTF9aU1REKSwtLXdpdGgtenN0ZD0i
JChTVEFHSU5HX0RJUikvdXNyIiwtLXdpdGhvdXQtenN0ZCkgXAogCSQoaWYgJChDT05GSUdfTElC
Q1VSTF9OR0hUVFAyKSwtLXdpdGgtbmdodHRwMj0iJChTVEFHSU5HX0RJUikvdXNyIiwtLXdpdGhv
dXQtbmdodHRwMikgXAogCVwKIAkkKGNhbGwgYXV0b2NvbmZfYm9vbCxDT05GSUdfTElCQ1VSTF9E
SUNULGRpY3QpIFwK
6

Would care to submit the patch to OpenWrt, either via mailing list or as a PR in GitHub?

8

Below is the proposed patch as cleartext.
(no idea why it was originally posted in encoded format)

At the first glance, I suspect that it should be "+LIBCURL_ZSTD:libzstd" instead of "+LIBCURL_ZSTD:zstd" on the dependency line:
zstd is the upper package that uses the underlying libzstd library, which also gets referenced by other packages.
Otherwise it looks ok to me.

--- a/package/network/utils/curl/Config.in
+++ b/package/network/utils/curl/Config.in
@@ -133,6 +133,10 @@ config LIBCURL_ZLIB
 	bool "Enable zlib support"
 	default n
 
+config LIBCURL_ZSTD
+	bool "Enable zstd support"
+	default n
+
 config LIBCURL_UNIX_SOCKETS
 	bool "Enable unix domain socket support"
 	default n
diff --git a/package/network/utils/curl/Makefile b/package/network/utils/curl/Makefile
index f53621a858..43d8661e61 100644
--- a/package/network/utils/curl/Makefile
+++ b/package/network/utils/curl/Makefile
@@ -37,6 +37,7 @@ PKG_CONFIG_DEPENDS:= \
   CONFIG_LIBCURL_LIBIDN2 \
   CONFIG_LIBCURL_SSH2 \
   CONFIG_LIBCURL_ZLIB \
+  CONFIG_LIBCURL_ZSTD \
   \
   CONFIG_LIBCURL_DICT \
   CONFIG_LIBCURL_FILE \
@@ -91,8 +92,8 @@ define Package/libcurl
   SECTION:=libs
   CATEGORY:=Libraries
   DEPENDS:= +LIBCURL_WOLFSSL:libwolfssl +LIBCURL_OPENSSL:libopenssl +LIBCURL_GNUTLS:libgnutls +LIBCURL_MBEDTLS:libmbedtls
-  DEPENDS += +LIBCURL_ZLIB:zlib +LIBCURL_THREADED_RESOLVER:libpthread +LIBCURL_LDAP:libopenldap +LIBCURL_LIBIDN2:libidn2
-  DEPENDS += +LIBCURL_SSH2:libssh2 +LIBCURL_NGHTTP2:libnghttp2 +ca-bundle
+  DEPENDS += +LIBCURL_ZLIB:zlib +LIBCURL_ZSTD:zstd +LIBCURL_THREADED_RESOLVER:libpthread +LIBCURL_LDAP:libopenldap
+  DEPENDS += +LIBCURL_LIBIDN2:libidn2 +LIBCURL_SSH2:libssh2 +LIBCURL_NGHTTP2:libnghttp2 +ca-bundle
   TITLE:=A client-side URL transfer library
   MENU:=1
   ABI_VERSION:=4
@@ -130,6 +131,7 @@ CONFIGURE_ARGS += \
 	$(if $(CONFIG_LIBCURL_LIBIDN2),--with-libidn2="$(STAGING_DIR)/usr",--without-libidn2) \
 	$(if $(CONFIG_LIBCURL_SSH2),--with-libssh2="$(STAGING_DIR)/usr",--without-libssh2) \
 	$(if $(CONFIG_LIBCURL_ZLIB),--with-zlib="$(STAGING_DIR)/usr",--without-zlib) \
+	$(if $(CONFIG_LIBCURL_ZSTD),--with-zstd="$(STAGING_DIR)/usr",--without-zstd) \
 	$(if $(CONFIG_LIBCURL_NGHTTP2),--with-nghttp2="$(STAGING_DIR)/usr",--without-nghttp2) \
 	\
 	$(call autoconf_bool,CONFIG_LIBCURL_DICT,dict) \

References to libzstd (from menuconfig search):

Symbol: PACKAGE_libzstd [=n]                                                                                                            │  
  │ Type  : tristate                                                                                                                        │  
  │ Defined at tmp/.config-package.in:34905                                                                                                 │  
  │   Prompt: libzstd.................................................... zstd library.                                                     │  
  │   Location:                                                                                                                             │  
  │     -> Libraries                                                                                                                        │  
  │ (1)   -> Compression                                                                                                                    │  
  │ Selects: PACKAGE_libc [=y] && PACKAGE_librt [=m] && PACKAGE_libpthread [=y]                                                             │  
  │ Selected by [n]:                                                                                                                        │  
  │   - PACKAGE_zstd [=n]                                                                                                                   │  
  │   - PACKAGE_btrfs-progs [=n] && BTRFS_PROGS_ZSTD [=n]                                                                                   │  
  │   - PACKAGE_squashfs-tools-mksquashfs [=n] && SQUASHFS_TOOLS_ZSTD_SUPPORT [=n]                                                          │  
  │   - PACKAGE_squashfs-tools-unsquashfs [=n] && SQUASHFS_TOOLS_ZSTD_SUPPORT [=n]                                                          │  
  │   - SQUASHFS_TOOLS_ZSTD_SUPPORT [=n] && (PACKAGE_squashfs-tools-mksquashfs [=n] || PACKAGE_squashfs-tools-unsquashfs [=n])              │  
  │   - PACKAGE_TAR_ZSTD [=n] && PACKAGE_tar [=n]                                                                                           │
9

@dedeckeh has committed a simple disabling fix as

1 Like
10

You are correct.

It's unfortunate, but I didn't experience it, I should rather have a full build prepared. Anyway, I think I like proposed patch much better.

11

Yeah, it only surfaces if you include packages pulling zstd in the build, not otherwise.

As buildbot builds "all packages", we regularly notice these "over-eager feature-inclusion" configure scripts only after the package gets built in the buildbot (as package updates are quite rarely tested with all other packages built). Nothing really special in this case.

But in any case, great that dedeckeh already patched the detection away.

1 Like
12

Base64'd because pasting patches mangles tabs/space/etc which really angers Makefiles?

13

I have taken the time to fork and PR including changes from comments:

I was on PatchTeam back in 2009-2011 and left for a while, but now I'm baaaack.

1 Like
14

Looks like a clean solution to me. As I recognize, that I did not fully implement your org. patch suggestion. For a newbie like me, org. patch description was not clear enough.

BTW: In case, you want to do another "good fix" for an issue I have reported without success few months ago already: wget/uclient-fetch using plain password protection (user/pwd) always tries to do the GET first without credentials, fails, and then tries including credentials. Not only waste of resources.

15

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.