Snapshot update issue with libustream-ssl.so

[efahl]

Updating my experimental generic x86 with auc this morning failed with an error.

Collected errors:
 * check_data_file_clashes: Package libustream-mbedtls20201210 wants to install file /home/aparcar/asu/worker1/cache/SNAPSHOT/x86/64/build_dir/target-x86_64_musl/root-x86/lib/libustream-ssl.so
        But that file is already provided by package  * libustream-wolfssl20201210
 * opkg_install_cmd: Cannot install package luci-ssl.

I think I saw something about mbedtls on the dev email list in the last week or two, but I can't find it now. Can this be resolved with a simple opkg remove libustream-wolfssl20201210 or is there more I need to do?

Some context:

$ ubus call system board
{
        "kernel": "5.15.90",
        "hostname": "rtr02",
        "system": "Intel(R) Celeron(R) N5105 @ 2.00GHz",
        "model": "Default string Default string",
        "board_name": "default-string-default-string",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "SNAPSHOT",
                "revision": "r21950-90dbdb4941",
                "target": "x86/64",
                "description": "OpenWrt SNAPSHOT r21950-90dbdb4941"
        }
}

[tmomas]

@efahl

I think I saw something about mbedtls on the dev email list in the last week or two, but I can't find it now.

Maybe this commit?
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=4bcc3fd3d222bfe545beb3906da9de860fcb83bd

[giuliomagnifico]

OpenWrt support for WAX206

Figured I'd stop by to check on the merge status, seems like it's still some way off sadly. When building from commit 097e9bf9d842dc20932e85e38f9034ab58f499d9 I eventually run into what appears to be a conflict between different TLS/SSL libraries. Has something about the build procedure changed? I've simply tried doing this again since it seemed to work for people. Collected errors: * check_data_file_clashes: Package libustream-wolfssl20201210 wants to install file /home/pngu/src/github.com/b…

[lleachii]

@efahl

/home/aparcar

@tmomas

Does a standard OpenWrt snapshot image from http://downloads.openwrt.org/ install?

I think this question is still relevant.

[hnyman]

@efahl

my experimental generic x86

your error is due to auc including all current packages in your current firmware image, while OpenWrt changed the default SSL from wolfssl to mbedtls. Auc includes the old/present wolfssl things and simultaneously the new defaults cause mbedtls variants of some packages to be included.

So, you should tell auc to exclude the wolfssl variant of libustream.
The same goes also for px5g-wolfssl etc. wolfssl related stuff.

[efahl]

Yup, that was my suspicion, thanks for the confirmation!

[efahl]

Just a follow up on the actual commands I used to effect the solution:

opkg update
opkg download libmbedtls12 libubox20220927 libustream-mbedtls20201210
opkg remove luci-ssl px5g-wolfssl libustream-wolfssl20201210 libwolfssl5.5.4.e624513f
opkg install --offline-root / ~/*.ipk
auc -y

... after reboot ...

$ grep RELEASE /etc/os-release
OPENWRT_RELEASE="OpenWrt SNAPSHOT r22000-6bc675c0be"

EDIT: Note that you must run (and verify) the opkg download before you do the remove, as remove kills your certs and you won't be able to do any downloads again until you've installed the new ones with the subsequent opkg install ....

[bittin]

that fixed it

[dominick-han]

Also running into this issue
Latest master

I generate my .config from this file:

CONFIG_TESTING_KERNEL=y

CONFIG_TARGET_mvebu=y
CONFIG_TARGET_mvebu_cortexa9=y
CONFIG_TARGET_mvebu_cortexa9_DEVICE_linksys_wrt3200acm=y

CONFIG_PACKAGE_acme-acmesh-dnsapi=y
CONFIG_PACKAGE_adguardhome=y
CONFIG_PACKAGE_avahi-dbus-daemon=y
CONFIG_PACKAGE_block-mount=y
CONFIG_PACKAGE_ddns-scripts-cloudflare=y
CONFIG_PACKAGE_htop=y
CONFIG_PACKAGE_kmod-fs-f2fs=y
CONFIG_PACKAGE_kmod-usb2=y
CONFIG_PACKAGE_luci-app-acme=y
CONFIG_PACKAGE_luci-app-advanced-reboot=y
CONFIG_PACKAGE_luci-app-ddns=y
CONFIG_PACKAGE_luci-app-nlbwmon=y
CONFIG_PACKAGE_luci-ssl=y
CONFIG_PACKAGE_nano=y
CONFIG_PACKAGE_tcpdump=y

And

cat .config | grep libustream
CONFIG_DEFAULT_libustream-wolfssl=y
CONFIG_PACKAGE_libustream-mbedtls=y
# CONFIG_PACKAGE_libustream-openssl is not set
CONFIG_PACKAGE_libustream-wolfssl=y

Error message

* check_data_file_clashes: Package libustream-wolfssl20201210 wants to install file ******/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/root-mvebu/lib/libustream-ssl.so
        But that file is already provided by package  * libustream-mbedtls20201210
 * opkg_install_cmd: Cannot install package libustream-wolfssl20201210.

[dominick-han]

EDIT: Ignore below

@dominick-han

CONFIG_TARGET_mvebu_cortexa9_DEVICE_linksys_wrt3200acm=y

Ok, so apprently this line adds in CONFIG_DEFAULT_libustream-wolfssl=y
I had to manually set CONFIG_PACKAGE_libustream-wolfssl=n to resolve this, but is this intended?

luci-ssl brings in mbedtls, but doesn't disable wolfssl

[anomeome]

@neheb a bit of fallout?

[hnyman]

There shouldn't be.
As far as I can see in sources, the global default was changed quite properly from wolfssl to mbedtls.

I would more suspect a somehow dirty build environment, not updated sources, or something similar.

@dominick-han might delete tmp/ and let the package data there get recreated.

[hnyman]

@dominick-han

@dominick-han

CONFIG_TARGET_mvebu_cortexa9_DEVICE_linksys_wrt3200acm=y

Ok, so apprently this line adds in CONFIG_DEFAULT_libustream-wolfssl=y

It doesn't.
Here is the proof with the up-to-date master r22063-f490295bf2

perus@ub2210:/Openwrt/wrt3200$ ./scripts/getver.sh 
r22063-f490295bf2

perus@ub2210:/Openwrt/wrt3200$ cat .config
# Use "make defconfig" to expand this to a full .config
CONFIG_TARGET_mvebu=y
CONFIG_TARGET_mvebu_cortexa9=y
CONFIG_TARGET_mvebu_cortexa9_DEVICE_linksys_wrt3200acm=y

perus@ub2210:/Openwrt/wrt3200$ make defconfig
#
# configuration written to .config
#

perus@ub2210:/Openwrt/wrt3200$ grep -E "mbedtls|wolfssl" .config
CONFIG_DEFAULT_libustream-mbedtls=y
CONFIG_DEFAULT_wpad-basic-mbedtls=y
# CONFIG_PACKAGE_micropython-mbedtls is not set
# CONFIG_PACKAGE_libzip-mbedtls is not set
# CONFIG_PACKAGE_libuhttpd-mbedtls is not set
# CONFIG_PACKAGE_libuhttpd-wolfssl is not set
# CONFIG_PACKAGE_libuwsc-mbedtls is not set
# CONFIG_PACKAGE_libuwsc-wolfssl is not set
CONFIG_PACKAGE_libmbedtls=y
# Option details in source code: include/mbedtls/mbedtls_config.h
# CONFIG_PACKAGE_libwolfssl is not set
CONFIG_PACKAGE_libustream-mbedtls=y
# CONFIG_PACKAGE_libustream-wolfssl is not set
# CONFIG_PACKAGE_libwebsockets-mbedtls is not set
# CONFIG_PACKAGE_umurmur-mbedtls is not set
# CONFIG_PACKAGE_openvpn-mbedtls is not set
# CONFIG_PACKAGE_openvpn-wolfssl is not set
# CONFIG_PACKAGE_eapol-test-mbedtls is not set
# CONFIG_PACKAGE_eapol-test-wolfssl is not set
# CONFIG_PACKAGE_hostapd-basic-mbedtls is not set
# CONFIG_PACKAGE_hostapd-basic-wolfssl is not set
# CONFIG_PACKAGE_hostapd-mbedtls is not set
# CONFIG_PACKAGE_hostapd-wolfssl is not set
# CONFIG_PACKAGE_wpa-supplicant-mbedtls is not set
# CONFIG_PACKAGE_wpa-supplicant-mesh-mbedtls is not set
# CONFIG_PACKAGE_wpa-supplicant-mesh-wolfssl is not set
# CONFIG_PACKAGE_wpa-supplicant-wolfssl is not set
CONFIG_PACKAGE_wpad-basic-mbedtls=y
# CONFIG_PACKAGE_wpad-basic-wolfssl is not set
# CONFIG_PACKAGE_wpad-mbedtls is not set
# CONFIG_PACKAGE_wpad-mesh-mbedtls is not set
# CONFIG_PACKAGE_wpad-mesh-wolfssl is not set
# CONFIG_PACKAGE_wpad-wolfssl is not set
# CONFIG_PACKAGE_shairport-sync-mbedtls is not set
# CONFIG_PACKAGE_px5g-mbedtls is not set
# CONFIG_PACKAGE_px5g-wolfssl is not set
# CONFIG_PACKAGE_rtty-mbedtls is not set
# CONFIG_PACKAGE_rtty-wolfssl is not set
# CONFIG_PACKAGE_cache-domains-mbedtls is not set
# CONFIG_PACKAGE_cache-domains-wolfssl is not set
# CONFIG_PACKAGE_mbedtls-util is not set

[dominick-han]

Yeah, deleting and cloning again worked, thanks!

Somehow make distclean didn't work I did git clean -dfx too

[NicholasKK]

Hi @hnyman ,

There is exactly the same problem with Dynalink /openwrt image. I know that you are also active in Dynalink thread. I get exactly the same error message when trying to install luci-ssl-openssl on a freshly upgraded snapshot image without any additional packages. Do you know how to fix this issue just by using the master snapshot build?
Is there anyway to cleanup the image, so it does not result in the same error when installing luci-ssl-openssl?
Thanks

[hnyman]

@NicholasKK

the same error message when trying to install luci-ssl-openssl on a freshly upgraded snapshot image without any additional packages. Do you know how to fix this issue just by using the master snapshot build?

Sure:
just opkg remove the libustream-mbedtls (or libustream-wolfssl) package before installing the luci-ssl-openssl.
(note that removing the old SSL capability prevents new downloads, so first install openssl, then download the libustream-openssl .ipk package, then opkg remove libustream-mbedtls, and then opkg install libustream-openssl.
Then you should be able to install luci-ssl-openssl.

Or use opkg force options to force the libustream-openssl installation from SSH console.
(or just force luci-ssl-openssl)

Opkg package manager

opkg install --force-overwrite luci-ssl-openssl

Ps.
I normally build with toolchain from sources, so it is about this:

CONFIG_PACKAGE_wpad-openssl=y                  
# CONFIG_PACKAGE_wpad-basic-mbedtls is not set 
# CONFIG_PACKAGE_libustream-mbedtls is not set 
CONFIG_PACKAGE_luci-ssl-openssl=y 

For Dynalink, I have used imagebuilder, and there you need similar actions of disabling/removing mbedtls stuff from the image and selecting openssl variants of all components:

make image \
 PROFILE="dynalink_dl-wrx36" \
 PACKAGES=" wget-ssl hostapd-utils wpad-openssl ca-certificates luci-ssl-openssl \
  -wpad-basic-mbedtls -libustream-mbedtls -libmbedtls" \
 FILES="../files"

[NicholasKK]

Thank you very much @hnyman