Compilation of upgraded mbedtls 3.6.2 fails in bootcore folder

[AkaAndy]

Hi mighty forum,
By reasons out of my control I had to upgrade mbedtls in openwrt 19.07 to a more secure version. I created a patch to go to the latest version as in the main-branch. When building it builds fine in the target build-dir but then mbedtls is built a second time for the bootcore and there it fails with this error message:

[12%] Building C object library/CMakeFiles/mbedcrypto.dir/bignum_core.c.o
cd /workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library &&
/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/staging_dir/toolchain-mips_24kc_gcc-8.3.0_musl/bin/mips-openwrt-linux-musl-gcc 
-Dmbedcrypto_EXPORTS 
-I/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/include 
-I/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library 
-I/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/3rdparty/everest/include 
-I/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/3rdparty/p256-m 
-I/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/3rdparty/p256-m/p256-m  
-pipe -mno-branch-likely -mips32r2 -mtune=24kc -g -fno-caller-saves -fno-plt -fhonour-copts 
-Wno-error=unused-but-set-variable -Wno-error=unused-result -
msoft-float -mips16 -minterlink-mips16 
-fmacro-prefix-map=/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2=mbedtls-3.6.2 
-Wformat -Werror=format-security 
-fpic -fstack-protector-strong -D_FORTIFY_SOURCE=2 
-Wl,-z,now -Wl,-z,relro -Wall -Wextra -Wwrite-strings -Wmissing-prototypes -Wformat=2 -Wno-format-nonliteral -Wvla -Wlogical-op -Wshadow -Wformat-signedness -Wformat-overflow=2 
-Wformat-truncation -Werror -Wmissing-declarations -O2 -fPIC   
-std=c99 -o CMakeFiles/mbedcrypto.dir/bignum_core.c.o   -c /workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library/bignum_core.c
{standard input}: Assembler messages:
{standard input}:2982: Error: invalid operands `lw $10,24($sp)'
{standard input}:2983: Error: invalid operands `lw $11,28($sp)'
{standard input}:2984: Error: invalid operands `lw $12,32($sp)'
{standard input}:2985: Error: invalid operands `lw $13,64($sp)'
{standard input}:2986: Error: invalid operands `lw $14,0($10)'
{standard input}i:2987: Error: invalid operands `multu $13,$14'
{standard input}:2988: Error: unrecognized opcode `addi $10,$10,4'
{standard input}:2989: Error: invalid operands `mflo $14'
...

The compilation flags for the one that works in the target-build-dir looks like this:

cd /workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_incoax_debug/mbedtls-3.6.2/library && 
/workspace/3UGW/ugw_sw/openwrt/staging_dir/toolchain-mips_24kc+nomips16_gcc-8.3.0_musl/bin/mips-openwrt-linux-musl-gcc
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2/include 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2/library 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2/3rdparty/everest/include 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2/3rdparty/p256-m -
I/workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2/3rdparty/p256-m/p256-m  
-pipe -mno-branch-likely -mips32r2 -mtune=24kc -g -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable 
-Wno-error=unused-result -msoft-float 

-fmacro-prefix-map=/workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2=mbedtls-3.6.2 
-Wformat -Werror=format-security -fpic -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wl,-z,now -Wl,-z,relro -Wall -Wextra 
-Wwrite-strings -Wmissing-prototypes -Wformat=2 -Wno-format-nonliteral -Wvla -Wlogical-op -Wshadow -Wformat-signedness 
-Wformat-overflow=2 -Wformat-truncation -Werror -Wmissing-declarations -O2 -fPIC   -std=c99 
-o CMakeFiles/mbedcrypto_static.dir/bignum_core.c.o   
-c /workspace/3UGW/ugw_sw/openwrt/build_dir/target-mips_24kc+nomips16_musl_prx321_in_debug/mbedtls-3.6.2/library/bignum_core.c

The one that fails looks like this:

cd /workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library &&
/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/staging_dir/toolchain-mips_24kc_gcc-8.3.0_musl/bin/mips-openwrt-linux-musl-gcc
-Dmbedcrypto_EXPORTS 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/include 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/3rdparty/everest/include 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/3rdparty/p256-m 
-I/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/3rdparty/p256-m/p256-m  
-pipe -mno-branch-likely -mips32r2 -mtune=24kc -g -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable 
-Wno-error=unused-result -msoft-float 
-mips16 -minterlink-mips16 
-fmacro-prefix-map=/workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2=mbedtls-3.6.2 
-Wformat -Werror=format-security -fpic -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wl,-z,now -Wl,-z,relro -Wall -Wextra 
-Wwrite-strings -Wmissing-prototypes -Wformat=2 -Wno-format-nonliteral -Wvla -Wlogical-op -Wshadow -Wformat-signedness 
-Wformat-overflow=2 -Wformat-truncation -Werror -Wmissing-declarations -O2 -fPIC   -std=c99 
-o CMakeFiles/mbedcrypto.dir/bignum_core.c.o   
-c /workspace/3UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library/bignum_core.c

Google led me to stack overflow

Assembler Messages: no such instruction when Compiling C++

Saying that the assembler program is not the same version as the gcc. Both the config file bootcore and the target shows this:

CONFIG_USE_MUSL=y
CONFIG_SSP_SUPPORT=y
CONFIG_BINUTILS_VERSION_2_31_1=y
CONFIG_BINUTILS_VERSION="2.31.1"
CONFIG_GCC_VERSION_8=y
CONFIG_GCC_USE_EMBEDDED_PATH_REMAP=y
CONFIG_GCC_VERSION="8.3.0"
CONFIG_LIBC="musl"
CONFIG_TARGET_SUFFIX="musl"

I don't think it is related but for completeness I've also updated ustream-ssl and I got an error discussed in this thread

ustream-ssl wants to build but fails if it's not selected

So I applied the provided update to the makefile

diff --git a/package/libs/ustream-ssl/Makefile b/package/libs/ustream-ssl/Makefile
index 3181f66c7e..9416fbe39f 100644
--- a/package/libs/ustream-ssl/Makefile
+++ b/package/libs/ustream-ssl/Makefile
@@ -44,7 +44,7 @@ endef
 define Package/libustream-mbedtls
   $(Package/libustream/default)
   TITLE += (mbedtls)
-  DEPENDS += +PACKAGE_libustream-mbedtls:libmbedtls
+  DEPENDS += libmbedtls
   CONFLICTS := libustream-openssl libustream-wolfssl
   VARIANT:=mbedtls
   DEFAULT_VARIANT:=1

Is there anyone out there who have any hint which can solve these error

/workspace/2UGW/ugw_sw/openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/library/bignum_core.c
{standard input}: Assembler messages:
{standard input}:2982: Error: invalid operands `lw $10,24($sp)'
{standard input}:2983: Error: invalid operands `lw $11,28($sp)'

I would appreciate it very much since I'm kind of running out of ideas.
Best regards

[brada4]

Try 23.05.5 which includes recent and uppdated mbedtls.

[AkaAndy]

Thanks for taking time to respond. I'm stuck with 19.07 and it is not a decision of mine to change that. So 23.05.5 is not an option.

[brada4]

You can disable assembly in ssl build config
You can upgrade binutils to handle modern assembly
gcc and mbedtls docoment version matches, nothing openwrt should fix, it is you backporting.

[AkaAndy]

When removing
-mips16 -minterlink-mips16
from
openwrt/build_dir/xRX500_BootCore/build_dir/target-mips_24kc_musl/mbedtls-3.6.2/CMakeCache.txt
The image actually builds. I'll test it on hardware and then figure out why these flags were added

[brada4]

Nothing at all to do with OpenWRT.

[AkaAndy]

Yes, you are right, thanks for your relevant comments.

[brada4]

mips -mISA flags not only dictate assembly generated by compiler but also are passed to assembler which filters only that architectures code. For example on x86 you can pass db "0xABCDEF01" in included assembly and assembler will not budge.

[slh]

It appears you are using firmware that is not from the official OpenWrt project.

When using forks/offshoots/vendor-specific builds that are "based on OpenWrt", there may be many differences compared to the official versions (hosted by OpenWrt.org). Some of these customizations may fundamentally change the way that OpenWrt works. You might need help from people with specific/specialized knowledge about the firmware you are using, so it is possible that advice you get here may not be useful.

You may find that the best options are:

1. Install an official version of OpenWrt, if your device is supported (see https://firmware-selector.openwrt.org).
2. Ask for help from the maintainer(s) or user community of the specific firmware that you are using.
3. Provide the source code for the firmware so that users on this forum can understand how your firmware works (OpenWrt forum users are volunteers, so somebody might look at the code if they have time and are interested in your issue).

If you believe that this specific issue is common to generic/official OpenWrt and/or the maintainers of your build have indicated as such, please feel free to clarify.