The typing issue was just a miss-connected TX pin.
The boot process can be stopped with Escape. Have the serial connection running before plugging in power. It's like instant.
How did you compile that?
Edit:
Ah I see, lzma compression is not working.
Can someone explain me how to get a bootable image from the original firmware? I need more information than just this:
The Flash is divided in two, an Active and Backup image. The STK files are standard u-boot packed images and can be decompressed, splitted and sent to u-boot via Xmodem, no flashing necessary. With a little bash-fu, and some tools like binwalk you can split them from the flash header and use u-boot's Transfer file via serial, and then boot to the memory address where the file was uploaded. You can also dump the flash via serial port if you're a bit involved.
https://community.ui.com/questions/ES-8-150W-uboot-recovery-options/aa3e58b5-18cc-4bf8-bd42-7ec1f16ce2cf#answer/7e21961b-8888-44d3-b9da-c8d441270e3c
binwalk -e ESWH.v1.9.3.5434938.stk
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
116 0x74 uImage header, header size: 64 bytes, header CRC: 0xCF06E05F, created: 2021-09-01 07:32:36, image size: 14996095 bytes, Data Address: 0x8000, Entry Point: 0x8000, data CRC: 0xFBB4B667, OS: Linux, CPU: ARM, image type: Multi-File Image, compression type: lzma, image name: "System for iproc_wolf"
200 0xC8 LZMA compressed data, properties: 0x5D, dictionary size: 67108864 bytes, uncompressed size: -1 bytes
1181380 0x1206C4 LZMA compressed data, properties: 0x5D, dictionary size: 67108864 bytes, uncompressed size: -1 bytes
2135552 0x209600 Executable script, shebang: "/bin/sh"
2136028 0x2097DC gzip compressed data, from Unix, last modified: 2021-09-01 07:32:35
WARNING: Extractor.execute failed to run external extractor 'unstuff '%e'': [Errno 2] No such file or directory: 'unstuff', 'unstuff '%e'' might not be installed correctly
8410598 0x8055E6 StuffIt Deluxe Segment (data): fP
Thing with .stk files is that there is additional header before the actual uImage. So far I haven't been able to find any sensible information on them, aside from sources of other switches using the Broadcom FastPath platform, which seem to be very, very similar. This applies to HP 1820-24G series (J9979A), or Netgear S3300-28X - for the latter, GPL code is available here: https://www.downloads.netgear.com/files/GPL/Smart_M4300_S3300_GPL_V6.6.4.7.zip
In the topic - has someone here tried contacting HPE for GPL sources?