RAVPower WD-03, Add LZMA Loader

That makes sense - and better said :slight_smile:.

Yep, changed that, and to 24 MB like you said (align where we can!). This change and one more (below) ... and it's back up and running! Glad I got the hammer out, we wouldn't have figured this out blind (or at least I wouldn't have!). The fact that this was set to start at 32 MB clearly is not gonna work for a 32 MB device :laughing:. But ... do we push this change for all (as it is in the image level Makefile)? I don't really see a downside, do you?

And BTW, what exactly is LZMA_TEXT_START? Assuming it's where lzma (decompressing) stores some of it's (dictionary?) info? Either way, it does fix what we were all thinking!

It is! This was the loader issue. From the other thread, you were asking to try other values - I had accidentally left it at 16k. Changed it back, all good. But that said ... what the heck is this padding for? To me, I don't know why it's needed. The loader should malloc to get needed memory ... or is that not available at this point, and that padding is the memory it makes use of?

Not needed, but thinking about it, I am a bit perplexed, because ...

  1. U-Boot extracts and decompresses the loader, at 0x8000000 (right?). Then it calls 0x8000000, the loader starts.
  2. The loader finds the OKLI magic, and then proceeds to decompress the kernel ... but isn't it doing that to 0x8000000 (also), so right on top of itself?

I'm sure I'm missing something here ... LOL!

Thanks again for the help! Just a couple more checks, and I think this one is ready to go as well, agreed?

Sorry, @mpratt14, wanted to get the boot going first, but here are some of the menus - anything else you want to see? Thanks!

|-------------------------------|
|         IOVST MAIN MENU       |
|-------------------------------|
| 6 Test USB                    |
| 8 Test ethernet               |
| B Boot the Kernel             |
| E Test PIO                    |
| I Test system params          |
| U SPIFlash Upgrade            |
| T SMT test program            |
| X Update the license          |
| R Reboot                      |
| Z Enter Command Line Interface|
|-------------------------------|

(Z) Command Line Interface,

MT7620 # help
?       - alias for 'help'
bootm   - boot application image from memory
cp      - memory copy
erase   - erase SPI FLASH memory
go      - start application at address 'addr'
help    - print online help
loadb   - load binary file over serial line (kermit mode)
md      - memory display
mdio   - Ralink PHY register R/W command !!
mm      - memory modify (auto-incrementing)
nm      - memory modify (constant address)
ping    - send ICMP ECHO_REQUEST to network host
printenv- print environment variables
reset   - Perform RESET of the CPU
rf      - read/write rf register
saveenv - save environment variables to persistent storage
setenv  - set environment variables
tftpboot- boot image via network using TFTP protocol
version - print monitor version

Oh, and BTW, it seems the loader is ~ 13 years old ... LMAO!
https://www.7-zip.org/sdk.html

FYI, all working - did a clean build, and cleaned up overall as well. Here is the diff, yell if you see any issues. Thanks!
https://paste.ubuntu.com/p/Qbs7P9rSn4/

In the end, looks trivial ... :stuck_out_tongue_winking_eye:

PS: To confirm, made the same 24 MB offset change for the HooToo (64 MB RAM device) ... all good!

Nah...it's just like the HooToo...

My idea was to have you checksum compare flash to ram to see if kernel is there. But you already discovered it wasn't :stuck_out_tongue:

you mean the LZMA compression algorithm? That's much more than 13 years old.

I'm pretty sure...this is the place in RAM where the loader copies the kernel before decompression (still lzma compressed). It then decompresses kernel on top of itself (or at LOAD_ADDRESS if it is different).

Should be fine, but make it different commit in same PR, and reference the commit by Adrian for ath79.

Also like in the commit message. It is possible to edit the Makefile so that LZMA_TEXT_START is a configurable option. You can see in the commit that he removed that line
LZMA_TEXT_START=0x80a00000 LOADADDR=0x80060000 \
from the loader-common build command because it was redundant

Alternatively it could be another parameter passed between all the commands, and if unset, would be in the lzma-loader Makefile for default value.

Sorry, I meant the version that is in the loader - I worded that poorly :frowning_face:.

That makes sense - so we need to "stay away" from the upper end of RAM by at least the size of the compressed kernel + rootfs, right?

But if it copies on top of itself - won't it trash itself in the process (as it is still running)? Just trying to wrap my head around it, sorry for the dumb questions!

That makes sense - will do!

Yep! I tried this as well, but LOADADDR is variable, so I can't set it in the other Makefile, like he did. Hence the change I made ... only removing the redundant LZMA_TEXT_START.

I would keep this 32 MB as the default, and use this 24 MB for these 32 MB RAM OKLI devices, like the LOADADDR is templated.

That was my thought as well, but as this is defined in the loader Makefile, it's applied (automatically) across everything on this platform. Right?

It's defined in the Build/loader-common recipe.

Just examine that recipe and follow LOADADDR's pattern how it is defined differently through plarforms (subtargets).

I would create as a device level variable, define it where it needed, and would define it at "global level" too with 32 MB value.

That makes sense - let me give this a try, will send you the diff to look at. Thanks!

1 Like

Sorry, I don't know how to mix and match the device variables with global variables.
If you don't find a solution, you could go with the "set default 32 MB if that variable is empty" way. :upside_down_face:

You will find examples for the latter in that Makefile!

That's exactly my plan - we're aligned! :laughing:. Will get back to you this evening, thanks!

1 Like

I would add a parameter to the loader-okli command. Not sure how it being blank would work but you can test that on the other device, otherwise there can be an if statement.

Remember in make, everything after the command is one big parameter $(1) so you have to split it up on the other side with the word function $(word 1, $(1)) and $(word 2, $(1))

OK, seeing something a bit strange, but I saw this before also when looking at LOADER_FLASH_OFFS ... thought I was losing my mind, but now not so sure (though that may still be true ... :laughing:). Here is what I did,

  1. Near the beginning of target/linux/ramips/image/Makefile, set LZMA_TEXT_START := 0x82000000 (default value, 32 MB). In the same file, inside define Build/loader-common, set LZMA_TEXT_START=$(LZMA_TEXT_START) LOADADDR=$(KERNEL_LOADADDR) \ . This part seems to work fine. Modeling it after KERNEL_LOADADDR somewhat.
  2. In target/linux/ramips/image/lzma-loader/Makefile, "remove" LZMA_TEXT_START, i.e. change it from 0x80a00000 to blank, so the value from 1) above is used. Also seems to work.
  3. In target/linux/ramips/image/mt7620.mk, inside the Ravpower (only!), set LZMA_TEXT_START := 0x81800000 (custom value, 24 MB).

The last step is where things get weird. In the build output (for the HooToo), I see,

export MAKEFLAGS= ;make -w -C lzma-loader PKG_BUILD_DIR="/mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src" TARGET_DIR="/mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/" LOADER_NAME="loader-hootoo_tm05.bin" BOARD="MT7620 based boards" PLATFORM="ralink" LZMA_TEXT_START=0x81800000 LOADADDR=0x80000000 FLASH_OFFS=0xFD051000 compile loader.bin
make[6]: Entering directory '/mnt/ProgSSD/openwrt/target/linux/ramips/image/lzma-loader'
mkdir /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src
cp -fpR ./src/* /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src/
touch /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src/.prepared
make -C /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src CROSS_COMPILE="mipsel-openwrt-linux-musl-" \
	LZMA_TEXT_START=0x81800000 \
	LOADER_DATA= \
	FLASH_OFFS=0xFD051000 \
	FLASH_MAX= \
	BOARD="MT7620 based boards" \
	PLATFORM="ralink" \
	clean all

But, the RAVPower build recipe should not be used ... so the value should be the default, no? The reason I say I saw this before also ... when I changed `LOADER_FLASH_OFFS=0x0' ONLY for the RAVPower, the value was also applied to the HooToo! It's like those values ... the last one in the file is used, not the one in a particular recipe. If that's the case, the LOADER_FLASH_OFFS isn't really working either.

Thoughts?

Thanks!

OK, before hitting send, went back and changed the RAVPower (only!) to LOADER_FLASH_OFFS := 0x0, then build for the HooToo (still has LOADER_FLASH_OFFS := 0xFD051000). It doesn't work! Here is the output ... the last value defined in mt7620.mk is taken into use!

export MAKEFLAGS= ;make -w -C lzma-loader PKG_BUILD_DIR="/mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src" TARGET_DIR="/mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/" LOADER_NAME="loader-hootoo_tm05.bin" BOARD="MT7620 based boards" PLATFORM="ralink" LZMA_TEXT_START=0x81800000 LOADADDR=0x80000000 FLASH_OFFS=0x0 compile loader.bin
make[6]: Entering directory '/mnt/ProgSSD/openwrt/target/linux/ramips/image/lzma-loader'
mkdir /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src
cp -fpR ./src/* /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src/
touch /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src/.prepared
make -C /mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-hootoo_tm05.bin.src CROSS_COMPILE="mipsel-openwrt-linux-musl-" \
	LZMA_TEXT_START=0x81800000 \
	LOADER_DATA= \
	FLASH_OFFS=0x0 \
	FLASH_MAX= \
	BOARD="MT7620 based boards" \
	PLATFORM="ralink" \
	clean all

So I'm not completely nuts, but this is not good :frowning_face:

Found my error - phew! :smile:. To make them device variables, need to make sure to add them to DEVICE_VARS

Confirming now, to see if this all works.

1 Like

Seems to all be working - built both images (HooToo, RAVPower), variables and build log look to me to be correct. Here are the changes,
https://paste.ubuntu.com/p/jcqbgVDfN5/

By all means comment! If this looks OK to you, I'll create a new PR, for the RAVPower - and add you two as reviewers, OK?

Thanks!

PS: Still want to understand that loader padding some day :stuck_out_tongue_closed_eyes:

1 Like

So cool - just re-built and re-flashed both devices, from the "ground up", just to make sure. Both working great, kernel > 1.5 MB, loader and boot all happy. Can't thank you two enough for all the the help. Really appreciate it!

1 Like

Yeah if you don't do this, the variable will be for the entire make process, and the last time it is defined will be the final value for building. Since all devices are in alphabetical, Ravpower device definition was setting the variable last, therefore you saw that every time.

a related commit (I wish they explained more in commit messages...):

Otherwise I was going to have you look at this, as maybe you need to make a file like target/linux/ramips/image/lzma-loader/src/mt7620.mk but idk.

1 Like

Yep, that's what I figured out, after a bit of pain ... LOL. Seems to all work fine though, matching to other definitions => and the default is 32 MB, but can be adjusted per device.

Thanks!

Yeah, thats why I linked to line 11 where thos device variables start ... but you are right, learning the hard way is the best way! :stuck_out_tongue_winking_eye:

And yes, the HooToo PR should be adjusted with the introduced LOADER_FLASH_OFFS device level variable - added comment to the PR!

1 Like