HooToo TM-05, Add LZMA Loader

For Developers
41

Forgot this...

You should also have padding between kernel and rootfs

append-kernel | pad-to $$$$(BLOCKSIZE) | append-rootfs

so recipe would be

  IMAGE/sysupgrade.bin := append-loader-okli $(1) | pad-to $$$$(BLOCKSIZE) | \
	lzma | uImage lzma | append-kernel | pad-to $$$$(BLOCKSIZE) | append-rootfs | pad-rootfs | \
	check-size | append-metadata

also to make all the numbers neat, you can pad-to 4k before the kernel, then your loader variable would be
LOADER_FLASH_OFFS := 0x51000

Edit: sorry now I'm getting confused with the addresses

43

No worries! I feel your pain ... LOL!

Finding that padding doesn't seem to be working, may be me though. That can some second though - if I can get it to boot first.

44

Probably it's required that you have to make a separate partition for the loader. I don't remember any example where this isn't done. However you might just have to skip the space with the loader

try this along with the 4k pad and LOADER_FLASH_OFFS := 0x51000

                        partition@50000 {
                                compatible = "openwrt,okli";
                                label = "firmware";
                                reg = <0x51000 0x7af000>;
                        };

Edit: forgot to subtract from length

45

OK, but I think you're saying ... for now, as none of the rest of that is written out, just skip that 0x1000, and offset the firmware partition - right?

But I'm not seeing the padding really working, at least based on binwalk. I do have the pad-to in there, but binwalk isn't showing nice boundaries?

Thanks!

46

All the numbers need to agree whatever they are, and the "firmware" partition cannot include the loader.

Other than that, 4k padding should make the math easier. If that doesnt work, skip a whole block with 64k padding

47

Figured it out ... padding needs to come after lzma, as that is what actually gets written to flash. Agreed? binwalk sort of shows me this ... LOL. Uncompressed size is 4k, not compressed.

DECIMAL       HEXADECIMAL     DESCRIPTION
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0xFE679B47, created: 2020-06-25 00:36:19, image size: 3415 bytes, Data Address: 0x80000000, Entry
                              Point: 0x80000000, data CRC: 0x6B26BC13, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "MIPS OpenWrt
                              Linux-4.14.180"
64            0x40            LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 8192 bytes
3543          0xDD7           LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 5291427 bytes
1679360       0x19A000        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 2715274 bytes, 1353 inodes, blocksize: 262144 bytes, created: 2020-06-25 00:36:19
48

We're trying to move the kernel, so padding goes immediately before append-kernel

49

Yep, I think we're on the same page. So even after the uImage (header add), right?

50

Yeah

lzma compresses the data

uImage lzma adds the header, with the information that the following data is lzma compressed. Otherwise the bootloader wouldn't know what it is or how to use it

51

:+1:. And Mr. binwalk agrees :laughing:

Let's see, will add the offset to .dts now, rebuild and test. Thanks!

FYI,

hexdump -C bin/targets/ramips/mt7620/openwrt-ramips-mt7620-ravpower_wd03-squashfs-sysupgrade.bin | grep OKLI
00001000  4f 4b 4c 49 fc cb 20 cc  5e f3 f1 83 00 19 8b f0  |OKLI.. .^.......|
52

And ... so close :wink:. Built, loaded -> and again the stuck at "Starting kernel ...". Seems that wasn't just the clean and line shortening. Something to it. Not sure why it's freezing on boot. Hmmm.

Let me add a print statement or two (temporary! :laughing:), to see if the loader is at least getting launched.

BTW, something else to fix later perhaps - as @xabolcs pointed out, to rebuild the loader you manually need to,
rm build_dir/target-*/linux-*/loader-*

53

make clean really should handle everything

I really don't know why that's happening to you, do you ever use git diff to take a look at your current changes? You didnt alter the loader source code right?

54

OK, confired ... loader_main is not even being called. Need to dig now, see why I'm not getting there.

55

I did modify AR71XX_FLASH_START and AR71XX_FLASH_END (other than that, just some prints to show me info for now). But we posted almost the same time - I'm not even gettign to loader_main.

56

OK, having to shut it down for tonight, thanks so much for all the pointers and ideas. Complete rebuild, same thing - just hangs starting the loader. Not quite sure what's causing it :frowning_face:. But I am optimistic that we are very close, just need to get over this hurdle.

One last thing I just noticed, tftp programming of the device - should the erase be rounded down (like it is), or up? This seems the wrong way to me ... corrupting part of the image?

Bytes transferred = 3822346 (3a530a hex)
NetBootFileXferSize= 003a530a
write(0x80500000, 0x50000, 3822346)
raspi_erase_write: offs:50000, count:3a530a
raspi_erase: offs:50000 len:3a0000
57

I would assume the bootloader does it properly

even though you see len:3a0000 it might just mean "the block starting at 0x5 + 0x3a" which would be 0x3f to 0x3fffff (I think)

58

I think you're right - that makes sense, and the problems would be worse if not programmed fully :wink:

Just very odd, that it doesn't even call the loader. Really baffled there, and not sure what to try :frowning_face:

Thanks!

59

Digging. Seeing a bunch of this error ... do you see this? I admit, not sure what is causing it - I just copied the loader-okli recipes over from ath79.

Makefile:214: warning: overriding recipe for target '/mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-ravpower_wd03.bin'
Makefile:214: warning: ignoring old recipe for target '/mnt/ProgSSD/openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_mt7620/loader-ravpower_wd03.bin'
60

I saw it when I cooked OKLI for DIR-860L.

I don't know what does it mean, but may be related to the need of human power to remove those bins while rebuilding.

61

I've never had a problem compiling the loader. I always make clean between builds. But then again, I only have done this for 1 board.

I would run git diff (space to continue, q to exit)
Look through everything very carefully, and revert all your changes to loader.c, no need to test it anymore.

another random thing to try
touch all makefiles before you make clean
for file in `find . | grep Makefile`; do touch $file; done;