HooToo TM-05, Add LZMA Loader

[mpratt14]

@arrmo

LOADER_FLASH_OFFS := 0x0

This is your last remaining issue. Its not possible for the offset to be 0x0, that is the location of the bootloader on the flash. From your binwalk output, the correct offset would be (offset of loader + 0xDC8). Anytime you see "offset" this is relative to the start of flash, not relative to the position of the loader. The loader doesn't know where it is, but the start of flash is always 0x0. I had the same issue until I realized this

Also its worth noting. We have to be careful including the loader in the sysupgrade.bin. If there is any change to the loader in the future, the size of the loader might change, which would then change the offset location of the actual kernel. This is why in my commit for the ENS202EXT there is a dedicated partition for the loader. I would recommend padding to the nearest 4k and then setting offset to 0x1000 (which is 4k in hex).

Here I was explaining how it worked on my commit, for the ENS202EXT, the numbers don't apply here, but the idea is the same.

@arrmo

@mpratt14

The bootloader always attempts to boot at 0x0a0000. This is where the kernel used to be. The old kernel partition took up 1536k (0x18) of space, which makes the next partition start at 0x220000 (0xa + 0x8 = 0x12). That partition used to be the rootfs partition. I needed 1 more block because I had to do the same thing for rootfs (a fake/substitute for rootfs). Therefore, the actual kernel starts at 0x230000 for me. Remember that offset is relative to start of flash, not relative to where the loader is.

OK, that part I'm still digesting ... LOL. Let me think / debug some more here, may come back with a few other questions.

[arrmo]

@mpratt14

This is your last remaining issue. Its not possible for the offset to be 0x0, that is the location of the bootloader on the flash. From your binwalk output, the correct offset would be (offset of loader + 0xDC8). Anytime you see "offset" this is relative to the start of flash, not relative to the position of the loader. The loader doesn't know where it is, but the start of flash is always 0x0. I had the same issue until I realized this

That makes sense! But, then a few questions ... ;-),

• this is where it starts it's search, looking for the magic header ... right? I have the step set to 1 (for now), so it will find it, just not as efficiently as it could, no?
• no issue changing the value, but why would that stop the loader from starting? I don't even get the "intro" message, saying that's going to go look for the kernel
• there is a pad in the recipe (append-loader-okli $(1) | pad-to $$$$(BLOCKSIZE)), but it's not really padding?

@mpratt14

Also its worth noting. We have to be careful including the loader in the sysupgrade.bin. If there is any change to the loader in the future, the size of the loader might change, which would then change the offset location of the actual kernel.

Agreed! But again, if it starts a bit too low (in terms of offset), and steps by 1 ... it will find it, right? Again, perhaps not 100% efficient, but it should work (I think)?

I may be missing your point, by all means disagree. To me the problem is why the loader isn't even starting.

Thanks!

[mpratt14]

@arrmo

Starting kernel ...

And that's where it hangs

Not sure, but if you modified loader.c make sure you reverse those changes

@arrmo

if it starts a bit too low (in terms of offset), and steps by 1 ... it will find it, right?

You either know where these things on the flash are or you don't. The value of LOADER_FLASH_OFFS is simply the answer to the question "Where is the kernel on the flash chip?"

Maybe post your partition table again, with some detailed notes?

EDIT:

If your flash partition is something like this

$ cat /proc/mtd
dev:	size   erasesize  name
mtd0: 00030000 00010000 "u-boot"
mtd1: 00010000 00010000 "u-boot-env"
mtd2: 00010000 00010000 "factory"
mtd3: 007b0000 00010000 "firmware"
mtd4: 00120f6e 00010000 "kernel"
mtd5: 0068f092 00010000 "rootfs"
mtd6: 00440000 00010000 "rootfs_data"
mtd7: 00000000 00000000 "art"

Then the loader starts at 0x50000 on the flash
Then the kernel starts at 0x50DC8 on the flash

It might be different for you at this point, look at your DTS

[arrmo]

@mpratt14

Not sure, but if you modified loader.c make sure you reverse those changes

I shortened the recipe line, like you suggested - that seemed to make a difference. It boots now! Pasting a couple things here, to make it easier to read (vs. all inline).
https://paste.ubuntu.com/p/2QrrZQG83M/

And it's almost all working! Boots now, finds the header by searching, boots ... and just a hiccup with the partitions now?

Thanks!

[mpratt14]

Nice, just a couple notes

"Where the loader is" would include the header, because thats what u-boot needs to find in order to boot it. That doesn't matter though, we care about the kernel location. I know the variable is called "loader" flash offset, but it is actually for telling the loader where the kernel is.

From your log:

OpenWrt kernel loader for MIPS based SoC
Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>
Looking for OpenWrt image...
> Flash, base address: 0xa0500000
> Flash, start offset: 0x00000000
> Flash, end offset: 0x0f000000
> Flash, step size: 0x00000001
> Flash, search size: 0x0f000000 (251658240 bytes)
found at 0xa0500dd8
> Kernel, offset: 0x00000040
> Kernel, size: 0x00146ae1
> Kernel, load address: 0x80000000
> LZMA data, offset: 0xa0500e18
> LZMA data, size: 0x00146ae1
LZMA stream properties valid!
Decompressing kernel... done!
Starting kernel at 80000000...

See how it found the kernel at 0xa0500dd8

This is what you would have to figure out ahead of time
and set the variable like this:

LOADER_FLASH_OFFS := 0x500dd8

However it seems to be doing the job for you...

@xabolcs I actually didn't know the loader will find the kernel for you. Is that what it does when set to 0x0?

[arrmo]

@mpratt14

@xabolcs I actually didn't know the loader will find the kernel for you. Is that what it does when set to 0x0?

FYI, I got that from this link - just to share what I can!

OpenWrt kernel loader for AR7XXX/AR9XXX

Before I forget! Searching for the correct LOADER_FLASH_OFFS, you should do the following temporarily modify the recipe Build/loader-okli-compile: Edit FLASH_MAX=0 to a huge value, e.g. FLASH_MAX=0x0f000000 temporarily edit #define CONFIG_FLASH_STEP 0x1000 in config.h to #define CONFIG_FLASH_STEP 0x1 (or 0x10) not to miss the OKLI signature optionally decrease your loader-okli $(1) 15288 parameter in KERNEL := to 4096 or 5120 to save some kB flash space. at first you…

Hmmm ... now why is the flash partioning failing. Wondering if it's related to that KERNEL line?!?!

[mpratt14]

@arrmo

I shortened the recipe line, like you suggested - that seemed to make a difference. It boots now!

Not sure how that made a difference LOL
Do you make clean between builds?

@arrmo

Hmmm ... now why is the flash partioning failing. Wondering if it's related to that KERNEL line?!?!

From your log:

[    0.834524] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[    0.849507] Please append a correct "root=" boot option; here are the available partitions:

This is related to your DTS. I had the same issue. Need to fix your "compatible" property, as it is not splitting the "firmware" mtd into kernel and rootfs.

Try this:
compatible = "openwrt,okli";
This goes in the firmware parition node

Right now in your DTS you have

			partition@50000 {
				compatible = "denx,uimage";
				label = "firmware";
				reg = <0x50000 0x7b0000>;
			};

so try this:

			partition@50000 {
				compatible = "openwrt,okli";
				label = "firmware";
				reg = <0x50000 0x7b0000>;
			};

Related commit if you are curious to read

kernel: mtdsplit_uimage: add support for okli image

[arrmo]

@mpratt14

Do you make clean between builds?

I admit - I do not . Just tried that, not fixing my issues ... so let me run a full make dirclean, and update / reset .config. Will see if that works. I admit, I struggle to reliably rebuild the kernel (often find it's "old"), so I don't clean very often. Will report back, as a full dirclean and rebuild takes a bit.

@mpratt14

This is related to your DTS. I had the same issue. Need to fix your "compatible" property, as it is not splitting the "firmware" mtd into kernel and rootfs.

Awesome - thanks! Trying this now (with the full rebuild ).

@mpratt14

Related commit if you are curious to read

I am, appreciate it!

[arrmo]

Dang it! OK, here is what I have in the dts file now (and a full dirclean and make),

                        partition@50000 {
                                compatible = "openwrt,okli";
                                label = "firmware";
                                reg = <0x50000 0x7b0000>;
                        };

Still the same, doesn't seem to be splitting that "firmware" partition. I did check, the link you sent does seem to be in the code I have locally (sanity check ).

[mpratt14]

Forgot this...

You should also have padding between kernel and rootfs

append-kernel | pad-to $$$$(BLOCKSIZE) | append-rootfs

so recipe would be

  IMAGE/sysupgrade.bin := append-loader-okli $(1) | pad-to $$$$(BLOCKSIZE) | \
	lzma | uImage lzma | append-kernel | pad-to $$$$(BLOCKSIZE) | append-rootfs | pad-rootfs | \
	check-size | append-metadata

also to make all the numbers neat, you can pad-to 4k before the kernel, then your loader variable would be
LOADER_FLASH_OFFS := 0x51000

Edit: sorry now I'm getting confused with the addresses

[arrmo]

@mpratt14

Edit: sorry now I'm getting confused with the addresses

No worries! I feel your pain ... LOL!

Finding that padding doesn't seem to be working, may be me though. That can some second though - if I can get it to boot first.

[mpratt14]

Probably it's required that you have to make a separate partition for the loader. I don't remember any example where this isn't done. However you might just have to skip the space with the loader

try this along with the 4k pad and LOADER_FLASH_OFFS := 0x51000

                        partition@50000 {
                                compatible = "openwrt,okli";
                                label = "firmware";
                                reg = <0x51000 0x7af000>;
                        };

Edit: forgot to subtract from length

[arrmo]

@mpratt14

Probably it's required that you have to make a separate partition for the loader. I don't remember any example where this isn't done. However you might just have to skip the space with the loader

OK, but I think you're saying ... for now, as none of the rest of that is written out, just skip that 0x1000, and offset the firmware partition - right?

But I'm not seeing the padding really working, at least based on binwalk. I do have the pad-to in there, but binwalk isn't showing nice boundaries?

Thanks!

[mpratt14]

All the numbers need to agree whatever they are, and the "firmware" partition cannot include the loader.

Other than that, 4k padding should make the math easier. If that doesnt work, skip a whole block with 64k padding

[arrmo]

@mpratt14

Other than that, 4k padding should make the math easier. If that doesnt work, skip a whole block with 64k padding

Figured it out ... padding needs to come after lzma, as that is what actually gets written to flash. Agreed? binwalk sort of shows me this ... LOL. Uncompressed size is 4k, not compressed.

DECIMAL       HEXADECIMAL     DESCRIPTION
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0xFE679B47, created: 2020-06-25 00:36:19, image size: 3415 bytes, Data Address: 0x80000000, Entry
                              Point: 0x80000000, data CRC: 0x6B26BC13, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "MIPS OpenWrt
                              Linux-4.14.180"
64            0x40            LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 8192 bytes
3543          0xDD7           LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 5291427 bytes
1679360       0x19A000        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 2715274 bytes, 1353 inodes, blocksize: 262144 bytes, created: 2020-06-25 00:36:19

[mpratt14]

We're trying to move the kernel, so padding goes immediately before append-kernel

[arrmo]

Yep, I think we're on the same page. So even after the uImage (header add), right?

[mpratt14]

Yeah

lzma compresses the data

uImage lzma adds the header, with the information that the following data is lzma compressed. Otherwise the bootloader wouldn't know what it is or how to use it

[arrmo]

@mpratt14

Yeah

. And Mr. binwalk agrees

Let's see, will add the offset to .dts now, rebuild and test. Thanks!

FYI,

hexdump -C bin/targets/ramips/mt7620/openwrt-ramips-mt7620-ravpower_wd03-squashfs-sysupgrade.bin | grep OKLI
00001000  4f 4b 4c 49 fc cb 20 cc  5e f3 f1 83 00 19 8b f0  |OKLI.. .^.......|

[arrmo]

And ... so close . Built, loaded -> and again the stuck at "Starting kernel ...". Seems that wasn't just the clean and line shortening. Something to it. Not sure why it's freezing on boot. Hmmm.

Let me add a print statement or two (temporary! ), to see if the loader is at least getting launched.

BTW, something else to fix later perhaps - as @xabolcs pointed out, to rebuild the loader you manually need to,
rm build_dir/target-*/linux-*/loader-*