HooToo TM-05, Add LZMA Loader

arrmo · June 24, 2020, 7:49pm

Continuation to this discussion, https://github.com/openwrt/openwrt/pull/2645#issuecomment-649031604

Will do some digging, come back here. Thanks!

xabolcs · June 24, 2020, 8:02pm

At first, please read the PB44 LZMA Loader thread, where I wrote a draft about finding the correct address for LOADER_FLASH_OFFS!

But please, don't steal that thread!

arrmo · June 25, 2020, 2:00am

I won't, no worries! . Will keep this discussion here, really appreciate the pointers and help!

arrmo · June 25, 2020, 2:41am

OK, reading through what you captured ... a few questions now, sorry. And I did read through it several times ... I'm sure it's exactly right, but my brain is getting about 60% of it ... LOL. That's definitely on me, so I do apologize for some questions that are dumb (oh ya, they are ... ). Here goes,

the code is different between the noted device and ramips. No biggie, just means some adapting. For example, the "searching" (FLASH_MAX, FLASH_STEP) don't seem to exist ... or am I just missing them?
but, to the point above, I think this is about finding the offset to use, right? That said, from https://github.com/arrmo/openwrt/blob/hootoo-tm05/target/linux/ramips/dts/mt7620n_hootoo_tm05.dts ... the offset in the flash starts from 0x50000, agreed?
I looked at the log you sent (which is way cool!), and I don't think it matches to what is in the repository currently, correct? I say that because for that device it seems to be a "stock" boot (not lzma loader), just a small version. Making sure I understand.
where I sort of get stuck is that it seems to me that the KERNEL and/or IMAGE commands inside the device Define seem backwards. Meaning ... they start with the kernel, but I would have thought the first item would be the loader (perhaps padded to a fixed size), then the kernel ... no?
I also struggle a bit with the Device logic ... meaning the definitions inside the Device (e.g. IMAGE or KERNEL), vs. what is in Device/Default. I'm not sure which one takes precedence.

Again, any pointers you have would be much appreciated! We'll get through it, but I admit - with more dumb questions from me first

xabolcs · June 25, 2020, 7:37am

Let me answer yout questions later, and just quote my mail, sent to ML.

Hi!

Chuanhong Guo ezt írta (időpont: 2020. ápr. 19., V, 17:42):

Hi!

My original thought on this device is to use a different loader. The first
4MB of SPI-NOR flash on mt7621 is mapped to 0x1fc00000 and lzma
loader can read compressed kernel directly from flash. If the kernel
can be put at a fixed offset in flash, we could compress lzma loader
separately and let u-boot decompress only the loader.
You could take a look at the tp-link-nolzma recipe in:
target/linux/ath79/image/common-tp-link.mk
and see if you could implement a similar solution for mt7621.
Note: You need to fix AR71XX_FLASH_START defined in:
target/linux/ramips/image/lzma-loader/src/loader.c
to 0x1fc00000 for this method to work.

I managed to make the loader-okli recipes work on this DIR-860L router.

It's ugly:
target/linux/ramips/image/Makefile | 20 ++++++++++++++++++++
target/linux/ramips/image/lzma-loader/src/loader.c | 2 +-
target/linux/ramips/image/mt7621.mk | 9 ++++++++-
3 files changed, 29 insertions(+), 2 deletions(-)

... compared to the my original change:
target/linux/ramips/image/mt7621.mk | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

And it's fragile!
LOADER_FLASH_OFFS consists of:

the address of the firmware partition (0x500000)

loader-okli recipe's "kernel offset" argument (and the compressed size of the loader)

the 64 byte length header from the Device/seama recipe

and the AR71XX_FLASH_START value in loader.c
And there is a "pad to 32k" step while building the loader separately
and I don't know why it's needed to be 32k - sure it doesn't work with 24k, 16k and below.

About the AR71XX_FLASH_START= 0x10000000 in loader.c:
This is now 100% DIR-860L specific value. As you can see in the serial log
that the loader finds the kernel at 0xb0501040 which consits of the following components:

0x00000040: SEAMA header and metadata (see Device/seama)

0x00001000: the 4096 "kernel offset" argument of loader-okli

0x00500000: start adress of "firmware" partition

0xa0000000: KSEG1 address

The rest (0x10000000) is AR71XX_FLASH_START and other unknown (at lest to me) parts.
So I chose AR71XX_FLASH_START= 0x10000000.
Sure there is a room to improvement. At least move AR71XX_FLASH_START to the board file, and rename!

The 4096 byte "kernel offset" is a freely chosen value: it's little bigger than the compressed lzma loader and it is padded to 4k.

Anyway! It works! And it works nicely!

Check the timestamped serial logs below!
The double compressed lzma-loader method needs

1.9s for u-boot ("Uncompressing SEAMA linux.lzma ... OK")

1.1s for the kernel loader ("Decompressing kernel... done!")

The OKLI method needs:

no time (0.05s) for u-boot

1.4s for the kernel loader

Serial log without this patch:

2020-05-03 20:09:55.367155157: 3: System Boot system code via Flash.
2020-05-03 20:09:55.373933177: ## Booting image at bfc50000 ...
2020-05-03 20:09:57.300117502: addr:80500000
2020-05-03 20:09:57.308441445: We have SEAMA, Image Size = 2490304
2020-05-03 20:09:57.315410096: Verifying Checksum ...
2020-05-03 20:09:59.205312048: Uncompressing SEAMA linux.lzma ... OK
2020-05-03 20:09:59.216341009: ## Transferring control to Linux (at address 00000000) ...
2020-05-03 20:09:59.222500408: ## Giving linux memsize in MB, 128
2020-05-03 20:09:59.224349797:
2020-05-03 20:09:59.226204307: Starting kernel ...
2020-05-03 20:09:59.227824058:
2020-05-03 20:09:59.289356732:
2020-05-03 20:09:59.290995300:
2020-05-03 20:09:59.297258876: OpenWrt kernel loader for MIPS based SoC
2020-05-03 20:09:59.306245212: Copyright (C) 2011 Gabor Juhos
2020-05-03 20:10:00.417134281: Decompressing kernel... done!
2020-05-03 20:10:00.425918694: Starting kernel at 80001000...
2020-05-03 20:10:00.432489649:
2020-05-03 20:10:00.453684838: [ 0.000000] Linux version 5.4.35 (builder@buildhost) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r13108-87c909e969)) #0 SMP Mon Apr 27 23:10:48 2020
2020-05-03 20:10:00.462322280: [ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
2020-05-03 20:10:00.471916788: [ 0.000000] printk: bootconsole [early0] enabled
2020-05-03 20:10:00.480939230: [ 0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
2020-05-03 20:10:00.491938368: [ 0.000000] MIPS: machine is D-Link DIR-860L B1
2020-05-03 20:10:00.503935082: [ 0.000000] Initrd not found or empty - disabling initrd

Serial log with OKLI:

2020-05-03 22:36:18.360052605: 3: System Boot system code via Flash.
2020-05-03 22:36:18.365448792: ## Booting image at bfc50000 ...
2020-05-03 22:36:20.299329593: addr:80500000
2020-05-03 22:36:20.320272229: We have SEAMA, Image Size = 2490304
2020-05-03 22:36:20.333914984: Verifying Checksum ...
2020-05-03 22:36:20.381790212: Uncompressing SEAMA linux.lzma ... OK
2020-05-03 22:36:20.397034839: ## Transferring control to Linux (at address 00000000) ...
2020-05-03 22:36:20.413734125: ## Giving linux memsize in MB, 128
2020-05-03 22:36:20.426230940:
2020-05-03 22:36:20.429622961: Starting kernel ...
2020-05-03 22:36:20.437932564:
2020-05-03 22:36:20.440335376:
2020-05-03 22:36:20.444103378:
2020-05-03 22:36:20.451227117: OpenWrt kernel loader for MIPS based SoC
2020-05-03 22:36:20.453709985: Copyright (C) 2011 Gabor Juhos
2020-05-03 22:36:20.491573227: Looking for OpenWrt image... found at 0xb0501040
2020-05-03 22:36:21.845561224: Decompressing kernel... done!
2020-05-03 22:36:21.854485577: Starting kernel at 80001000...
2020-05-03 22:36:21.866553455:
2020-05-03 22:36:21.895973592: [ 0.000000] Linux version 5.4.36 (xabolcs@ut1804) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r13060-471b8bf8c1)) #0 SMP Sat May 2 23:19:08 2020
2020-05-03 22:36:21.902983786: [ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
2020-05-03 22:36:21.907483498: [ 0.000000] printk: bootconsole [early0] enabled
2020-05-03 22:36:21.912569091: [ 0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
2020-05-03 22:36:21.920921080: [ 0.000000] MIPS: machine is D-Link DIR-860L B1
2020-05-03 22:36:21.931420797: [ 0.000000] Initrd not found or empty - disabling initrd

I'm glad, that I had success with this OKLI method, but does it really worth it?

Cheers,
Szabolcs

--
Regards,
Chuanhong Guo

And the WIP patch is:

 target/linux/ramips/image/Makefile            | 20 +++++++++++++++++++
 .../ramips/image/lzma-loader/src/loader.c     |  2 +-
 target/linux/ramips/image/mt7621.mk           |  9 ++++++++-
 3 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/target/linux/ramips/image/Makefile b/target/linux/ramips/image/Makefile
index f93ea8ab2a..76568d7c6c 100644
--- a/target/linux/ramips/image/Makefile
+++ b/target/linux/ramips/image/Makefile
@@ -88,6 +88,26 @@ define Build/loader-kernel
        $(call Build/loader-common,LOADER_DATA="$@")
 endef

+define Build/loader-okli-compile
+       $(call Build/loader-common,FLASH_OFFS=$(LOADER_FLASH_OFFS) FLASH_MAX=0)
+endef
+
+# Arguments: <output name> <kernel offset> <loader type>
+define Build/loader-okli-with-type
+       dd if=$(KDIR)/loader-$(word 1,$(1)).$(word 3,$(1)) bs=$(word 2,$(1)) conv=sync of="$@.new"
+       cat "$@" >> "$@.new"
+       mv "$@.new" "$@"
+endef
+
+# Arguments: <output name> <kernel offset>
+define Build/loader-okli
+       $(call Build/loader-okli-with-type,$(word 1,$(1)) $(word 2,$(1)) $(LOADER_TYPE))
+endef
+
+define Build/append-loader-okli
+       cat "$(KDIR)/loader-$(word 1,$(1)).$(LOADER_TYPE)" >> "$@"
+endef
+
 define Build/relocate-kernel
        rm -rf $@.relocate
        $(CP) ../../generic/image/relocate $@.relocate
diff --git a/target/linux/ramips/image/lzma-loader/src/loader.c b/target/linux/ramips/image/lzma-loader/src/loader.c
index c73b60b351..8b7756b931 100644
--- a/target/linux/ramips/image/lzma-loader/src/loader.c
+++ b/target/linux/ramips/image/lzma-loader/src/loader.c
@@ -28,7 +28,7 @@
 #include "printf.h"
 #include "LzmaDecode.h"

-#define AR71XX_FLASH_START     0x1f000000
+#define AR71XX_FLASH_START     0x10000000
 #define AR71XX_FLASH_END       0x1fe00000

 #define KSEG0                  0x80000000
diff --git a/target/linux/ramips/image/mt7621.mk b/target/linux/ramips/image/mt7621.mk
index 88db43cb65..9438e5c09e 100644
--- a/target/linux/ramips/image/mt7621.mk
+++ b/target/linux/ramips/image/mt7621.mk
@@ -221,7 +221,14 @@ define Device/dlink_dir-860l-b1
   BLOCKSIZE := 64k
   SEAMA_SIGNATURE := wrgac13_dlink.2013gui_dir860lb
   LOADER_TYPE := bin
-  KERNEL := kernel-bin | append-dtb | lzma | loader-kernel | \
+  LOADER_FLASH_OFFS := 0x501040
+  COMPILE := loader-$(1).bin loader-$(1).lzma
+  COMPILE/loader-$(1).bin := loader-okli-compile
+  COMPILE/loader-$(1).lzma := append-loader-okli $(1) | pad-to 32k | \
+       relocate-kernel | lzma
+  KERNEL := kernel-bin | append-dtb | lzma | uImage lzma -M 0x4f4b4c49 | \
+       loader-okli-with-type $(1) 4096 lzma
+  KERNEL_INITRAMFS := kernel-bin | append-dtb | lzma | loader-kernel | \
        relocate-kernel | lzma -a0 | uImage lzma
   IMAGE_SIZE := 16064k
   DEVICE_VENDOR := D-Link

arrmo · June 26, 2020, 2:42am

Thanks for the info! This does help, gradually working my way though - and I realized one of my questions above was dumb (see, I warned you ... LOL!). I understand better now the sequencing of processing, meaning the row below, that I think applies to my device (or at least, a bit of a guess ),

Taking this from the default device (Makefile), and then what we want to do with it ... add the loader, relocate the kernel, compress and build a final / new uImage. But what is confusing me is where / how to tell the lzma loader where in flash to get the real image. I think it seemed to be related to LZMA_TEXT_START ... or perhaps not. But even that value - it's different in,

target/linux/ramips/image/lzma-loader/Makefile => LZMA_TEXT_START := 0x80a00000 (but FLASH_OFFS and FLASH_MAX not set?)
target/linux/ramips/image/Makefile => LZMA_TEXT_START=0x82000000 (in Build/loader-common)

Thoughts?

BTW, I can get an image to build, for now forcing small_flash (but need to get that out, it's temporary ... allows debug though!). When I do get an image ... is binwalk the best way to "inspect" it? I admit, I'm not getting too much out of it, but it may just be me. Perhaps look at the binwalk output for that test log (binary) that you mentioned / sent a link to? Or ... is there a better tool to use to inspect the output file "format"?

Thanks!

arrmo · June 27, 2020, 7:36pm

Making some progress, I think ... . First off, I said ... this flying blind is for the ! LOL. So I followed this great info,
Post #18, https://forum.archive.openwrt.org/viewtopic.php?id=61445&p=1#p325769

and now have a serial port on my HooToo (working). BTW, u-boot is default on this device for 57600 baud, so I have also made that change in my build, so the rate quits changing on me. But that aside, I think I am getting to lzma-loader, just a bit fuzzy on what variables to set to get it to jump to the right place. Here is the serial log, https://paste.ubuntu.com/p/DJrC352bfz/

Thoughts?

Thanks!

arrmo · June 27, 2020, 10:08pm

OK, I think I'm making progress - just one thing left (I think ... LOL).

I believe I have it booting now - here is the log, https://paste.ubuntu.com/p/pSXvmmm5Mz/

But, the case above is with small_flash (where I wanted to start, to test). If I then remove this, I get the following, https://paste.ubuntu.com/p/YVKZNz4vTk/

This latter one also makes sense - the u-boot CRC check fails, as it isn't loading the full kernel / image, so then the CRC doesn't pass. So, how to have u-boot ignore it and proceed? Or, just run the CRC on the lzma-loader? Or some other way around it? I really think this is the only sticking point now (but I may be wrong ... LOL).

Thanks!

xabolcs · June 28, 2020, 12:24am

As I already wrote previously in the PR: using the lzma-loader recipe wont solve your 1.5M kernel limit!

You have to exercise the LOADER_FLASH_OFFS variable!
And, because of that, you should introduce in the ramips target those new (copied from ath79) (append-)loader-okli recipe.

Btw, if your kernel works without therelocate-kernel then don't use it!

Sorry for not writing so detailed answers to you, but I am short on free time.

arrmo · June 28, 2020, 12:27am

Yes, that makes sense! I was thinking (incorrectly?) that using lzma-loader pulled directly from flash ... no? I admit, that's where I get a bit confused. I am pointing lzma-loader to the flash address ... I think . Will keep digging - it does seem that the other boot log you sent only really seems the 3k lzma loader - and then I think directly pulls from flash?

Understand, NP!

Thanks!

xabolcs · June 28, 2020, 12:35am

Follow my instructions from the PB44 thread I linked at the beginning of this thread!
And copy some missing recipes from ath79!
And change that AR71XX_FLASH_START address to your needs!

arrmo · June 28, 2020, 1:59am

I am trying! ... LOL. Just struggling a bit to find my way through the twists and turns, but I will get there .Thanks!!!

arrmo · June 28, 2020, 2:49am

OK, trying to follow your example, made the changes like in the link (and removed the LOADADDR from the lzma-loader Makefile - that I had added). I think I have the right recipes in place, but I'm skeptical. I say that because binwalk on the files still shows a large kernel (below), vs. I would think it would show the ~ 3k file ... right?

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0xD9CFC9AA, created: 2020-06-25 00:36:19, image size: 1346273 bytes, Data Address: 0x80000000, Entry Point: 0x80000000, data CRC: 0x84171163, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: none, image name: "MIPS OpenWrt Linux-4.14.180"
5628          0x15FC          Copyright string: "Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>"
8256          0x2040          LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 4156823 bytes
1301289       0x13DB29        MySQL ISAM compressed data file Version 11
1346337       0x148B21        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 2579482 bytes, 1353 inodes, blocksize: 1048576 bytes, created: 2020-06-25 00:36:19

When I try to boot, it almost immediately says,

Looking for OpenWrt image... not found!

System halted!

I tried 0x1 and 0x10 search, and I THINK I have the flash address right in loader.c

#define AR71XX_FLASH_START	0x80500000
#define AR71XX_FLASH_END	0x80cb0000

Hmmm. Still trying, but pulling my hair out ... LOL.

BTW, here is the device config now,

  SOC := mt7620n
  IMAGE_SIZE := 7872k
  DEVICE_VENDOR := Ravpower
  DEVICE_MODEL := WD03
  DEVICE_PACKAGES := kmod-usb2 kmod-usb-ohci
  LOADER_TYPE := bin
  LOADER_FLASH_OFFS := 0x22000
  BLOCKSIZE := 256k
  COMPILE := loader-$(1).bin
  COMPILE/loader-$(1).bin := loader-okli-compile
  KERNEL := kernel-bin | append-dtb | lzma | uImage lzma -M 0x4f4b4c49 | loader-okli $(1) 8128 | uImage none
  KERNEL_INITRAMFS := kernel-bin | append-dtb | uImage none```

Thanks!

arrmo · June 28, 2020, 12:00pm

OK, this part was key!

Ahh, I had to delete those loaders at build_dir/target-/linux-/loader-* between rebuilds!

Deleting those files, rebuilding (and adding some debug code), and I think it's working (mostly),

Looking for OpenWrt image...
> Flash, base address: 0xa0500000
> Flash, start offset: 0x00000000
> Flash, end offset: 0x0f000000
> Flash, step size: 0x00000001
> Flash, search size: 0x0f000000 (251658240 bytes)
found at 0xa0502840

This address partially makes sense. I say that because from the programming, I see that the flash address is 0x80500000 => so the 0x500000 makes sens (as does the 2840, but more on that below). Can't explain the 0xa0000000 though ... can you?

The only remaining item is the binwalk output, below,

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0x4AB4FFC3, created: 2020-06-25 00:36:19, image size: 1348385 bytes, Data Address: 0x80000000, Entry Point: 0x80000000, data CRC: 0xD175EC5B, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: none, image name: "MIPS OpenWrt Linux-4.14.180"
5740          0x166C          Copyright string: "Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>"
10368         0x2880          LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 4156823 bytes
1303401       0x13E369        MySQL ISAM compressed data file Version 11
1348449       0x149361        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 2579482 bytes, 1353 inodes, blocksize: 1048576 bytes, created: 2020-06-25 00:36:19

So the image is still not quite right, and it will still break the CRC check. Any pointers on how to have the header only CRC and capture the lzma-loader? The build can still be larger, and loaded directly, but need uboot only look at the lzma-loader "image".

Thanks!

xabolcs · June 28, 2020, 1:41pm

This is the KSEG1 address. See how I describe the DIR-860L devices loadaddress in my mailing list mail - pasted above.

Please paste here full serial log about the CRC error!

arrmo · June 28, 2020, 1:54pm

Yep, saw that - just not sure what "KSEG1" means ... some sort of virtual addressing I assume, but not sure where it is defined / explained. Not a huge issue, just trying to understand! But it seems that flash space is re-mapped to KSEG1?

NP! I think the link above is what is needed, but yell if not. Re-adding here - it can be hard to follow my (lack of) logic, sorry.
https://paste.ubuntu.com/p/YVKZNz4vTk/

The issue seems to be the header, wanting CRC over the full image ... whereas it really needs to be the "3k lzma-loader" image, right? I think it may just be how I'm generating / packing the image file overall.

Thanks!

xabolcs · June 28, 2020, 2:30pm

How did you manage to work the OKLI loader if it dies at u-boot stage with bad data crc?
One comes from initramfs (through tftp?!), other scomes from a freshly flashed factory.bin?

Sure the u-boot's point of view the kernel should be a few kbyte sized kernel, the (uncompressed) LZMA Loader, as in this (previously linked) bootlog: https://gist.github.com/mpratt14/e9dd3efb5e5f67114d27526e0bbdcece

arrmo · June 28, 2020, 2:44pm

Sorry, tried to say this above - likely poorly . If I force (for testing only!) small_flash, then it works ... I'm doing this to have a valid load, to debug the lzma_loader. Now that is working, need to go back to the full build (i.e. remove small_flash) => then it crosses the 1.5 MB boundary, and the CRC fails.

So I need to "fix" the image - need it to CRC (and "see") only the 3 kB portion ... let that boot then, and it will load the "oversized" kernel directly from flash at 0xa0502840 (as found, above).

Right?

So to me, it's about building the sysupgrade.bin file, so the "correct" header is seen for CRC, boots lzma-loader. But I may be wrong!

Thanks!

Oh, and yes - agree with you on that boot log. Just struggling to get to where u-boot sees the image as lzma-loader (3 kB).

arrmo · June 28, 2020, 3:09pm

OK, figured out KSEG1, from a MIPS32 book . Now, just to get my image / CRC properly going - and I think this may / will work. Thanks!

arrmo · June 28, 2020, 5:04pm

I admit, pulling my hair out though - just can't seem to get the right recipe for the header that tells u-boot to only CRC the 3kB, and then run the loader, read from flash. Dang it!

Trying to mimic from here, https://github.com/mpratt14/openwrt/commit/05a649c9a3ee26270ead8af3fe597c1ccb8f836a, but then end up with ... arrgh!

Initialize vs configure module
raspi_read: from:1d0000 len:1000
Initialize GPIO
check: 0
Input i key to enter menu 0
raspi_read: from:50000 len:180000
## Booting image at 80500000 ...
Bad Magic Number,4F4B4C49