HooToo TM-05, Add LZMA Loader

xabolcs · June 29, 2020, 7:43pm

The loader find the kernel when you let it to find!
The flash offset is 0x0, but the step size is 0x00000001 and the end offset is 0x0f000000.

The current configuration on OpenWrt's master has a "production mode" configuration for the Loader: the end offset is 0x0 - it will check the base + LOADER_FLASH_OFFS address, and dies if it doesn't find anything.

So in development time (like this thread), you modify step size, end offset and friends, you basically just switch off this "production mode".

xabolcs · June 29, 2020, 7:49pm

Two successive make rebuilds too much for me. A make clean would cause even more rebuild!

mpratt14 · June 29, 2020, 8:02pm

We have the offset figured out, so I recommend going back to the original source code exactly as it was just to be safe.

arrmo · June 29, 2020, 9:35pm

I have, well - except for the slight delta below (which is required, right?),

diff --git a/target/linux/ramips/image/lzma-loader/src/loader.c b/target/linux/ramips/image/lzma-loader/src/loader.c
index c73b60b351..ee8379a715 100644
--- a/target/linux/ramips/image/lzma-loader/src/loader.c
+++ b/target/linux/ramips/image/lzma-loader/src/loader.c
@@ -28,8 +28,8 @@
 #include "printf.h"
 #include "LzmaDecode.h"

-#define AR71XX_FLASH_START     0x1f000000
-#define AR71XX_FLASH_END       0x1fe00000
+#define AR71XX_FLASH_START     0xa0500000
+#define AR71XX_FLASH_END       0xaf000000

 #define KSEG0                  0x80000000
 #define KSEG1                  0xa0000000

This is just about where the flash is on my hardware. BTW, with this - still the error message above, and still hanging when trying to boot . I did a complete rebuild ... even removed the full build and staging directories, build from scratch. I also did the same, built a non-loader version ... and that booted. Very odd. Will keep digging.

xabolcs · June 29, 2020, 10:57pm

Very interesting finding.

With my mt7621 based DIR-860L I found AR71XX_FLASH_START to be 0x10000000.
You just found it for mt7620, and it looks like 0x0!
(And gch suggested me 0x1fc00000 originally! )

Please:

revert AR71XX_FLASH_END as it's unused
modify AR71XX_FLASH_START to 0x00000000
and set up the LZMA Loader debug environment: flash step, flash max, messages

After those preparings please verify that the loader finds the OKLI magic at 0xa0500dd8 ("found at 0xa0500dd8")!

xabolcs · June 29, 2020, 11:03pm

About the missing rootfs ...

I would try rollback the DTS in it's original state!
And just pad the rootfs to eraseblock boundary, as @mpratt14 suggested!

mpratt14 · June 29, 2020, 11:15pm

The rootfs was missing because the wrong mtdsplit parser was used in combination with the loader. It doesn't matter how much padding there is before or after, pad-rootfs image command is sufficient. Like I said I ran into the exact same issue, and then following the Pisen DTS, changed the compatible property and a separate partition for the loader. I'm sure you remember as you showed it to me first

github.com

openwrt/openwrt/blob/master/target/linux/ath79/dts/ar9341_pisen_wmb001n.dts

// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
/dts-v1/;

#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>

#include "ar9341.dtsi"

/ {
	model = "PISEN WMB001N";
	compatible = "pisen,wmb001n", "qca,ar9341";

	aliases {
		serial0 = &uart;
		led-boot = &led_wifi;
		led-failsafe = &led_wifi;
		led-running = &led_wifi;
		led-upgrade = &led_wifi;
	};

This file has been truncated. show original

mpratt14 · June 29, 2020, 11:18pm

I would honestly revert all changes to source, even this one, as we know the offset well enough. At this point getting the loader to run but not find the kernel is better than the loader not running at all.

xabolcs · June 29, 2020, 11:23pm

If you think you are right then please explain how could I boot with this OKLI method my DIR-860L without touching the DTS?!

The openwrt,okli compatible was only needed to not waste the 1.5Mbyte - 3Kbyte flashspace, IMHO.

Sure, a working loader is the first step, a working rootfs is the second!

arrmo · June 30, 2020, 12:00am

Sorry, was offline just a bit - but back . Awesome comments, and I agree with them. My struggle is just what you note here - I can't get the loader to boot any more. Let me do this ... revert many of my changes, and then capture a diff (for you to comment on) ... sound reasonable?

OK, and here is the diff, https://paste.ubuntu.com/p/3WhC5TMvWV/

Will make clean, rm loader, rebuild ... hopefully the loader boots now!

xabolcs · June 30, 2020, 12:12am

If it helps, you could try the relocate-kernel and pad-to (a few kbyte, 16-24-32-48-64 == blocksize) after append-loader-okli.
My DIR-860L originally needed the relocate-kernel, so I had to relocte the LZMA Loader too!
I don't think you have to use it (as the small flash works without relocate-kernel), but let's check if it helps!

arrmo · June 30, 2020, 12:14am

Sure, will try that if the first run fails - thanks!! I actually have tftpboot working also (to download code - boot was still failing). But let's see with the new build.

arrmo · June 30, 2020, 1:22am

Arrgh! OK, went all through that, and the same issue. I really think I have an address wrong here somewhere, but I admit - I just don't see it. I build the load, flashed it, then booted - but it just hangs like below,

============================================
Ralink UBoot Version: 4.1.0.0
--------------------------------------------
ASIC 7620_MP (Port5<->None)
DRAM component: 512 Mbits DDR, width 16
DRAM bus: 16 bit
Total memory: 64 MBytes
Flash component: SPI Flash
Date:May  7 2014  Time:05:35:03
============================================
icache: sets:512, ways:4, linesz:32 ,total:65536
dcache: sets:256, ways:4, linesz:32 ,total:32768

 ##### The CPU freq = 580 MHZ ####
 estimate memory size =64 Mbytes
raspi_read: from:40028 len:6

Initialize vs configure module
raspi_read: from:1d0000 len:1000
Initialize GPIO
check: 0
Input i key to enter menu 0
raspi_read: from:50000 len:180000
## Booting image at 80500000 ...
   Image Name:   MIPS OpenWrt Linux-4.14.180
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    3240 Bytes =  3.2 kB
   Load Address: 80000000
   Entry Point:  80000000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 80000000) ...
## Giving linux memsize in MB, 64

Starting kernel ...

Booting image at 80500000 ... => this does seem write, as during flashing,

write(0x80500000, 0x50000, 3932938)
raspi_erase_write: offs:50000, count:3c030a
raspi_erase: offs:50000 len:3c0000

It seems to see the image there, but then jumps to 0x8000000 - perhaps this is not right? Do I need to point the load address somewhere else?

Thanks!

mpratt14 · June 30, 2020, 2:07am

I was hoping you could explain
any other value for the compatible property does not allow my ENS202EXT to boot anymore. How else would I do it?

I have tried
denx,uimage
engenius,firmware

arrmo · June 30, 2020, 2:17am

OK, sort of got it to boot, so the code is there and functional - but had to set the load address / entry point to 0xa0050000 ... so directly from flash. Doesn't really run properly there, stops after,

OpenWrt kernel loader for MIPS based SoC
Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>
Looking for OpenWrt image...

But at least I know it's there . Hmmm ... need to figure out this dang addressing!

mpratt14 · June 30, 2020, 2:35am

Load Address and Entry point should not be touched. You had it working in the past as it was and as it should be, with all the software running from RAM.

From your git diff:

+loadaddr-$(CONFIG_TARGET_ramips_mt7620) := 0x80000000
 loadaddr-$(CONFIG_TARGET_ramips_mt7621) := 0x80001000

You could try a different space in RAM if you wanted, just like the mt7621

+define Build/loader-okli-compile
+	$(call Build/loader-common,FLASH_OFFS=$(LOADER_FLASH_OFFS) FLASH_MAX=0x0f000000)
+endef

Lets have FLASH_MAX back to 0

+  LOADER_FLASH_OFFS := 0x0

flash offset of kernel is still not 0. You are making the loader try to load the bootloader.
I see you have pad-to $$$$(BLOCKSIZE) before append-kernel
so the flash offset of kernel will be 0x60000
And in that case, firmware partition would be
reg = <0x60000 0x7a0000>;

BTW, since you are defining BLOCKSIZE within the same device make config, you only need 2 dollar signs
pad-to $$(BLOCKSIZE)

arrmo · June 30, 2020, 2:38am

Ya, but I reverted all the changes, now can't get it working again ... . I think it was config.h or something else that was getting me there, trying to figure it out now.

Thanks!

xabolcs · June 30, 2020, 5:14am

Please revert your device back to OEM firmware, and check the serial log while booting!

BTW how does your TFTP recovery works?
It downloads and flashes the sysupgrade(?), factory(?) OpenWrt image? And then reboots?

arrmo · June 30, 2020, 1:13pm

Did that, works just fine! Serial log here (before it changes serial port speed, crap on the screen ) ... https://paste.ubuntu.com/p/TSQcq8wmrH/

It's from the CLI in u-boot ... I can load to memory, try to boot from there. Or even flash, boot SPI flash or memory. Basically, u-boot, pretty flexible, works well!

And this is where I'm pulling my hair out. Cannot get back to where I get the image to boot . I tell it to read from 0x80000000 - it just hangs (which I have figured out means it's not really calling the code). I admit, I'm confused by RAM vs. flash addresses, and virtual addressing is not helping. Any guidance you have would be greatly appreciated! My sysupgrade.bin file looks OK, headers, content, etc. But when I try to boot it, it either hangs, or I get this error message,

hanging case (sysupgrade stored in flash)

raspi_read: from:50000 len:180000
## Booting image at 80500000 ...
   Image Name:   MIPS OpenWrt Linux-4.14.180
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    3310 Bytes =  3.2 kB
   Load Address: 80000000
   Entry Point:  80000000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 80000000) ...
## Giving linux memsize in MB, 64

Starting kernel ...

error message (manually load file to 80000000, tftp ... boot 80000000)

MT7620 # bootm 80000000
## Booting image at 80000000 ...
   Image Name:   MIPS OpenWrt Linux-4.14.180
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    3310 Bytes =  3.2 kB
   Load Address: 80000000
   Entry Point:  80000000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... LZMA ERROR 1 - must RESET board to recover

BTW, a tidbit perhaps? In u-boot, I see,

KSEG1ADDR(NetTxPacket) = 0xA3FE3F00

And one more, u-boot gives me the option to boot from RAM, it loads the sysupgrade.bin file over tftp. I get (hangs! But loaded to 0x80500000, booting from 0x80000000 ... code not there?),

Loading: Got ARP REPLY, set server/gtwy eth addr (d8:eb:97:b8:40:28)
#################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ######################################################
done
Bytes transferred = 3932938 (3c030a hex)
NetBootFileXferSize= 003c030a
## Booting image at 80500000 ...
   Image Name:   MIPS OpenWrt Linux-4.14.180
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    3310 Bytes =  3.2 kB
   Load Address: 80000000
   Entry Point:  80000000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 80000000) ...
## Giving linux memsize in MB, 64

Starting kernel ...

Thoughts? Thanks! Going crazy trying to get this booting again .

xabolcs · June 30, 2020, 1:23pm

I will try your tree from GitHub - the small-flash one. And will try my luck with the OKLI version of LZMA Loader in the evening!