RAVPower WD-03, Add LZMA Loader

Yes, agreed! Was thinking getting the RAVPower up would take longer .... :laughing:. Next to see how much can be made common, then agreed - common dtsi where possible.

Checking out your new branch,

$ git fetch arrmo
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Total 26 (delta 19), reused 19 (delta 19), pack-reused 7
Unpacking objects: 100% (26/26), done.
From https://github.com/arrmo/openwrt
 + f0bfa630c8...a87b7c2b55 hootoo-tm05   -> arrmo/hootoo-tm05  (forced update)
 * [new branch]            ravpower-wd03 -> arrmo/ravpower-wd03
$ git fetch arrmo ravpower-wd03:ravpower-wd03 
From https://github.com/arrmo/openwrt
 * [new branch]            ravpower-wd03 -> ravpower-wd03
$ git checkout ravpower-wd03 
Switched to branch 'ravpower-wd03'
$ cd target/linux/ramips/dts/

and diffing the two DTS (diff -U8 mt7620n_hootoo_tm05.dts mt7620n_ravpower_wd03.dts), shows a lot common stuff.

--- mt7620n_hootoo_tm05.dts	2020-07-15 20:55:08.542224119 +0200
+++ mt7620n_ravpower_wd03.dts	2020-07-15 20:55:08.542224119 +0200
@@ -2,40 +2,34 @@
 /dts-v1/;
 
 #include "mt7620n.dtsi"
 
 #include <dt-bindings/gpio/gpio.h>
 #include <dt-bindings/input/input.h>
 
 / {
-	compatible = "hootoo,tm05", "ralink,mt7620n-soc";
-	model = "HooToo TM05";
+	compatible = "ravpower,wd03", "ralink,mt7620n-soc";
+	model = "Ravpower WD03";
 
-	aliases {
-		led-boot = &led_power;
-		led-failsafe = &led_power;
-		led-running = &led_power;
-		led-upgrade = &led_power;
-		label-mac-device = &ethernet;
+	chosen {
+		bootargs = "console=ttyS0,57600";
 	};
 
 	leds {
 		compatible = "gpio-leds";
 
-		led_power: power {
-			label = "tm05:blue:power";
-			gpios = <&gpio3 0 GPIO_ACTIVE_LOW>;
-			default-state = "on";
+		green-wifi {
+			label = "wd03:green:wifi";
+			gpios = <&gpio2 0 GPIO_ACTIVE_HIGH>;
 		};
 
-		wifi {
-			label = "tm05:green:wifi";
-			gpios = <&gpio2 0 GPIO_ACTIVE_HIGH>;
-			linux,default-trigger = "phy0tpt";
+		blue-wifi {
+			label = "wd03:blue:wifi";
+			gpios = <&gpio3 0 GPIO_ACTIVE_HIGH>;
 		};
 	};
 
 	keys {
 		compatible = "gpio-keys";
 
 		reset {
 			label = "reset";
@@ -138,17 +132,17 @@
 	status = "okay";
 };
 
 &ohci {
 	status = "okay";
 };
 
 &ethernet {
-	mtd-mac-address = <&factory 0x28>;
+	mtd-mac-address = <&factory 0x4000>;
 };
 
 &wmac {
 	ralink,mtd-eeprom = <&factory 0x0>;
 };
 
 &state_default {
 	gpio {

As I see the above:

  • please think about migrating compatible from ravpower,wd03 to ravpower,rp-wd03
    • in case of migration, please rename the dts: mt7620n_ravpower_wd03.dts -> mt7620n_ravpower_rp-wd03.dts
  • plese refresh the model to RAVPower RP-WD03 FileHub ... or without FileHub
  • please harmonize the aliases node
  • please drop the chosen node
  • please verify the ethernet address at both address

2 of those 3 already done and working :slight_smile:. We're on the same page!

Just a slight bit sidetracked right now - seems like sysupgrade isn't saving + restoring settings ... huh?!?!

Maybe it's a regression of including append-metadata to rootfs on the initial flash? :thinking:

Use $$(sysupgrade_bin) as factory.rootfs.bin (and rename factory.bin to factory.kernel.bin)
You could find this recipe (and binaries) in one of my hootoo-tm05-*.tar.gz :wink:

define Build/append-metadata
	$(if $(SUPPORTED_DEVICES),-echo $(call metadata_json,$(SUPPORTED_DEVICES)) | fwtool -I - $@)
	[ ! -s "$(BUILD_KEY)" -o ! -s "$(BUILD_KEY).ucert" -o ! -s "$@" ] || { \
		cp "$(BUILD_KEY).ucert" "$@.ucert" ;\
		usign -S -m "$@" -s "$(BUILD_KEY)" -x "$@.sig" ;\
		ucert -A -c "$@.ucert" -x "$@.sig" ;\
		fwtool -S "$@.ucert" "$@" ;\
	}
endef

I don't understand all of OpenWrt recipes but this doesn't look something which causing a sysupgrade regression.

I need to take a look - sysupgrade is supposed to backup / restore the config files ... not so sure this is an image issue, vs. that backup process. But let me look.

1 Like

Hmmm ... getting a bit worried this is related to mtd-concat. Let me explain why I say that,

  1. I don't think it's metadata related - that's more about file / board compatibility (e.g. Fwtool - please explain)
  2. I did a command line sysupgrade. It shows me all the files being backed up (Saving config files...), and then afterwards, Commencing upgrade. Closing all shell sessions.. After reboot, the files are not restored. But I do see on the serial port (and as noted here, https://openwrt.org/docs/guide-user/installation/sysupgrade.cli) ... and the "firmware" note is what scares me a bit.
Appending jffs2 data from /tmp/sysupgrade.tgz to firmware..                                                                                                                             
Upgrade completed
Rebooting system...
  1. So then, I tried a manual backup of the config (download file), sysupgrade (config lost), and then manual restore of the config (upload saved file). It lists the same files (as were supposed to be backed up by sysupgrade), but it restores them properly (and includes a reboot).

Thoughts?

Thanks!

Try to grab the /tmp/sysupgrade.tgz after a sysupgrade. I am thinking this is a RAM issue

That would be good! I can try, but it immediately jumps to reboot. Let me see what I can do.

OK, checked out sysupgrade, tried a couple options,

  1. -b, create backup. Downloaded the resulting file (/tmp/sysupgrade.tgz), opened it ... files are there, it looks fine. Of course, if it's not properly stored somewhere persistent, that would be a problem.
  2. -T, test. Got the error, Image metadata not found => so perhaps the metadata is the issue? Seems odd.

Thoughts?

Yeah this is what I'm thinking. Config restore occurs on the firstboot after upgrade. Somewhere in between generation, storage in RAM, storage on flash, something goes wrong...

In the meantime, factory install process should not have append-metadata anyway. For now, make another image, you can rename the images too like factory-kernel.bin and factory-rootfs.bin and the latter one would be copy of the sysupgrade.bin recipe except for metadata.

I've never done this...but you might have done it wrong...

sysupgrade program assumes you have a firmware image, you specify a config with the -f flag. So to test a config file you probably have to do

Edit: I think you can only test a config file if you are also testing an image...

sysupgrade -T -f something.tgz firmware.bin

You can interrupt firstboot after upgrade by triggering failsafe mode

Entirely possible :wink:

Not sure I need to, checked file that was generated ... it's fine. Wondering more about adding metadata, but really thinking that's only about board vs. image check, agreed?

Tried going throug sysupgrade (script), it's not clear to me yet where it's storing the config file :frowning_face:

Might be a difference between you generating it VS. sysupgrade generating it, moving it to flash and back to RAM on firstboot

OK, will force sysupgrade - and failsafe. Not quite sure what to look for yet ... :rofl:. Will poke around though!

Hmmm ... in failsafe, did mount_root, but got,

mount_root: no usable overlay filesystem found, using tmpfs overlay

Thinking this may be part of the issue? Nothing in /overlay, which is where I assume the config should be stored?

OK, not so sure now this is specific to this device / load. I see on the restore (from file), in the serial log,

[    7.335532] jffs2: notice: (456) jffs2_build_xattr_subsystem: complete building xattr subsystem, 3 of xdatum (2 unchecked, 1 orphan) and 6 of xref (1 dead, 0 orphan) found.
[    7.368041] mount_root: switching to jffs2 overlay
[    7.409920] overlayfs: upper fs does not support tmpfile.

And checking, files are there (in /overlay/upper). But, on sysupgrade (and I have noticed this before),

[   82.932356] jffs2_scan_eraseblock(): End of filesystem marker found at 0x7000
[   82.963302] jffs2_build_filesystem(): unlocking the mtd device...
[   82.963396] done.
[   82.979592] jffs2_build_filesystem(): erasing all blocks after the end marker...
[  121.966001] done.
[  121.984820] jffs2: notice: (1599) jffs2_build_xattr_subsystem: complete building xattr subsystem, 0 of xdatum (0 unchecked, 0 orphan) and 0 of xref (0 dead, 0 orphan) found.
[  122.253091] overlayfs: upper fs does not support tmpfile.

This is roughly 2 minutes in! But, way back at 7 seconds again,

[    7.157593] mount_root: no usable overlay filesystem found, using tmpfs overlay

So when mount_root runs, jffs2 isn't there yet.

Thoughts?

AFAIK this is normal behavior for a firstboot. preinit cycle attempts to mount jffs2, does not find it. That is what triggers the rest...jffs2_scan_eraseblock() and jffs2_build_filesystem()

I'm more interested in the integrity of sysupgrade.tgz before jffs2 is made

Before you do this, get rid of the append-metadata from tftp install first i guess...


Okay so looking at what you said...looks like this is more complicated than I thought...

Lets do a hacky thing instead...

The file that triggers config restore is
package/base-files/files/lib/preinit/80_mount_root

it does so with the lines

tar xzf /sysupgrade.tgz
sync

So do a make clean, build an image after removing those lines (and you can edit the echo if you want). Then after firstboot the file will still be there in root after jffs2 is built. You can check it's integrity with binwalk or whatever and unpackage it yourself with the same command.

An alternative hack...stop the automatic reboot after upgrade

in this file remove the last section, everything after v "Rebooting system..."

package/base-files/files/lib/upgrade/do_stage2

Does this error affects HT-TM05 too? Or just RP-WD03? :thinking:
What's your device recipe?

I'm unsure if it's caused by mtd-concat, because if it affects then ENS202EXT would be affected too.

@mpratt14, append-metadata shouldn't do anything bad here, as the current RP-WD03 recipe uses sysupgrade.bin for the initial OpenWrt flashing over OEM firmware since the beginning.

About the sysupgrade problem, ... I don't have anything in mind whats happening here. :confused: