The hash works out if that's what you mean:
root@LEDE:~# sha256sum /boot/boot/apollo3g.dtb
fb2f91822c74e8b9f80f310c5cd7c8f64dfc67fc2eaf740ff8a342a312360df9 /boot/boot/apollo3g.dtb
If not, please tell me how I can verify it's loading the correct dtb. I am running the 17.01.2 release build, and an 80GB 2.5" WD Scorpio (my "test drive"), if that matters.
Ok, the checksum matches. Without the same hardware it's difficult to tell. Is 17.01.2 using 4.4 or 4.9 kernels? In any case I do have a backup script that could help in this situation. It requires that you install hdparm though.
#!/bin/sh
# soft-off program
# This program uses OpenWRT/LEDE's sysupgrade functionality to get
# a device into a "soft-off" state. This is because not all devices
# do have a working "poweroff" and if a user wants to unplug the
# device in order to move it. she/he might want this functionality.
# Currently, this script tries to put all attached harddrives into
# standby mode (which is why it requires hdparm) and tries to powers
# off all LEDs.
. /lib/functions.sh
. /lib/functions/system.sh
. /lib/upgrade/common.sh
HDPARM=$(command -v hdparm)
[ -z "$HDPARM" ] && {
(>&2 echo hdparm not installed.)
exit 1
}
case "$1" in
stage2)
for dev in /dev/sd*[a-z]; do
hdparm -y $dev
done
for leds in /sys/class/leds/*; do
echo 0 > $leds/brightness
done
while :; do
sleep 10000;
done
;;
*)
install_bin /sbin/upgraded
install_bin $0
install_bin /sbin/hdparm
ifdown -a
ubus call system sysupgrade "{
\"prefix\": $(json_string "$RAM_ROOT"),
\"path\": $(json_string "badfile"),
\"command\": $(json_string "$0 stage2")
}"
;;
esac
simply copy it to the MBL (like /sbin/softoff.sh) . make it executable and run it without any argument.
17.01.2 is using kernel 4.4.71
That's not a problem, I use hdparm to monitor and standby the drive anyway (I got a barebones replacement for MBL's original "monitorio.sh" in the works). However, your script bugs out:
/sbin/softoff.sh: line 51: json_string: not found
(Edit: If it does what I think it does, do we really need a function to convert a simple string to JSON?)
After that, the system halts (without triggering stage 2).
If I may ask, I'm not familiar with sysupgrade called that way, what exactly are you doing there?
ah ok, 4.9.37 here. Since I'm obviously using trunk
ah ok, this doesn't work because:
is not available for 17.01.02 (This was a series of NeoRaider, so this is not the only patch, but I think this is were he started). So, no there is no easy fix... I mean you could try to replace the /lib/upgrade/common.sh and /lib/upgrade/stage2 from the current trunk. But this could very well break the "real sysupgrade".
The idea behind this script is, that the new staged sysupgrade (which is standard in trunk) copies a minimal system to a ramdisk and calls pivot_root. As you might know, sysupgrade is designed to overwrite the rootfs, so it needs to terminate all programs and unmount everything. This script uses sysupgrade's infrastructure to do all the cleanups, but instead of doing any upgrades, it calls hdparm -y for all /dev/sd*[a-z], power off all leds and do a endless sleep loop.
So I went ahead and upgraded to latest snapshot build (r4651 as I'm writing this, hello people from the future!)
Well it's not the kernel or anything else connected to the LEDE version, I still get the same behaviour, drive comes up 20~25 seconds later. Must be something with my drive then. Maybe I'll hook up the serial port when I'm in the mood, which right now I am not.
Your "softoff" script, however, works now. Which is most definitely a solution. Thank you for all your effort, and your patience with my stupid questions.
I'm not entirely sure if I want to remain on a snapshot build at the moment. Maybe we'll get a release build with the new sysupgrade soon. From your experience, is there anything in the snapshot that makes a huge difference? (For example, does hardware AES acceleration work?)
What really peeves me out, though, is that the partition table does not survive a sysupgrade. Which it should, there explicitly are functions in sysupgrade that are supposed to save and restore the partition table. Of course I can recreate the missing partition(s) after sysupgrade without blanking the ext4 records, they are available again and seemingly fine after a reboot. But I feel that's not the way it's supposed to be. (And, yes, I do remember your warnings in your e-mail.)
I just used hdparm to spinoff drive...(just that, nothing else needed) It stays in that state (tested all night), previously it is safe to sync and umount data partition, and stop services that access files to avoid problems... you could unplug hardrive securely (I tested it) and unplug power cord securely too (since it is sync and unmounted there is no data corruption or loss at all).
Yes, these are all part of the package feeds. So it isn't possible to add them to the DEFAULT_PACKAGE for the MBL Target.
That's actually on of the "features" of LEDE. Every essential userspace daemon can run of a read-only mounted root filesystem. So, the HDD isn't accessed as often as with Debian. The reason why I chose the "sysupgrade" way, was to prevent the dreaded "fs wasn't unmounted properly - run fsck" on the next boot.
Well, some good news. The "sysupgrade repratitions the hdd" and the crypto4xx have been partially fixed. I sent the crypto4xx patches upstream (currently in Herbert Xu's cryptodev-2.6 branch - but hopefully it will tickle down/up).
From what I can tell, the MyBookworld? people just couldn't be bothered to look into it?
The crypto4xx driver (the one in the MBL GPL tar) was just broken. APM/AMCC developer just disabled the testcases in the cryptotest-manager. So it wasn't much of use.
As for AES-CBC,RFC3686-CTR,GCM,CCM Performance: it's a bit of a mixed bag. The hardware utilized interrupt mitigation. So, unless the hardware is pushed, it will lead to poor results. And many cryptobenchmarks suffer because of this. For example: cryptsetup benchmark will just report 8Mb/s for aes-cbc. Which is lower than the CPU.
Note: the crypto hardware does not officially support aes-xts.
However in IPSeC with aes-gcm/ccm the driver can sustain 200-210 MBit/s (with 1500 MTU - it's much more with 4500 and 9000 MTU / 16k pages). The aes-cbc dm-crypt can read at 34 MIB/s (~280 Mbps with 512 Bytes/Sector. having a driver with 4K sectors would be interesting to test, it could be around 500-600 Mbps).
Still, It's now possible sent data (samba) over an encrypted link (ipsec) and read from/write it to an encrypted partition.
As for the sysupgrade partition stuff: everything is in my apm821xx-tree. But it hinges on a patch that fixes the get_partitions function. https://patchwork.ozlabs.org/patch/825110/ . I can post it once the problem with get_partitions gets fixed.
Anyway. If there are any comments about the crypto or the partition stuff please make a new thread.
Nope, I don't have any comments, just a huge thank you for all your effort.
Just a quick question that I feel doesn't warrant a new thread (and may be relevant to this device in general): What's the process of getting all your patches/improvements into the LEDE codebase, and how long will that possibly take? I guess what I'm asking is, will we see that in a stable release anywhere soon?
The crypto changes have been sent upstream and they are on their way to the vanilla linux kernel. In theory, these can then be added as backports then. As for the sysupgrade fix: this requires a working get_partition() function. If you want to have it anytime soon, the best bet is to bribe someone with commit access to look at it.
The sysupgrade patch would be good to get soon, and I guess it also concerns other platforms, not just ours. But I can live without hardware crypto accel. All in all, I just wanted some sort of idea about the process, since I'm utterly unfamiliar with it. Thank you. 
I posted the improved sysupgrade patch on the lede-dev mailing list. The get_partition function fix has been accepted recently, but it's still queued in Felix staging tree. I hope it will move to master soon.
http://lists.infradead.org/pipermail/lede-dev/2017-November/010076.html
Interesting. That patch/enhancement is quite more sophisticated than I expected. From how I read the original(ly intended) way of "keeping the partition table", it simply went ahead and temporarily saved the partition layout, did a regular sysupgrade (that only overwrites boot and rootfs anyway), and restored the partition layout afterwards.
Do you think they might take issue that the functions are so specifically targeted at the MBL (even if just in function names)? I could imagine that this "nondestructive" sysupgrade could apply to many more devices.
At any rate, thank you for your continued efforts. If you by any chance happen to be at 34c3, let me issue one or more liquid thank yous. 
Oh, The sysupgrade function bodies are a 1:1ish copy from the x86's current /lib/upgrade/platform.sh.
so, it pretty much what you are describing... But in reverse.
The problem was, that one crucial function "get_partitions()" malfunctioned on power-pc due to
big. vs little endian. That's why I had to wait for the fix.
In that case, be sure to thank Chris Blake too. He started this apm821xx idea back with the MR24. 
@chunkeey: I just noticed your crypto patches having been commited to trunk today. I can't wait to try the now-working crypto and sysupgrade patches, thank you for your effort!
So we will have hw accelerated crypto on this device???? 
I finally got around to testing the new builds, and I have to say, they work an absolute treat. This will replace my stock firmwares now. At least for my smaller disks ...
I'm doing my own images (using ImageBuilder to include block-mount and samba and a few custom scripts). When sysupgrading using one of those, the MBL comes up after upgrade as if nothing ever changed, samba shares and everything.
There's just one thing, though:
The "nondestructive" sysupgrade fails with a GPT partition layout, it only works correctly with MBR layouts. When run on a GPT layout, it converts the disk back to MBR, and any partitions beyond the MBR sector limit of 4294967295 get cropped to 4294967294. Which effectively means that 2 TiB is the limit.
Snapshot also does not have gdisk as a package anymore, maybe because fdisk is now able to deal with GPT and was deemed a suitable replacement? Unfortunately, fdisk is not able to convert MBR to GPT in the same way as gdisk.
So handling disks larger than 2 TB is turning out to be a major source of pain in the posterior.
Pinging @chunkeey once again, do you think there's anything that can be done about that?
Nah, UEFI x86 will sooner or later be in the same boat. So, why not wait until they can cook up a good solution? GPT is finicky to deal with due to multiple headers (current + backup), versioning, checksums and indirections.
In the mean time, people have told me that there's a way to do it via "Hybrid MBR" route. This way, there's a "valid-looking" MBR that does work with the existing sysupgrade scripts and have a GPT that allows more than 4 partitions and can deal with >2TB+ HDDs properly.
The easiest way to do this is to disassemble the MBL and connect the HDD to a PC (or get the old gdisk package from lede-17.01 packages). Start gdisk there on the HDD and use the "r = recovery and transformation options (experts only)" menu to do a "h = make hybrid MBR". When gdisk asks about the sequence of GPT Partitions that should be added to the Hybrid MBR. Just add the "boot and rootfs" partitions (1 2). Next Say 'N' to "Place EFI GPT (0xEE) partition first in MBR (good for GRUB)? (Y/N):". As this would mess up the partition order and the MBL would no longer boot. From there on forward just follow through the process (the default gdisk provides should work fine).
That said, I never tested this due to a severe lack of 3TB+ HDDs.
@tmomas: really?
Guys, this thread became offtopic in November last year.
If you want to discuss topics different to the one of this thread, please create a new topic in a suitable category.
Since the topic is already marked as solved, I will close it now.