I've tried flashing from multiple places and it did not work out
Ultimately I tried flashing my working firmware to firmware2 partition within my working initramfs image with mtd and I "wiped" the device's official firmware.
The only good part is that now the system loads into U-Boot directly, so I can go into initramfs without having to press a ton of keys, so I can test way faster.
I don't understand what I shall need to create a booting firmware if initramfs image is already working very well.
bootnum =2
bootnum =2
Checking FW2 combo magic and checksum ...Attempt to read outside the flash area
Fail.
Default combo magic is 5a4e4554, FW2 combo magic is 73797375
Checking FW1 combo magic and checksum ...Attempt to read outside the flash area
Fail.
Default combo magic is 5a4e4554, FW1 combo magic is 73797375
******ALL image are broken******
Loading from nand0, offset 0x400000
** Unknown image type
=>
Checking in Google this "combo magic" thing must be very specific from this ZyXEL device because there are no other references... Not 100% sure what is trying to check. Checking other devices I find something in common: all add some sort of "header" to the firmware which probably is what checksums to pass this ZyXEL control. To analyze such, I probably may need an official firmware to compare their headers. I may try to ask other devs to see how did they find such headers.
I asked 1 week ago to ZyXEL an official firmware, just in case they send me I would flash it straight away to see if it actually passes the possible controls they are setting there.
We may assume that by flashing the bootloader, Annick was able to remove this control somehow.
I still can boot my working initramfs image to keep using OpenWRT but now it doesn't load anything by default as a router.
Now I'm in a dead end for this device, again
PS: For example Linsys E7350 (Belkin RT1800) also had this @hecatae any ideas on this topic?
I saw you commented something related to GND, same happens to me, the only way to boot UART is by inserting the serial in the PC USB port after i plug in the device or without the GND pin to insert it after (both ways are valid). So find many commonalities in this aspect.
If you check the hex dump of a valid partition (e.g. RAS1), you can see that it starts with exactly these bytes: 0x5A 0x4E 0x45 0X54. Then there is some checksum (or whatever it is) and the version information. The actual image starts at offset 0x40000. So now you'll have to figure how this header is created - and then your firmwareupgrade from the OEM GUI will probably also work.
Yes, that is already working. Now I'm trying to flash a working factory build based on that image. The problem is that, as I said, there is a CRC check during bootup. I saw that your E7350 also had that CRC verification. How did you find the solution for this?
Anything that you can find Have a look at zytrx.c, it's in firmware-utils. It creates the header for two other ZyXEL devices, it's similar but not identical. If I'm not mistaken, this tool was written by @bmork, maybe he can shed some light in how he discovered the required bits?
Then have a look at the OEM rootfs: There are several scripts, notably /lib/upgrade/platform.sh, that check for this 0x5a4e4554 magic.
Your best bet is probably the U-Boot code, I suppose you haven't received a reply from ZyXEL yet?
And the relevant code that actually performs the upgrade (although I'm unsure if that is really called or if they just use default_do_upgrade instead):
mstc_do_upgrade() {
BOOT_NUM=`/sbin/mstc_persist read bootnum`
if [ "1" == "1" ]; then #If the uImage type is FIT.
if [ "$BOOT_NUM" -eq 2 ]; then
mtd erase RAS1
mtd mstcwrite $1 RAS1
fw_up_rslt=$?
echo "### Write into RAS1 ###" > /dev/console
elif [ "$BOOT_NUM" -eq 1 ]; then
mtd erase RAS2
mtd mstcwrite $1 RAS2
fw_up_rslt=$?
echo "### Write to RAS2 ###" > /dev/console
fi
else #else uImage type is Legacy.
mtd erase firmware
mtd mstcwrite $1 firmware
fw_up_rslt=$?
fi
if [ "$fw_up_rslt" != 0 ]; then
echo "FW update failed"
# turn on power led after FW upgrade failed
if [ -f /sbin/mstc_led.sh ]; then
if [ "0" == "1" ]; then
mode=`uci get system.main.mode 2>/dev/null`
if [ "$mode" == "router" ]; then
/sbin/mstc_led.sh led_status_done_router multiple
elif [ "$mode" == "ap" ]; then
/sbin/mstc_led.sh led_status_done_ap multiple
elif [ "$mode" == "repeater" ]; then
repeater_mode=`uci get system.main.repeater 2>/dev/null`
if [ "$repeater_mode" == "normal" ]; then
/sbin/mstc_led.sh led_status_done_repeater_normal multiple
elif [ "$repeater_mode" == "hanareya" ]; then
/sbin/mstc_led.sh led_status_done_repeater_hanareya multiple
fi
fi
else
/sbin/mstc_led.sh power on
fi
else
power_gpio=`cat /tmp/gpioinfo | grep gpio_num | awk -F'=' '{print $2}'`
power_active_low=`cat /tmp/gpioinfo | grep active_low | awk -F'=' '{print $2}'`
/tmp/gpio l $power_gpio 1 0 0 0 0 $power_active_low
fi
rm -f $1
exit 1;
else
if [ "1" == "1" ]; then #If the dual image has been enabled.
if [ "$BOOT_NUM" -eq 2 ]; then
/sbin/mstc_persist write bootnum 1
echo "### Switch to bank 1 on next boot! ###" > /dev/console
fi
if [ "$BOOT_NUM" -eq 1 ]; then
/sbin/mstc_persist write bootnum 2
echo "### Switch to bank 2 on next boot! ###" > /dev/console
fi
fi
fi
# make pppoe sending terminate request to server
if [ "0" == "1" ]; then
ubus call network.interface.wan down
else
/bin/busybox killall pppd
fi
sleep 1
reboot -f
}
So basically, they are using the standard sysupgrade from OpenWrt with an additional check for the magic in the first four bytes - the same four bytes, that U-Boot checks as well. And you can see the dual image support ...
There might be additional checks imposed by the process handling the uploaded files - I haven't found the respective file yet (some zapi stuff).
Either you didn't read my message are you didn't understand it. There is much more in the header that needs to be understood before U-Boot will accept it. This will only get you rid of the "invalid magic" error, but the checksum still won't match.
Ok, I understand this. The problem here is how I could research this. I wonder how other developers did such research. I will not expect to receive U-Boot source code from this routers code ever, at most, their DTS
Do you think those, all devs, received such source code? I doubt so. They might have reversed engineered somehow.
ZyXEL usually provides the GPL dump, including U-Boot code - at least, they did so for the Realtek-based switches. Just give them some time. My GPL request was answered after a few hours with "we need some time to check your request".
IDK.
You can try to figure out what the fields mean by playing around a bit. Usually, there is some sort of CRC32 involved and the length of the image is usually also encoded. Maybe the load address. Compute your own numbers based on the stock firmware, compare them. According to zytrx.c, they often invert the bits (e.g. for the CRC32).
I looked at the OEM images and tried to create a similar header. The magic ("HDR0") identified it as TRX, but that didn't quite fit. I used that as a starter and added fields to make it fit. Started out with most of them as unknown. Then it was just guessing and trying.
Some of the strings are obvious, and often not validated so it doesn't really matter. But of course, validation can differ between bootloader and OEM firmware updater. Ideally we want to create an images which passes both so that it can be used to install directly from OEM. So this has to be tested.
Size fields are also pretty easy. Just get yourself a list of possible sizes (header, total, data, etc) as 32bit hex numbers and see if you find anything matching. Or maybe close to matching, which could happen of the size is calculated with some offset you didn't think of. Like a tralier not being part of the "data" sections etc.
Then there are checksums. A number of them. This is harder, although you can get a few hints from existing tools. First you'll have to identify the fields of course. Maching 32bit values against the output of the bootloader is the safest way. But in general, you can assume that anything that looks like an arbitrary 32bit number is a checksum. The exception is a maigic value. This is easy to spot if you compare multiple OEM firmware versions. The magic will be constant.
As you'll see in the zytrx.c, I never figured out how to calculate the "kernelChksum" field. I got this field name from the bootloader, which reports it. But it turns out it isn't validated in any way so I just ignored it. As log as it works, then I'm happy
I hacked up the tool below from add_header.c while trying to figure this out. It simplifies the task of calculating checksums over different byte ranges in different formats. This is how I found the other two checksum algorithms zytrx.c
Good luck!
crc32range.c
/*
* add_header.c - partially based on OpenWrt's motorola-bin.c
*
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
* Gabor Juhos <juhosg@openwrt.org>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License,
* version 2 as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*/
/*
* The add_header utility used by various vendors preprends the buf
* image with a header containing a CRC32 value which is generated for the
* model id + reserved space for CRC32 + buf, then replaces the reserved
* area with the actual CRC32. This replacement tool mimics this behavior.
*/
#include <stdio.h>
#include <stdlib.h>
#include <stddef.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <string.h>
#include <netinet/in.h>
#include <inttypes.h>
#define BPB 8 /* bits/byte */
static uint32_t crc32[1<<BPB];
static void init_crc32()
{
const uint32_t poly = ntohl(0x2083b8ed);
int n;
for (n = 0; n < 1<<BPB; n++) {
uint32_t crc = n;
int bit;
for (bit = 0; bit < BPB; bit++)
crc = (crc & 1) ? (poly ^ (crc >> 1)) : (crc >> 1);
crc32[n] = crc;
}
}
static uint32_t crc32buf(const unsigned char *buf, size_t len)
{
uint32_t crc = 0xFFFFFFFF;
for (; len; len--, buf++)
crc = crc32[(uint8_t)crc ^ *buf] ^ (crc >> BPB);
return ~crc;
}
static void usage(const char *mess)
{
fprintf(stderr, "Error: %s\n", mess);
fprintf(stderr, "Usage: crc32range input_file from to crcoffset\n");
fprintf(stderr, "\n");
exit(1);
}
static void errexit(const char *msg)
{
fprintf(stderr, "ERR: %s: %s\n", msg, strerror(errno));
exit(1);
}
int main(int argc, char **argv)
{
off_t len; // of original buf
int i, fd;
long from = -1, to = -1, offset = -1;
void *input_file; // pointer to the input file (mmmapped)
uint32_t crc, *p;
struct stat stat;
if (argc < 2)
usage("wrong number of arguments");
errno = 0;
if (argc > 2)
from = strtol(argv[2], NULL, 0);
if (argc > 3)
to = strtol(argv[3], NULL, 0);
if (errno)
errexit("invalid input");
// mmap input_file
fd = open(argv[1], O_RDONLY);
if (fd < 0)
errexit(argv[1]);
// fprintf(stderr, "opened '%s' fd=%d\n", argv[1], fd);
if (fstat(fd, &stat) < 0)
errexit("stat");
len = stat.st_size;
input_file = mmap(NULL, len, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
if (close(fd) < 0)
errexit("close");
init_crc32();
if (from < 0 || from > len)
from = 0;
if (to < from)
to = len;
/* clear crc value */
for (i = 4; i < argc; i++) {
offset = strtol(argv[i], NULL, 0);
if (errno)
errexit("invalid input");
if (offset >= from && offset <= to - 4) {
p = input_file + offset;
fprintf(stderr, "old checksum at offset %d: %08x\n", offset, htonl(*p));
*p = 0;
}
}
//fprintf(stderr, "here from=%d, to=%d, len=%d, fd=%d\n", from, to, len, fd);
// CRC of temporary header + buf
crc = crc32buf(input_file + from, to - from);
fprintf(stderr, "%08x\t%08x\t%08x\t%08x\n", crc, htonl(crc), ~crc, htonl(~crc));
munmap(input_file, len);
return 0;
}
Well, 5a4e4554 is the string "ZNET" (or "TENZ" depending on how you look at the endianness). While 73797375 is "sysu", which looks like an OpenWrt sysupgrade tar file (starts with the string "sysupgrade-..." since it contains a directory with that name).
I don't thnk it's likely that an OEM bootlader will accept a tar file...
Doesn't look like we have any the device using that "ZNET" magic. At least I can't find any match in existing tools. But guessing the contents of a header isn't that complicated really. Pick apart an OEM firmware (or preferably two or more different versions) and see if you can make a tool which can reproduce the same headers.
Thanks, I'm already on it and I'm starting to understand a few bits Unfortunately, I only have the partition I dumped from my device, as there is no firmware download available.
My firmware has the following header which I'm trying to structure:
uint32_t magic = 0x5a4e4554
uint32_t jffs2_offset = 04004c01 (the firmware has the JFFS2 marker 0xdeadc0de at this location, when the header is stripped)
uint32_t?
uint32_t fdt_checksum = 926ef4a1 (it's from the start of the firmware, i.e. at 0x40000 until the end of the image as reported by the first embedded FDT, i.e. 0x36e1d4 in my case)
uint32_t?
char[28] (+null-terminator) firmware version string; it's maybe split into other parts?
Now I'm probably missing some lengths and checksums.
Ok, so here we have arrived to a dead end, because there are no firmwares available publicly
I will need to go back to the sniffing part to see if I can pick the firmware from their update servers. I asked ZyXEL to see if they can provide me a firmware, but nothing answered yet.
Don't give up so quickly! See my post above, I'm only missing two fields and I'm not sure if they are even required ... and it took me less than an hour to discover them, thanks to @bmork's great tool, a hex editor and a calculator!
Interersting! I'm checking the dump I also have for firmware2, and as you say, the ZNET is there in the beginning. I'm going to try flashing this through MTD this evening to see if it's fits again.
Then A TON of FF and then
If we compare to my current OpenWRT build
Basically what I see is:
5A 4E 45 54 04 00 4C 01 and then
90
the that 0 that I'm not sure what means
And then
The Firmware version
A couple of 00 00 00 00 00 00
And then 1 million FF
This is the difference. I think I can get the previous firmware (the ABZF.4) from another router I own to dump and compare.
I've checked my firmware1 dump and.... it's the V2 of the firmware!!
Very good news
The format for the first block is identical
And for the version is also identical
The Only tricky part is this one:
The million FF start in 00030 and end in 3FFFF in both firmwares
PS: @Annick now I know who is PTPT I don't think he could help us here, because basically what they did with x-wrt, is using an alternative custom bootloader (which you got lucky it worked and did not seriously completely bricked your device without recovery), probably to bypass what we are trying to solve here right now.