Mikrotik RouterOS v7.x and OpenWrt sysupgrade

wwortel · January 11, 2023, 3:51pm

Since some time a warning is given in the general Mikrotik/OpenWrt instructions to downgrade to a v6 ROS before attempting a sysupgrade.
Of course it would be better to figure out what is wrong and correct that.
Would like to share some experiences so far and perhaps somebody has an idea to overcome the last hurdle.
Tests here: Mikrotik SXTsq-5ac upgraded to ROS 7.7rc5 and OpenWrt Dec. 2022 home compile r21516+5-1b8c8864c5.
What works:
100% OK: loading and booting kernel.bin image via bootp.
almost OK: sysupgrade of sysupgrade.bin image
Problem: after the flashing has been done the device reboots but does not load the fresh image from flash, but tries to get bootp going again.
When I press the reset button after the flashing has completed and the device is making its bootp requests, this is seen by the bootloader and it then boots the image from flash without issues.
However, a soft or hard reboot gets the endless bootp calls going again.
When logged into the flashed image, this is what the firmware soft_config show:
bios_version: 7.7rc5
boot_delay: 1 [2] 3 4 5 6 7 8 9
boot_device: eth [flasheth] ethonce flash cfg cfgonce
boot_key : [any] del
boot_proto: [bootp] dhcp
booter : [regular] backup
commit: clean
cpu_mode: [powersave] regular
silent_boot: [off] on
uart_speed: off 1200 2400 4800 9600 19200 38400 57600 [115200]

Any ideas how to get the bootloader to load from flash without having to use the reset switch?

NPeca75 · January 11, 2023, 5:16pm

hi @wwortel

if i remember right, when MT device boot with reset switch, it will load [backup] boot loader

so, looks like, in your case, primary boot loader does not work, but secondary [backup] boot loader does

somewhere in forum was discussed this issue with "new" v7.xx bootloaders

wwortel · January 11, 2023, 7:18pm

€Npeca75
Thanks for explanation of the effect of pressing reset.
Noticed in /sys/firmware/mikrotik/soft_config that the 'commit' entry cannot be written.
Although ls gives 'rw' as rights, the error thrown is 'Read-only file system'.
Read in another thread that one should set boot_device to cfg, followed by 1 > commit, before sysupgrade. But the configuration appears read only ....

NPeca75 · January 11, 2023, 8:53pm

@wwortel

i am sorry i could not help more
last time when i messed with MT+OpenWRT it was something like

insmod mtd-rw
read mtdX data
change bits
erase mtdX partition
write new data

but... it was years ago with rb750/rb760 and wanted to enable serial console bit

so, hope somebody smarter than me will be more helpful

johnth · January 12, 2023, 10:09am

RouterOS version does not matter. RouterBOOT does (except for ipq40xx NAND devices)

yes, you remembered right, that is it. It is in the Mikrotik manual page for each device I have seen.

The sysfs file only asks the driver to do something. To change anything it needs writable at the partition level cat /sys/class/mtd/$(cat /proc/mtd | grep soft_config | cut -d: -f1 )/flags, and in all the parent partitions of soft_config, then needs to be able to erase that sector without overflowing the partition.

Only if you have RouterOS installed and want this? https://help.mikrotik.com/docs/display/ROS/Flashfig

If you (or anyone else) wants to dig into this: Generally:
There are two version of the bootloader in the parent RouterBOOT partition. They are stored on NOR compressed with UCL NRV2B (there, they are the fwf file, less headers, plus booter-selection and decompression preamble for backup booter). The device boot starts at the backup booter preamble, then if the soft_config tag for backup is set, or the reset button was held, the backup booter is decompressed and loaded, otherwise the primary booter.

Either dump from the NOR, or the FWF files (for updates) are stored in the RouterOS NPK squashfs under /etc/*.fwf One for each processor (or family), L- fwfs are for devices without NAND.

Either decompress and decompile one of these, to work out what RouterBOOT v7 is looking for, or look through a RouterOSv7 NOR flash dump, and start to work it out.

robimarko · January 12, 2023, 1:57pm

MTik completely changed the storage format, now they have some kind of a summary header with various device info and this completely breaks OpenWrt support for v7 bootloaders

wwortel · January 12, 2023, 3:04pm

thank you all for the enlightning info!
Can confirm that the problem does not occur with a Mikrotik ipq4019 nand + spi-nor device. Did a successful sysupgrade on a recent (Dec. 2022) hAPac3 that ex-factory came with v7.5; in retrospect I was lucky since I had not yet seen the generic v7 warning. It reboots quickly and does not get caught in these endless bootp tries.
@robimarko
after booting OpenWrt via bootp, is there a way to erase mtd1 "RouterBoot" v7 and write a v6 mtd1 instead?
It would be quicker than downgrading to v6 within ROS.

robimarko · January 12, 2023, 3:19pm

I am not sure, but I think that not directly as the partition will contain more than just RouterBoot

johnth · January 12, 2023, 8:00pm

Sounds like their NPK package format? Magic is 1E F1 D0 BA, which is in all the decompressed ipq4000L fwfs I have checked.

These are different, because they use UBI, so we don't need to use https://github.com/adron-s/kernel2minor. My ipq40xx NAND device worked fine with an early v7 RouterBOOT when I checked.

Not especially safe, but you could. Get it wrong, and you have a brick until you manually flash the NOR chip (SOIC8 clip handy). Take full device dumps first…
mtd1 "RouterBoot" contains: backup bootloader, hard_config, dtb (not on every platform), bios??, primary bootloader, soft_config. The order changes on different platforms and device families. You only want to replace primary bootloader. On the ipq40xx devices I have looked at, primary bootloader is at <0x30000 0xd000> within the RouterBOOT partition. You will need to cut some of the FWF file header to match what is flashed to NOR. My notes for ipq40xx said fwf upgrade file has a 0x20 header which is cut, then what follows is stored on NOR. Flashed primary booter starts with the compressed length, uncompressed length, and the last bytes are a CRC32. Don't trust this though, take a full dump, find the primary booter version in soft_config, download the same NPK version of RouterOS, extract the FWFs (binwalk handy), and make sure you can get the bytes to match.

More about the boot process and how these files can be packed / unpacked: For rb5009, Sergey replaces the primary bootloader with a modified version able to load FIT images. More details on their repos (but the commands and partition numbers are specific to their images and that device): https://github.com/adron-s/mtik_routerboot_encdec https://github.com/adron-s/aux-loader2

I have wanted to compress u-boot (or u-boot spl, or jump-to-uboot-location assembly) with nrv2b, add the headers and CRC32 tail, and use that as primary bootloader, but have not got around to it yet.

robimarko · January 12, 2023, 8:02pm

Not just the NPK, but rootfs is actually offset on the RouterOS partition and there is like a device/firmware info header at the start.

There was a dump somewhere in the from ROS v7 but I cannot find it now

wwortel · January 12, 2023, 9:15pm

@johnth thanks for the extensive explanation. This 'SXTsq 5 ac' has the complication that the PCB is not accessible and the encasing appears a thermally welded plastic sandwich. Not meant to be ever opened in a reversible way. So a brick will indeed remain a brick. Will not attempt this now.
Will further study what is in mtd1. With nrv2b am getting 'segmentation fault' but perhaps I am not feeding it the proper segment out of mtd1.

johnth · January 12, 2023, 9:44pm

That would be nice. Eventually I will commandeer of my in-use ipq40xx NOR devices, netinstall v7, and dump. Might try to root as well to see loaded partitions / files layout.

yes, it happens. Try a different offset. I would not be surprised if the rb5009 fwf headers are different to ipq40xx. On ipq40xx, need to cut at least the two long length headers from the primary booter read from flash.

johnth · January 18, 2023, 3:57am

I had a dig on my mmips 760igs. Wall of text follows.

TLDR:
Might be getting there…
On RouterBOOT booter 6.46.5:
I can modify an NPK zlib-packed data section, and replace boot/kernel in it with an OpenWrt kernel, and repack it. Format detailed here: https://github.com/kost/mikrotik-npk/blob/master/createnpk.py, but the system NPK zlib-packed data is uncompressed, with a 0x8000 block size (python script for this below)
I can then modify an NPK (https://github.com/botlabsDev/npkpy) to use my updated NPK zlib-file-container. I can reduce the squashfs section of the NPK, but that section data size needs to be a factor of 0x1000 long.
I can then pack that NPK into bootimage (instead of kernel) with modified kernel2minor.
I can then BOOTP normal OpenWrt initramfs image, erase firmware, write this very packed image to firmware, and RouterBOOT did boot it. Success.
Now need to clean this up, find minimal example, and build the tools to do the packing.

Some parts of how I got there:

On 6.43.12 (for cleaner_wrasse), the yaffs RouterOS partition has:

ls /flash/
boot        bootimage   etc         lost+found  nova        rw          var

ls -altrh /flash/var/pdb/routeros-mmips/
total 3K     
lrwxrwxrwx    1 root     root          16 Jan  1 00:04 image -> /flash/bootimage

sha256sum /flash/bootimage 
9a5a6ecfeda82c42a2b9da163d5190d7b560b60427f694c252f6d35e0e63b8be  /flash/bootimage

#downloaded file
sha256sum mikrotik/routeros-mmips-6.43.12.npk 
9a5a6ecfeda82c42a2b9da163d5190d7b560b60427f694c252f6d35e0e63b8be  mikrotik/routeros-mmips-6.43.12.npk

Yup, that's the NPK, as downloaded from Mikrotik, flashed to /bootimage on the yaffs filesystem.

Found that I had an old 7.1b3 dump (I think I took this immediately after netinstall, before first RouterOS boot), and walked through the YAFFS object headers & sparse data to see it has very nearly the same layout.

zzz YAFFS walking. could not find a tool that just worked to list or extract

The skips to show the YAFFS object headers are specific to this file, and were done manually here.
Didn't show any first, then noted where a new objectID was listed in the sparse data, and showed the object header prior to this new objectID. https://mattboyer.github.io/PYaffs/2014/08/12/Hacklog%234.html helped.

for i in $(seq 0 10700); do
offset=$((i*(0x400+0x10)+(i/63*0x10)-0x10))
#printf "sparse data at offset: 0x%x. iteration %d\n" $offset $i;
if [ $(( i % 63 )) -eq 0 ]; then
:
#printf " extra sparse data at: 0x%x\n" $(( offset ));
#hexdump -C -n 0x10 -s $(( offset-0x10 )) hexs_7b3-routeros-partition.img | head -n-1
fi
[ $i -gt 0 ] && \
hexdump -C -n 0x10 -s $(( offset )) hexs_7b3-routeros-partition.img | head -n-1 | tee -a yaffs-dump
if [ $i -eq 0 ] || [ $i -lt 8 ] || [ $i -gt 265 -a $i -lt 268 ] || [ $i -eq 10592 ] || [ $i -eq 10593 ] || [ $i -gt 10604 ]; then
hexdump -C -n 0x20 -s $(( offset + 0x10 )) hexs_7b3-routeros-partition.img | head -n-1 | tee -a yaffs-dump
fi
if [ $i -gt 10616 ]; then
hexdump -C -s $(( offset )) hexs_7b3-routeros-partition.img 
break
fi
done

00000000  03 00 00 00 01 00 00 00  ff ff 72 77 00 00 00 00  |..........rw....|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000400  01 10 00 00 01 01 00 30  01 00 00 80 00 00 00 00  |.......0........|
type,parent,name,yaffs objectID
dir, root, rw, objectid 1
00000410  03 00 00 00 01 00 00 00  ff ff 6e 6f 76 61 00 00  |..........nova..|
00000420  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000810  01 10 00 00 02 01 00 30  01 00 00 80 00 00 00 00  |.......0........|
dir, root, nova, objectid 2
00000820  03 00 00 00 02 01 00 00  ff ff 65 74 63 00 00 00  |..........etc...|
00000830  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000c20  01 10 00 00 03 01 00 30  02 01 00 80 00 00 00 00  |.......0........|
dir, nova, etc, objectid 3
00000c30  03 00 00 00 01 00 00 00  ff ff 6e 6f 76 61 00 00  |..........nova..|
00000c40  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001030  01 10 00 00 02 01 00 30  01 00 00 80 00 00 00 00  |.......0........|
dir, root, nova, objectid 2
00001040  03 00 00 00 00 00 00 00  ff ff 00 00 00 00 00 00  |................|
00001050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001440  01 10 00 00 01 00 00 30  00 00 00 80 00 00 00 00  |.......0........|

00001450  03 00 00 00 01 00 00 00  ff ff 76 61 72 00 00 00  |..........var...|
00001460  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001850  01 10 00 00 04 01 00 30  01 00 00 80 00 00 00 00  |.......0........|
dir, root, var, objectid 4
00001860  03 00 00 00 04 01 00 00  ff ff 70 64 62 00 00 00  |..........pdb...|
00001870  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001c60  01 10 00 00 05 01 00 30  04 01 00 80 00 00 00 00  |.......0........|
dir, var, pdb, objectid 5
00001c70  01 00 00 00 01 00 00 00  ff ff 69 6d 61 67 65 00  |..........image.|
00001c80  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00002070  01 10 00 00 06 01 00 10  01 00 00 80 00 00 00 00  |................|
file, root, image, objectid 6
00002480  01 10 00 00 06 01 00 00  01 00 00 00 00 04 00 00  |................|
00002890  01 10 00 00 06 01 00 00  02 00 00 00 00 04 00 00  |................|
00002ca0  01 10 00 00 06 01 00 00  03 00 00 00 00 04 00 00  |................|
000030b0  01 10 00 00 06 01 00 00  04 00 00 00 00 04 00 00  |................|
000034c0  01 10 00 00 06 01 00 00  05 00 00 00 00 04 00 00  |................|
000038d0  01 10 00 00 06 01 00 00  06 00 00 00 00 04 00 00  |................|
00003ce0  01 10 00 00 06 01 00 00  07 00 00 00 00 04 00 00  |................|
...NPK...
000438e0  03 00 00 00 01 00 00 00  ff ff 76 61 72 00 00 00  |..........var...|
000438f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00043ce0  05 10 00 00 04 01 00 30  01 00 00 80 00 00 00 00  |.......0........|
dir, root, var, objectid 4
00043cf0  03 00 00 00 00 00 00 00  ff ff 00 00 00 00 00 00  |................|
00043d00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000440f0  05 10 00 00 01 00 00 30  00 00 00 80 00 00 00 00  |.......0........|

...more of NPK... (object 6)
00a82070  a9 10 00 00 06 01 00 00  4f 29 00 00 00 04 00 00  |........O)......|

00a82080  01 00 00 00 01 00 00 00  ff ff 74 6d 70 52 74 67  |..........tmpRtg|
00a82090  32 71 73 00 00 00 00 00  00 00 00 00 00 00 00 00  |2qs.............|
00a82480  a9 10 00 00 07 01 00 10  01 00 00 80 00 00 00 00  |................|
file, root, tmpRtg2qs, object 7
00a82490  01 00 00 00 01 00 00 00  ff ff 74 6d 70 52 74 67  |..........tmpRtg|
00a824a0  32 71 73 00 00 00 00 00  00 00 00 00 00 00 00 00  |2qs.............|
00a82890  a9 10 00 00 07 01 00 10  01 00 00 80 00 00 00 00  |................|
file, root, tmpRtg2qs, object 7

00a82ca0  a9 10 00 00 06 01 00 00  53 29 00 00 00 04 00 00  |........S)......|
...more of NPK... (object 6)
00a85540  a9 10 00 00 06 01 00 10  01 00 00 80 48 63 a5 00  |............Hc..|

00a85550  03 00 00 00 05 01 00 00  ff ff 73 79 73 74 65 6d  |..........system|
00a85560  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a85950  a9 10 00 00 08 01 00 30  05 01 00 80 00 00 00 00  |.......0........|
dir, pdb, system, object 8
00a85960  02 00 00 00 08 01 00 00  ff ff 69 6d 61 67 65 00  |..........image.|
00a85970  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a85d60  a9 10 00 00 09 01 00 20  08 01 00 80 00 00 00 00  |....... ........|
symlink, system, image, object 9

00a85d70  01 00 00 00 01 00 00 00  ff ff 55 50 47 52 41 44  |..........UPGRAD|
00a85d80  45 44 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |ED..............|
00a86170  a9 10 00 00 07 01 00 10  01 00 00 80 00 00 00 00  |................|
file, root, UPGRADED, obj7

00a86180  01 00 00 00 01 00 00 00  ff ff 62 6f 6f 74 69 6d  |..........bootim|
00a86190  61 67 65 00 00 00 00 00  00 00 00 00 00 00 00 00  |age.............|
00a86580  a9 10 00 00 06 01 00 10  01 00 00 80 48 63 a5 00  |............Hc..|
file, root, bootimage, objectid 6

00a86590  01 00 00 00 01 00 00 00  ff ff 53 48 4f 57 5f 4c  |..........SHOW_L|
00a865a0  49 43 45 4e 53 45 00 00  00 00 00 00 00 00 00 00  |ICENSE..........|
00a86990  a9 10 00 00 0a 01 00 10  01 00 00 80 00 00 00 00  |................|
file, root, SHOW_LICENSE, objectid a

00a869a0  01 00 00 00 01 01 00 00  ff ff 52 45 42 4f 4f 54  |..........REBOOT|
00a869b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a86da0  a9 10 00 00 0b 01 00 10  01 01 00 80 00 00 00 00  |................|
file, root, REBOOT, objectid b

00a86db0  03 00 00 00 05 01 00 00  ff ff 73 79 73 74 65 6d  |..........system|
00a86dc0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a871b0  a9 10 00 00 08 01 00 30  05 01 00 80 00 00 00 00  |.......0........|
dir, pdb, system, object 8

00a871c0  03 00 00 00 04 01 00 00  ff ff 70 64 62 00 00 00  |..........pdb...|
00a871d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a875c0  a9 10 00 00 05 01 00 30  04 01 00 80 00 00 00 00  |.......0........|
dir, var, pdb, objectid 5

00a875d0  03 00 00 00 01 00 00 00  ff ff 72 77 00 00 00 00  |..........rw....|
00a875e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a879d0  a9 10 00 00 01 01 00 30  01 00 00 80 00 00 00 00  |.......0........|
dir, root, rw, objectid 1

00a879e0  03 00 00 00 00 00 00 00  ff ff 00 00 00 00 00 00  |................|
00a879f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00a87de0  a9 10 00 00 01 00 00 30  00 00 00 80 00 00 00 00  |.......0........|

00a87df0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00a881f0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
00a88200  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00a88600  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
00a88610  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00a88600  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00fc0000

7.1b3 also has NPK at /bootimage

kernel2minor currently packs our ELF to kernel, in a YAFFS file system, but searching through my Mikrotik RouterOS NOR dumps, I cannot find references to kernel, nor ELF magic that I cannot discard as something else, so this may be a very old method of booting

fn=hapac2_nor_dump.img
for offset in $(binwalk --raw='kernel' $fn | grep 'Raw' | awk '{print $2}'); do hexdump -C -n 0x100 -s $(( (offset >> 4) * 0x10 -0x20)) $fn ; done
for offset in $(binwalk --raw='\x7fELF' $fn | grep 'Raw' | awk '{print $2}'); do hexdump -C -n 0x100 -s $(( (offset >> 4) * 0x10 -0x20)) $fn ; done
#npk magic
for offset in $(binwalk --raw='\x1e\xf1\xd0\xba' $fn | grep 'Raw' | awk '{print $2}'); do hexdump -C -n 0x100 -s $(( (offset >> 4) * 0x10 -0x20)) $fn ; done

It seems the only references to kernel or ELF magic that I can find now are the ones packed into the NPK file-container section, which is a zlib packed (before I came across the zlib mention in npkpy, I had been stripping those chunks in my binwalk plugin… ouch) image of some sort. I have guessed a little about this file-container format.
Example, if we use routeros-7.7-arm.npk. Walk the NPK. The file-container (type 0004) NPK header is at 0xa45007, data at 0xa4500d, size of 0x26ca17.

dd if=routeros-7.7-arm.npk of=routeros-7.7-arm.npk-file-container bs=1 count=$((0x26ca17)) skip=$((0xa4500d))
printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - routeros-7.7-arm.npk-file-container | gzip -dc>routeros-7.7-arm.npk-file-container-uncompressed

Then this file-container, after zlib unpack, format has

short i_mode (81ed:file,rwx,rx,rx, 81a4:file,rw,r,r, 41ed:dir,rwx,rx,rx): 
00 00 00 00 00 00 (possibly flags, uid, gid)
long ctime
long ?relatime
long mtime
00 00 00 00
long object_size
short name_size
name[name_size]
object_data
[, next i_mode… ]

That gives us:

#offset,len,name
0x26,0x3274,bin/bash
0x32bc,0,boot
0x32e5,0x269560,boot/kernel
0x26c86b,0x20,UPGRADED
# all with ctime: 2023-01-12 17:35:45

After working this out myself, found it listed in createnpk…

I have a few horrible quality binwalk plugins I have cobbled together. If someone wants to keep poking, let me know I can put up my files.

I have not yet been able to get Ghidra to map the strings to functions in an nrv2b decompressed RouterBOOT, so I have not got far going from the other end. Wild guess that RouterBoot finds the NPK (through YAFFS /bootimage), walks to the file-container, then loads bin/bash (which is definitely not the bash shell), or boot/kernel from there.

In RouterOS 6.43.12 root shell, I do some in-place adjustments to this file-container of the NPK in yaffs. This will invalidate the adler32 at the end of the compressed file-container, as well as the NPK hash and signature.

overwrite just the \x7fELF bytes header of bin/bash: dd if=/dev/zero of=/flash/bootimage bs=1 count=4 conv=notrunc seek=$((0x7a6060)), or rename UPGRADED: printf "\x41\x41\x41\x41" | dd of=/flash/bootimage bs=1 count=4 conv=notrunc seek=$((0x7a6035))

Press any key within 2 seconds to enter setup..

loading kernel... OK
setting up elf image... OK
jumping to kernel code
ERROR: no system package found!
Kernel panic - not syncing: Attempted to kill init!
Rebooting in 1 seconds..

Looks like RouterBOOT loaded the boot ELF fine, but it could not load the NPK (system package).
The same happens if I cut the bash, or both the bash and UPGRADED file from this container:

#cut bin/bash from 6.43.12:
dd if=routeros-mmips-6.43.12.npk-file-container-extracted bs=1 count=$((0x26)) of=routeros-mmips-6.43.12.npk-file-container-extracted-upgraded
dd if=routeros-mmips-6.43.12.npk-file-container-extracted bs=1 skip=$((0x26+2+2+4+12+4+0x879c+4+0x8+2)) of=routeros-mmips-6.43.12.npk-file-container-extracted-kernel

cat routeros-mmips-6.43.12.npk-file-container-extracted-upgraded routeros-mmips-6.43.12.npk-file-container-extracted-kernel >routeros-mmips-6.43.12.npk-file-container-extracted-nobash


cat <<EOF>pack-mikrotik-file-container.py
import sys
import zlib
import struct
if (len(sys.argv) < 2):
  print ("require filename-to-compress as arg1")
  exit(1)
fn=sys.argv[1]
fp=open(fn,"rb")

fp.seek(0)

blocksize=0x8000
buffer_in=fp.read(blocksize)
buffer_out=b"\x78\x01"
while (buffer_in):
  if (len(buffer_in) == blocksize):
    # otherwise compress marks as final block (78 01 01 + 4 byte ADLER tail)
    block=b"\x00\x00\x80\xff\x7f"
    compressed=zlib.compress(buffer_in, level=0)
    block+=compressed[7:-4]
  #print("offset:0x{:x}".format(fp.tell()))
  else:
    compressed=zlib.compress(buffer_in, level=0)
    block=compressed[2:-4]
  buffer_in=fp.read(0x8000)
  if (len(buffer_out) == 2):
    adler32=zlib.adler32(compressed[7:-4])
  else:
    adler32=zlib.adler32(compressed[7:-4],adler32)
  buffer_out+=block


#adler32=zlib.adler32(buffer_out)
buffer_out+=struct.pack(">L",adler32)
#buffer_out[:0x10].hex(" ")


fo="{}.compressed".format(fn)
fop=open(fo,"wb")
fop.write(buffer_out)
fop.close()

fp.close()
EOF

pack-mikrotik-file-container.py routeros-mmips-6.43.12.npk-file-container-extracted-nobash

#scp to RouterOS

hexdump -C -n $((0x60)) -s $((0x7A600D)) /flash/bootimage
dd if=/ram/routeros-mmips-6.43.12.npk-file-container-extracted-nobash.compressed bs=1 seek=$((0x7A600D)) conv=notrunc of=/flash/bootimage

Netinstall again, then the same with boot/kernel: dd if=/dev/zero of=/flash/bootimage bs=1 count=4 conv=notrunc seek=$((0x7ae84c)):

Press any key within 2 seconds to enter setup..

loading kernel... OK
setting up elf image... not an elf header
kernel loading failed

Looks like this file is what RouterBOOT tries to load.

kill the NPK header of boot/kernel dd if=/dev/zero of=/flash/bootimage bs=1 count=4 conv=notrunc

Press any key within 2 seconds to enter setup..

loading kernel... bad image header
 kernel not found

Does look like RouterBOOT v6.46.5 is loading the yaffs /bootimage (NPK file), section 4 (file container), boot/kernel

I replaced boot/kernel (in all it's packing) (left boot dir) in-place in bootimage with an OpenWrt kernel:

loading kernel... OK
setting up elf image... not an elf header
kernel loading failed

Then started packing the NPK with kernel2minor and flashing from OpenWrt initramfs to reduce recycle time.

ALL 7.x have the same order of: (7.7)

NPK item header: at 0x0      type:81ed ELF  size 0x335c   name: bin/bash (unzlib) data offset 0x26 ctime: 2023-01-12 17:35:45 mtime: 2023-01-12 17:39:45
NPK item header: at 0x3382   type:41ed dir  size 0x0      name: boot (unzlib) data offset 0x33a4 ctime: 2023-01-12 17:35:45 mtime: 2023-01-12 18:16:28
NPK item header: at 0x33a4   type:81ed ELF  size 0x18e9e0 name: boot/kernel (unzlib) data offset 0x33cd ctime: 2023-01-12 17:35:45 mtime: 2023-01-12 17:59:50
NPK item header: at 0x191dad type:81a4 file size 0x20     name: UPGRADED (unzlib) data offset 0x191dd3 ctime: 2023-01-12 17:35:45 mtime: 2023-01-12 18:16:41

7.7 decompressed zlib-container
copy out the start, including bin/bash, boot, and boot/kernel (header)
append the openwrt kernel
append the 7.7 UPGRADED tail
set the boot/kernel size

dd if=routeros-7.7-mmips.npk-file-container-extracted bs=1 count=$((0x33cd)) | cat - mikrotik_routerboard-760igs-kernel.bin  | cat - routeros-7.7-mmips.npk-file-container-extracted-tail-upgraded > mikrotik_routerboard-760igs-kernel.bin.packed

printf "0x%x\n" $(stat -c%s ../../mikrotik_routerboard-760igs-kernel.bin)
0x30f7e0

printf "\xe0\xf7\x30\x00" | dd bs=1 conv=notrunc seek=$((0x33bc)) of=mikrotik_routerboard-760igs-kernel.bin.packed

python ../pack-mikrotik-file-container.py mikrotik_routerboard-760igs-kernel.bin.packed

If I tried to modify NPK squashfs (unused for us) to null, or anything short of 0x1000, I got a loading kernel... bad section header error.

wwortel · January 18, 2023, 9:36am

impressed! Will be glad to give the beta version a try.

johnth · January 22, 2023, 10:08pm

minimised the NPK down to:

squashfs section, which must have a payload \x00 works well
zlib-file-container section, which must have dir boot, file boot/kernel, file UPGRADED

Demo branch for RouterBOOT v7 NOR openwrt sysupgrade here: https://github.com/john-tho/openwrt/tree/routerboot-v7
Which uses a framework for NPK testing & zlib-container packing here: https://github.com/john-tho/npkpy/tree/create-npk
Could pack the zlib-container and NPK without using the python framework, but task for someone else, or another day.

Have not tested backwards, to see which RouterBOOT v6 loaders are compatible.

Various RouterBOOT error messages in testing:

loading kernel... bad image header #bootimage? NPK?
loading kernel... bad section header #no squashfs payload
loading kernel... kernel not found or data is corrupted #no squashfs section
setting up elf image... not an elf header #no UPGRADED

Only tested on my ramips device for now. Should be able to make RouterBOOTv7 only NOR devices work if this carries through to the ARM devices as well.

This was with RouterBOOT v7.7
RouterBOOT backup booter 6.42.4 will not load this: loading kernel... bad section header, so I think I will need to pad out the squashfs at least

updated with squashfs payload starting and ending on 0x1000 boundary, this boots on both RouterBOOT backup booter 6.42.4 & RouterBOOT 7.7

dchard · January 23, 2023, 2:00pm

Hi John.

I am working on the Chateau 12 device support, and have exactly the same issue as you: the device is ROS7 only, 16MB NOR, initramfs on routerboot 7.2 works, but I cant sysupgrade my image to the device. This is ARMv7 based, but the NOR part seems to be exactly the same. I will try to pull your latest mods and see if it works for me. In the mean time, if you need the complete factory dump of this device's flash, I can send it to you.

MOD: your method does seem to work on the Chateau LTE12 device as well. Just netbooted an initramfs image (this part did worked before), then sysupgraded my device and Openwrt boots correctly. The only thing not needed was this patch: https://github.com/john-tho/openwrt/commit/f5029749cf0f46e0955f48962c9402cd75bafd68

So it seems that the ipq40xx target can also benefit greatly from this.

dchard · January 23, 2023, 4:53pm

@johnth one more thing: it seems MikroTik changed the hard_config compression algo at least for this device:

Left is Hap ac3 lte6 kit (ROS6), right is Chateau 12 (ROS7).

I am not sure if this is general ROS7, or only this device. Non the less, due to this, the wifi board file is not loading. Robi recommended to share this with you, maybe the platform driver needs some tweaking? The new scheme is LZ77

MOD: I checked the "hard_config" part of my Hap Ac3 Lte6 Kit with ROS7 + Routerboot6, and with ROS7 + Routerboot7 and it seems the scheme stayed the same (LZOR) before and after the upgrade. So it seems this LZ77 thing is not a general change, although it might be for the "ROS7 only" devices, like the Chateau 12.

wwortel · January 23, 2023, 6:13pm

could check a recent hAPac3 (just router, no lte) with ROS7.5 ex-factory. It uses the LZOR.

dchard · January 23, 2023, 6:15pm

Yeah it seems like they do not change it for devices released with ROS6, but this Chateau 12 is already using LZ77. If someone has access to other ROS7-only devices, it might worth to check. Non the less, it seems we need LZ77 support as well.

johnth · January 23, 2023, 6:54pm

Terrific.

I remember seeing this in an extracted NPK, but never looked into it. If you want to have a crack, try binwalk -Me routeros*npk, then load _routeros*npk.extracted/squashfs-root/lib/modules*/misc/flash.ko into ghidra or your preferred decompiler. At least 7.7-arm has some pretty strings for pointers:
000161fd "flash ERROR: radio data lz77 decompress failed: %d\n" ds
00016653 "lz77_compress verify ERROR: orig len %d, new len %d\n" ds
It looks like there is not a magic header like in the lzo decompress.

Yes, in https://github.com/openwrt/openwrt/blob/d40f59825a7abc0fe6f1116d599db39fcfc0f489/target/linux/generic/files/drivers/platform/mikrotik/rb_hardconfig.c#L455

Otherwise, I can have a look if you pass me that dump to test on, thanks?

Cheers