Somebody please school me on MTD partitions and DTS

I am working to create an image for a new device - Belkin F9K1109v1. As you will see below I don't know what I am doing. This is a ramips/rt3883 and uses DTS. I am able to TFTP and openwrt runs nicely as a initramfs (no flashing). I am failing miserably on getting a squashfs image to flash and boot. I am 100% sure I am missing some fundamentals on how the MTD partitions are setup.

My main question is: do I need to set partitions for kernel, rootfs and rootfs_data in the DTS? I can see how that would make sense, but the existing DTS examples I have been looking at only have "firmware". Some other magic appears to create kernel, rootfs and rootfs_data based on the firmware?

Here is a an example with a RT-N56U which only has a "firmware" partition
https://github.com/openwrt/openwrt/blob/master/target/linux/ramips/dts/RT-N56U.dts

Here is a boot log snippet from the RT-N56U running openwrt (not positive this is from build using the above DTS)

[    2.200000] 4 ofpart partitions found on MTD device 1c000000.nor-flash
[    2.210000] Creating 4 MTD partitions on "1c000000.nor-flash":
[    2.220000] 0x000000000000-0x000000030000 : "u-boot"
[    2.230000] 0x000000030000-0x000000040000 : "u-boot-env"
[    2.240000] 0x000000040000-0x000000050000 : "factory"
[    2.250000] 0x000000050000-0x000000800000 : "firmware"
[    2.270000] 2 uimage-fw partitions found on MTD device firmware
[    2.280000] 0x000000050000-0x000000171408 : "kernel"
[    2.290000] 0x000000171408-0x000000800000 : "rootfs"
[    2.300000] mtd: device 5 (rootfs) set to be root filesystem
[    2.310000] 1 squashfs-split partitions found on MTD device rootfs
[    2.330000] 0x000000650000-0x000000800000 : "rootfs_data"

I think this means the kernel is using partitions from the firmware image? Note it is also setting the root fs here

Here is a bootlog from this F9K1109v1 I am working on

[    2.354987] spi spi0.0: force spi mode3
[    2.363529] m25p80 spi0.0: mx25l6405d (8192 Kbytes)
[    2.373358] 4 fixed-partitions partitions found on MTD device spi0.0
[    2.386026] Creating 4 MTD partitions on "spi0.0":
[    2.395585] 0x000000000000-0x000000030000 : "uboot"
[    2.406314] 0x000000030000-0x000000040000 : "uboot-env"
[    2.417730] 0x000000040000-0x000000050000 : "factory"
[    2.428806] 0x000000050000-0x0000007f0000 : "firmware"

no other partitions created. Also no MTD is set as root fs, which causes the panic below. I know I can set the root=/dev/mtdblockX as a kernel parameter, but I haven't found many DTS-es that do that.

[    4.861710] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[    4.876689] Please append a correct "root=" boot option; here are the available partitions:
[    4.893344] 1f00             192 mtdblock0
[    4.893350]  (driver?)
[    4.906379] 1f01              64 mtdblock1
[    4.906383]  (driver?)
[    4.919401] 1f02              64 mtdblock2
[    4.919405]  (driver?)
[    4.932423] 1f03            7808 mtdblock3
[    4.932428]  (driver?)
[    4.945452] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

Any help is much appreciated. I can post DTS, bootlogs whatever if any of that is helpful.

thanks

  1. You can specify mtdparts / root-block / root with kernel boot parameters

  2. Wikidevi is.... EDIT remove misguided techdata.

How exactly are you flashing / reverting.... ?

The general method is to;

  1. Obtain an OEM bootlog from uboot/cfe/whatver

  2. Obtain sources which are as near as can be to that then go from there.... Assuming they are available.

In general, that would mean zapping a substitute kernel and rootfs over the OEM layout..... hence the need to see what you had from factory.

Post an OEM bootlog so we can verify some hardware constraints ( flash size - cpu and wired network ) being the most critical, and you might get a more help :slight_smile:

Thanks Wulfy any tips help.

I haven't seen a "V2" anywhere on wikidev have you found one? The F9K1109v1 is almost identical to the "1103" here https://wikidevi.com/wiki/Belkin_F9K1103_v1. Minor chip model number differences on flash and memory, but they are the same size 8M/64M.

I am creating/modifying DTS and various make files etc to try to get this going. A couple pasted below.

The other routers I have looked at to based this on are the other RT3883 and the older RT3662 that are supported in openwrt. Here are some keys ones: DIR-645, BR-6475nD, CY-SWR1100, TEW-692GR and the RT-N56U. I also have the OEM source for the F9K1103v1. I think this 1109 I have may even the same exact firmware or very close. You can see in the bootlog it is marked as F9K1103v1

Flashing - flashing attempts from the stock belkin firmware. After that crashes and burns, I flash back to the stock FW from the bootloader (u-boot). Serial console, TFTP copy.

Note this thing runs this openwrt build with wifi, leds, luci etc from a ramfs copy. Just haven't figured out the flashing.

Here comes way too much info. Hopefully something in here is helpful.

Stock bootlog
https://pastebin.com/raw/H4nNuKv2

u-boot env values
https://pastebin.com/raw/2D0zuQs1

stock MTD (also in the bootlog)
https://pastebin.com/raw/rdH8WBKA

current DTS I am working with
https://pastebin.com/raw/QTchLVLb

current snippet from the make file rt3883.mk that is attempting to build the image
https://pastebin.com/raw/4AwURqaG

openwrt boot from memory (not flash) works
https://pastebin.com/raw/gKN64Rbr

openwrt flash attempt from stock admin web and failed boot
https://pastebin.com/raw/HJnytr6u

1 Like

DOH! Damn confusing model numbers!

Thanks for the info, will help alot :slight_smile:

have you tried;

setenv addmiscO "setenv bootargs $(bootargs) console=ttyS0,$(baudrate)"
setenv addmisc "setenv bootargs $(bootargs) console=ttyS0,$(baudrate) root=/dev/mtdblock3"

or

setenv addmisc "setenv bootargs $(bootargs) console=ttyS0,$(baudrate) root=1f03"

etc ..... etc ....... when you get it, you can put in a DTS like this;

    chosen {
                bootargs = "console=ttyS0,57600 root=1f05 rootfstype=squashfs";
        };

( or similar )..... takes too long to recompile everytime..... just to test!!! :slight_smile:

This relic has some relevant tidbits: https://forum.archive.openwrt.org/viewtopic.php?id=67215

Also, I don't know how relevant this is to your board... but it's worth a look:

cat target/linux/ar71xx/image/legacy.mk | grep -A 20 elkin | head -n 24

This is getting me closer. If I set the root= bootarg I get bunch of jffs errors like this

[    4.959128] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x3ec4 instead

Interestingly, this is very similar to the thread in your link. Basically with root= param jffs errors without root param cannot mount rootfs. This might be a hint in the right direction. I'll keep digging

Please try moving the mtd partitions into the "partitions" node.

&spi0 {
	status = "okay";

	nor-flash@0 {
		compatible = "jedec,spi-nor";
		reg = <0>;
		spi-max-frequency = <25000000>;
	
		partitions {
			compatible = "fixed-partitions";
			#address-cells = <1>;
			#size-cells = <1>;

			partition@0 {
				label = "uboot"; /* Bootloader */
				reg = <0x0 0x30000>;
				read-only;
			};

			partition@30000 {
				...
			};

			...
		};
	};
};

If each partitions are defined directly under the node of flash, the "firmware" partition specified any compatible string may not be recognized properly.

Thanks. I tried this and same result so far.

Based on this link from wulfy above https://forum.archive.openwrt.org/viewtopic.php?id=67215

I am assuming the normal case is to not specify the rootfs manually. Just use the firmware partition and then "the magic happens" where openwrt builds the kernel and rootfs on boot somehow. The missing piece for me is how to trigger that? I noticed in that same link it looks like they are building a initramfs factory image? The image builder creates squashfs for me. Also they are using a mkedimaximg tool to build it. Any hints in here?

Minor update. I realized there have been some updates around at least ramips DTS in master recently. I was finding different DTS for the same devices based on 18.06 and master e.g. compatible = "denx,uimage" on the firmware partition. I was running my changes on top of the 18.06 branch. Pulled down master and will apply my changes and see if I get any different results. I will post up as I get into it.

I'm hardly an expert on DTS, and even farther from being an expert on the OpenWrt style standards, but the following links may be helpful

as well as the general information at

https://devicetree-specification.readthedocs.io/en/latest/source-language.html

https://elinux.org/Device_Tree_Mysteries

https://elinux.org/Device_Tree_Source_Undocumented

1 Like

Thanks Jeff. These links help. I had read some of them but not all and I wouldn't say they sank in entirely the first time. Just went over them again and indeed more sank in. Unfortunately, I am still air-balling on getting this going.

I think my main problem is I can't get the MTD splits/parses to happen as this link is describing https://openwrt.org/docs/guide-developer/defining-firmware-partitions

Note from that link

If you see:
2 uimage-fw partitions found on MTD device firmware
please use “denx,uimage”; e.g.

I never see the "2 XYZ partitions found on MTD device firmware" in my bootlogs. Stock doesn't either because they have set MTDs for all the blocks. I thought building from master with the “denx,uimage” might do it, but it did not.

Here is the bootlog, no rootfs built or mounted, so panic.

https://pastebin.com/raw/P0QxsfhB

I may try creating kernel and rootfs partitions manually and see if that works. I will have to set based on the output size of the kernel I think.

Oops one more. Is it possible I am screwing up the image creation? I tried to make it as vanilla as possible. The oddities so far are the smaller lzma block size and the UIMAGE_NAME, which the stock firmware checks as a "PDTAG".

Here is the stanza in the make file

define Device/f9k1109
  DTS := F9K1109
  BLOCKSIZE := 64k
  DEVICE_TITLE := Belkin F9K1109
  DEVICE_PACKAGES := kmod-usb-core kmod-usb-ohci kmod-usb2 swconfig
  IMAGE_SIZE := 7224k
  KERNEL := kernel-bin | patch-dtb | lzma -d16 | uImage lzma
  # Stock firmware checks for this uImage image name during upload.
  UIMAGE_NAME := N750F9K1103VB
  IMAGES += factory.bin
  IMAGE/factory.bin := $$(sysupgrade_bin) | check-size $$$$(IMAGE_SIZE)
endef
TARGET_DEVICES += f9k1109

binwalk

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0x285FAA7E, created: 2018-08-16 07:51:15, image size: 1587960 bytes, Data Address: 0x80000000, Entry Point: 0x80000000, data CRC: 0xAD3E3507, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "N750F9K1103VB"
64            0x40            LZMA compressed data, properties: 0x6D, dictionary size: 65536 bytes, uncompressed size: 4642508 bytes
1588024       0x183B38        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 1953470 bytes, 846 inodes, blocksize: 262144 bytes, created: 2018-08-16 07:51:15
1 Like

Probable.... if you have a buildroot environment.... run the command at the bottom of my last post.... it will show you some magic used on another device of the same brand....

I know what your doing can get pretty disheartening from time to time....

I actually cheated early on..... and kinda avoided having to fly into the wall you have....... by deciding, partly because I didn't want to fry the OEM mtd / mmc data.....

I figured it would be rapidworthy to just build usb kernel modules into my kernel..... and then fire up the kernel / rootfs off USB.

In the end I ran into some major dramas getting the modules compiled in.... good news is that i'm happy to help you do that if you decide that might be an effortworthy tangent :slight_smile:

P.S.

-The "recipe" ( cat target/linux/ar71xx/image/legacy.mk | grep -A 20 elkin | head -n 24 OR SIMILAR )ensures that appropriate padding / format / fs pointers are in the right place..... A comparible DTS might be this one cat target/linux/ramips/dts/F5D8235_V1.dts ( see the last line )

EDIT: Has something like this: ralink,mtd-eeprom = <&factory 0x0>;

-The flashing tool MAY use that "metadata" to write your created firmware cleanly. So depending on a) the tool ( did you try writing just a root.squashfs from your initramfs? ) and b) The data substructure of your "image"/parts what ends up on your flash is determined.

So that would be a good next step if you decide your up for it. Use your initramfs ( or OEM system ) to interrogate / write to the flash..... but watch out to not zap UBOOT and ART. And make backups first of everything.

Magic bitmask 0x1985 not found at 0x00000000: 0x3ec4 instead
( binwalk -e test.img; file 183B38 .... etc .... )
( from initram ... dd if=/dev/mtd3 of=/usb or server; binwalk / hexdump -C / strings )

EDIT: What I mean, is if you can find tools that will inspect the created images..... that will inspect an OEM mtd / flash backup..... then you will be able to clearly see inconsistencies. i.e. Perhaps a newer model does not use part/s of the recipe for that similar branded device.....

hexdump -C mtd3.img ( etc. etc. etc. ) will tell you.

as will it tell you things after new firmware is written. the advantage of such tools is in the future, you can plug in almost any device.... run exactly the same commands, and let the pudding tell you where the proof is :wink:

Oh man, so many new hints. I missed the "elkin" search from your earlier post. Interesting stuff in there, like the edimax header.

Not sure what you mean by the last line? Last line in that DTS file?

This is a definite possibility. Maybe the stock tool does? I have only been flashing from the stock UI uploader.

Tonight I'll try a couple things. 1- check what was written to the "rootfs" mtd and see if it lines up to the image. I think the squashfs header should be at the start. 2- writing just rootfs.squash via a running initramfs install.

Not following here, which means it is a likely clue! I snipped the log more of it below. I assumed it was scanning for jffs magic and not finding it, but killed it as it keeps running. Looking at it closer I am not sure where it is looking. In hexdumps of the bin before writing there aren't any 0x3ec4 near the start of the bin or the start of the rootfs mtd block. Maybe it is getting moved by the stock flash tool. I'll sniff around here too after I get the initramfs running tonight.

Thanks for all the hints

[    2.757727] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x38a9 instead
[    2.776700] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0x2e95 instead
[    2.795604] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000008: 0x1944 instead
[    2.814502] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000000c: 0xb7ae instead
[    2.833402] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000010: 0xccda instead
[    2.852301] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000014: 0xb66c instead
[    2.871199] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000018: 0x25e4 instead
[    2.890107] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000001c: 0x6e46 instead
[    2.909006] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000020: 0x5624 instead
[    2.927904] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000024: 0xf677 instead
[    2.946795] jffs2: Further such events for this erase block will not be printed
[    2.970788] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001000: 0xf011 instead
[    2.989704] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001004: 0x6aaa instead
[    3.008607] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001008: 0xb85e instead
[    3.027506] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000100c: 0x0340 instead
[    3.046406] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001010: 0x6f74 instead
[    3.065305] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001014: 0xf704 instead
[    3.084207] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001018: 0x9d24 instead
[    3.103106] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000101c: 0x28a7 instead
[    3.122005] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001020: 0xf2a1 instead
[    3.140904] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001024: 0xedeb instead
[    3.159797] jffs2: Further such events for this erase block will not be printed
[    3.183775] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00002000: 0x3c1b instead```

Oh yeah. I should mention later last night after my posts. I did create kernel and rootfs partitions manually in the DTS and removed "firmware". Then changed the make file to build the image to pad after kernel to where I have the rootfs lined up in the DTS. Found rootfs and set a root, but then

mtdsplit: no squashfs found in "rootfs"

Also those jffs2_scan_eraseblock() errors are from that boot too. This was flashed through stock FW web too. So like above I wonder if a) I whiffed on the partition offsets or b) the stock FW altered it. Will find out and report back

OEM

0             0x0             uImage header, header size: 64 bytes, header CRC: 0xB7FB57E3, created: 2017-07-18 06:54:31, image size: 7402350 bytes, Data Address: 0x80000000, Entry Point: 0x8033E000, data CRC: 0xB0B3E3E, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "N750F9K1103VB"
64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 65536 bytes, uncompressed size: 3526536 bytes
1364910       0x14D3AE        Squashfs filesystem, little endian, non-standard signature, version 3.0, size: 6035208 bytes, 1672 inodes, blocksize: 65536 bytes, created: 2017-07-18 06:54:16

Extracting 1364910 bytes of uimage header image at offset 0
Extracting squashfs file system at offset 1364910

strings //* | grep -A30 -B30 mtd

%s: MTD write OK
N750F9K1103VB
Kernel_RootFS
usage: write [path] [device]

sasquatch 14D3AE.squashfs

vert@sone 09:29:51 /xsrc/F9K1109v1/_BINWALK/_SQUASHINLY > sasquatch 14D3AE.squashfs 
SquashFS version [3.0] / inode count [1672] suggests a SquashFS image of the same endianess
Non-standard SquashFS Magic: shsq

And if you needed to reqsuash ( install firmware mod kit );

./src/others/squashfs-3.2-r2-rtn12/unsquashfs 14D3AE.squashfs.lzma

Looks like OEM uses "shsq" lzma.... So if the OEM fw unlzma's before write then that might have corrupted the rootfs....

I'm not up on this, maybe a kind dev can tell us if that's how it works...

It feels you are getting me very close to the solution. I did notice last night the stock bin is Squash 3 and the image from the imagebuilder is Squash 4 and they have different squash magic something like "hsqs" instead of "shsq". I assumed stock FW wouldn't care since openwrt would be mounting it anyway.

I hadn't found sasquatch either. More puzzle pieces.

My first plan is to boot a ramfs and check out what is at the start of the rootfs MTD.

Side note, as I am looking at the binwalks closer I hadn't noticed this before, but ...

Stock

0             0x0             uImage header, header size: 64 bytes, header CRC: 0xB7FB57E3, created: 2017-07-18 06:54:31, image size: 7402350 bytes, Data Address: 0x80000000, Entry Point: 0x8033E000, data CRC: 0xB0B3E3E, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "N750F9K1103VB"
64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 65536 bytes, uncompressed size: 3526536 bytes
1364910       0x14D3AE        Squashfs filesystem, little endian, non-standard signature, version 3.0, size: 6035208 bytes, 1672 inodes, blocksize: 65536 bytes, created: 2017-07-18 06:54:16

Mine recent (I have done recent tweaks to pad kernel, but not important here)

0             0x0             uImage header, header size: 64 bytes, header CRC: 0x285FAA7E, created: 2018-08-16 07:51:15, image size: 1587960 bytes, Data Address: 0x80000000, Entry Point: 0x80000000, data CRC: 0xAD3E3507, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "N750F9K1103VB"
64            0x40            LZMA compressed data, properties: 0x6D, dictionary size: 65536 bytes, uncompressed size: 4642508 bytes
1588024       0x183B38        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 1953470 bytes, 846 inodes, blocksize: 262144 bytes, created: 2018-08-16 07:51:15

Stock has squash 3 vs squash 4, but maybe more significant.. the openwrt one has xv compression on the squashfs, no compression on the stock one. I am out of my depth here, but certainly feels plausible the stock FW has no idea what do with that.

... getting close I think

1 Like
[    0.610542] 6 fixed-partitions partitions found on MTD device spi0.0
[    0.623215] Creating 6 MTD partitions on "spi0.0":
[    0.632780] 0x000000000000-0x000000030000 : "uboot"
[    0.643531] 0x000000030000-0x000000040000 : "uboot-env"
[    0.654973] 0x000000040000-0x000000050000 : "factory"
[    0.666096] 0x000000050000-0x000000250000 : "kernel"
[    0.676995] 0x000000250000-0x0000007f0000 : "rootfs"
[    0.687865] mtd: device 4 (rootfs) set to be root filesystem
[    0.703365] 1 squashfs-split partitions found on MTD device rootfs
[    0.715751] 0x000000473000-0x0000007f0000 : "rootfs_data"
[    0.727582] 0x0000007f0000-0x000000800000 : "user-cfg"

Ka .... boom!

Wulfy, at this point I am close to certain you nailed it. I checked the mounted MTD from stock FW UI flash and the squash header was not there. I am assuming was corrupted by the stock FW. I flashed from the same image from the bootloader / TFTP and it mounts rootfs, splits out rootfs_data, puts on the overlay and all of it.

This image has the kernel and rootfs defined in the MTD and the make file pads the kernel part of the image so they align. Next steps are to clean up this master based image a bit, then try it with just a "firmware" partition in the DTS and see if it will split firmware and size the rootfs dynamically if I flash from the bootloader. Last step will see if I can build an image that will flash from the stock FW, maybe there is a way to not compress the rootfs?

I'll report back with updates. Huge thanks Wulfy and others for the tips.

1 Like

Dude, you nailed it. I didn't have the balls to get into that kinda low level tango.

Good job :slight_smile: BOOOOMBAHHHHHH!!! :stuck_out_tongue:

Hey man huge thanks for all the tips and motivation. I was about to bail out on this project. Now on to the clean up efforts...