Changing partition structure in buildroot

For Developers
1

I use a buildroot to create OpenWRT 18.06 images for a MT7628 device (Vocore2). It has 16MB internal flash and no external storage. I'd like to add a partition to it where I can store things that survive not only sysupgrade but sysupgrade -n aka, jffs2reset (they don't need to be copied back automatically or anything, I just want to be able to access them later if need be).

Can this be done in the buildroot? I note that there's a file VOCORE2.dts that contains, amongst other things:

&spi0 {
	status = "okay";

	cs-gpios = <&gpio0 6 GPIO_ACTIVE_HIGH>,
		   <&gpio0 10 GPIO_ACTIVE_HIGH>;

	m25p80@0 {
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "jedec,spi-nor";
		reg = <0>;
		spi-max-frequency = <10000000>;
		m25p,chunked-io = <32>;

		partition@0 {
			label = "u-boot";
			reg = <0x0 0x30000>;
			read-only;
		};

		partition@30000 {
			label = "u-boot-env";
			reg = <0x30000 0x10000>;
			read-only;
		};

		factory: partition@40000 {
			label = "factory";
			reg = <0x40000 0x10000>;
		};

		partition@50000 {
			label = "firmware";
			reg = <0x50000 0xfb0000>;
		};
	};

	spidev@1 {
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "rohm,dh2228fv";
		reg = <1>;
		spi-max-frequency = <100000000>;
	};
};

(This file is not in the OpenWRT repo, it comes from the Vocore repo.) It seems like this defines some mappings for the memory of the flash, but I'm not sure whether this is the correct level for defining the kind of partitioning I'm talking about, or whether it's more for memory-mapped peripherals seen by the CPU. If it's not the right thing to hack on, what is?

Mostly I'm struggling to find documentation on this, so if someone could point me to any docs on customising the partitioning scheme in OpenWRT builds, I'd really appreciate it. (It can be for more recent versions than 18.06, I realise that's quite old at this point, anything that helps me get started is good.)

2

I'm still working out the details, but for anyone else searching these terms, these OpenWRT wiki pages are immensely useful:

It looks like what I want is a new "layer 1" partition (using the terms of the flash layout wiki page), that I can format and do what I need to do with. The big piece of information I'm missing is how to see the real structure of these paritions, including nesting, which cat /proc/mtd doesn't show.

3

Your device in master, partitions are defined in the .dtsi file, but the firmware size (different in normal and lite versions of the device) are in .DTS

You should decrease the firmware size from 0xfb0000 by amount you want, and then define in the .dtsi a new partition for your needs.
E.g.
Split <0x50000 0xfb0000> as
for firmware <0x50000 0xbb0000>
For new <0xc00000 0x400000>

And then it is upto you to initialise a file system there once the compiled for ware is running.

Edit
For 18.06 it seems to be one one file

4

Thanks, this is a huge help! Two smaller questions:

  1. Is it important to have the partition be in multiples of the erase size? I've rounded my "extra" partition space up to enforce this, but I don't know if it's necessary.

  2. Some partitions have a text label in the stanza:

     partition@0 {
   label = "u-boot";
   reg = <0x0 0x30000>;
   read-only;
};

    ...and some have extra C-style labels prefixing the whole stanza:

    factory: partition@40000 {
  label = "factory";
  reg = <0x40000 0x10000>;
};

    I've looked in the DTS spec and can't find anything about this syntax. Do you know what it means? Specifically, the extra factory: at the start? (As in, I know what it's for, it's for various Vocore and peripheral device settings, but why is there an extra label?)

5

Yes. Based on your .dts, looks like 64 kB , 0x10000

the C-style prefix labels should be used when the item is referenced somewhere in the DTS.
(and they should be omitted if there is no reference to the item.)

The actual "label" property is different and is compiled into the devicetree blob.

From the DTS standard you referenced:

C-style compilation label:

6.2 Labels
The source format allows labels to be attached to any node or property value in the devicetree. Phandle and path references can be automatically generated by referencing a label instead of explicitly specifying a phandle value or the full path to a node. Labels are only used in the devicetree source format and are not encoded into the DTB binary.
A label shall be between 1 to 31 characters in length, be composed only of the characters in the set Table 6.1, and must not start with a number.
Labels are created by appending a colon (‘:’) to the label name. References are created by prefixing the label name with an ampersand

Real property:

label Property
Table 4.3: label Property
Property label
Value type
Description The label property defines a human readable string describing a device. The binding for a given device specifies the exact meaning of the property for that device.

That is actually strange, because I found no reference to it.

6

Okay, that makes sense. The labels inside the blocks are the ones that show up in eg. cat /proc/mtd or dmesg. The ones outside the block are for the thing that processes the DTS file itself.

7

This all works beautifully, I now have /dev/mtdblock7 (and the other corresponding character devices) with the correct label. One last set of related questions though -

Am I right in thinking that I have to manage it from the running image, and not the buildroot? Since I want it to "survive" sysupgrades, my thinking is that if I did try to generate an image for it in the buildroot, it would just clobber whatever is there when I flash it. Correct? (Same as the U-boot partitions etc etc).

In that case, I can easily write a uci-defaults script and maybe some other infrastructure to deal with it as needed. However, where are the tools to do so? I tried searching the repos for "jffs2" and "ubifs" (and even "yaffs"), but I got no results. I can't find them via make menuconfig in the core packages either.

Last question - after modifying the DTS file, building an image and doing the sysupgrade (without the -n flag), I ran some block info commands:

root@00dc:/# block info /dev/mtdblock5
/dev/mtdblock5: UUID="a6475090-1f00cd55-757d3786-b9373b99" VERSION="4.0" MOUNT="/rom" TYPE="squashfs"
root@00dc:/# block info /dev/mtdblock6
/dev/mtdblock6: MOUNT="/overlay" TYPE="jffs2"
root@00dc:/# block info /dev/mtdblock7
/dev/mtdblock7: TYPE="jffs2"

My new partition is /dev/mtdblock7 - why is block info detecting it as jffs2? I wouldn't have thought it had the appropriate metadata for that. Is it picking up on whatever bytes are sitting there from before the new partition scheme?

8

(I've just discovered that the partition is too small for jffs2, but the rest of the questions are still valid: where are the fs utils?)

9

You likely/possibly need to install them. Depends a bit on which filesystem you want.

For some devices the ubifs tools are there by default, but for others you need to install ubi-utils

root@router1:~# opkg info ubi-utils
Package: ubi-utils
Version: 2.1.4-1
Depends: libc
Status: install user installed
Section: utils
Architecture: arm_cortex-a15_neon-vfpv4
Size: 77172
Filename: ubi-utils_2.1.4-1_arm_cortex-a15_neon-vfpv4.ipk
Description: Utilities for manipulating memory technology devices.
Installed-Time: 1648145936

E.g. ubiformat

root@router1:~# ubiformat -h
ubiformat version 2.1.4 - a tool to format MTD devices and flash UBI images

Usage: ubiformat <MTD device node file name> [-s <bytes>] [-O <offs>] [-n]
                        [-Q <num>] [-f <file>] [-S <bytes>] [-e <value>] [-x <num>] [-y] [-q] [-v] [-h]
                        [--sub-page-size=<bytes>] [--vid-hdr-offset=<offs>] [--no-volume-table]
                        [--flash-image=<file>] [--image-size=<bytes>] [--erase-counter=<value>]
                        [--image-seq=<num>] [--ubi-ver=<num>] [--yes] [--quiet] [--verbose]
                        [--help] [--version]

Example 1: ubiformat /dev/mtd0 -y - format MTD device number 0 and do
           not ask questions.
Example 2: ubiformat /dev/mtd0 -q -e 0 - format MTD device number 0,
           be quiet and force erase counter value 0.

-s, --sub-page-size=<bytes>  minimum input/output unit used for UBI
                             headers, e.g. sub-page size in case of NAND
                             flash (equivalent to the minimum input/output
                             unit size by default)
-O, --vid-hdr-offset=<offs>  offset if the VID header from start of the
                             physical eraseblock (default is the next
                             minimum I/O unit or sub-page after the EC
                             header)
-f, --flash-image=<file>     flash image file, or '-' for stdin
...

For example the ext2/ext3/ext4 tools are in e2fsprogs package.

opkg install e2fsprogs

root@router1:~# mkfs.ext4
Usage: mkfs.ext4 [-c|-l filename] [-b block-size] [-C cluster-size]
        [-i bytes-per-inode] [-I inode-size] [-J journal-options]
        [-G flex-group-size] [-N number-of-inodes] [-d root-directory]
        [-m reserved-blocks-percentage] [-o creator-os]
        [-g blocks-per-group] [-L volume-label] [-M last-mounted-directory]
        [-O feature[,...]] [-r fs-revision] [-E extended-option[,...]]
        [-t fs-type] [-T usage-type ] [-U UUID] [-e errors_behavior][-z undo_file]
        [-jnqvDFSV] device [blocks-count]
10

Thanks! I discovered there's also a busybox flag to enable the mke2fs command. I think the UBI packages aren't in 18.06, so I'll try it under a 21.x build. I don't think there's any point backporting them since they rely on kernel support as well.

(You're always extremely helpful @hnyman! I really appreciate it.)

11

Yep, 18.06 has been end-of-life for some time now, and also 19.07 is supposed to be deprecated soon. 21.02 is the currently available stable, while 22.03 will be there soon, hopefully.

2 Likes
12

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.