DTS for a UBI / MTD device (Tilgin HG2381)

For Developers
1

Ref Tilgin HG2381_UK - #2 by tmomas
Request: OpenWrt on Tilgin 2381

Need a bit of design help. There are lots of the above devices in the wild, so they'd have a user-base.

Bootlog snip (mtd2???):

NAND device: Manufacturer ID: 0xef, Chip ID: 0xf1 (Unknown W29N01GV)
128 MiB
Creating 1 MTD partitions on "nand0":
0x000000000000-0x000000080000 : "U-Boot"
Creating 1 MTD partitions on "nand0":
0x000000080000-0x000008000000 : "ubi"
UBI: attaching mtd2 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: smallest flash I/O unit:    2048
UBI: sub-page size:              512
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI: max. sequence number:       1443
UBI: attached mtd2 to ubi0
UBI: MTD device name:            "ubi"
UBI: MTD device size:            127 MiB
UBI: number of good PEBs:        1020
UBI: number of bad PEBs:         0
UBI: number of corrupted PEBs:   0
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    256
UBI: number of internal volumes: 1
UBI: number of user volumes:     10
UBI: available PEBs:             848
UBI: total number of reserved PEBs: 172
UBI: number of PEBs reserved for bad PEB handling: 10
UBI: max/mean erase counter: 257/2
UBI: image sequence number:  1377090127

Here's info from uboot (ubi0+1 with PEBs and LEBs, so I guess this explains the difference in erasesizes in MTD):

GRX330 # ubi info

UBI: MTD device name:            "ubi"
UBI: MTD device size:            127 MiB
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: number of good PEBs:        1020
UBI: number of bad PEBs:         0
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    256
UBI: number of internal volumes: 1
UBI: number of user volumes:     10
UBI: available PEBs:             848
UBI: total number of reserved PEBs: 172
UBI: number of PEBs reserved for bad PEB handling: 10
UBI: max/mean erase counter: 257/2
GRX330 # 


GRX330 # ubi info layout

UBI: volume information dump:
UBI: vol_id          1
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        8
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      4096
UBI: last_eb_bytes   4096
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            Config-C

UBI: volume information dump:
UBI: vol_id          2
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        8
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      24576
UBI: last_eb_bytes   24576
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            Config-A

UBI: volume information dump:
UBI: vol_id          3
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        3
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      24576
UBI: last_eb_bytes   24576
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            Log

UBI: volume information dump:
UBI: vol_id          4
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        11
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      7936
UBI: last_eb_bytes   7936
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            Environment

UBI: volume information dump:
UBI: vol_id          5
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        6
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      4096
UBI: last_eb_bytes   4096
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            Misc-A

UBI: volume information dump:
UBI: vol_id          6
UBI: reserved_pebs   11
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        6
UBI: usable_leb_size 129024
UBI: used_ebs        11
UBI: used_bytes      1407723
UBI: last_eb_bytes   117483
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            kernel

UBI: volume information dump:
UBI: vol_id          7
UBI: reserved_pebs   139
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        6
UBI: usable_leb_size 129024
UBI: used_ebs        139
UBI: used_bytes      17899520
UBI: last_eb_bytes   94208
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            rootfs

UBI: volume information dump:
UBI: vol_id          8
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        5
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      4096
UBI: last_eb_bytes   4096
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            appfs

UBI: volume information dump:
UBI: vol_id          9
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        3
UBI: name_len        7
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      129024
UBI: last_eb_bytes   129024
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            caldata

UBI: volume information dump:
UBI: vol_id          10
UBI: reserved_pebs   1
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        4
UBI: name_len        9
UBI: usable_leb_size 129024
UBI: used_ebs        1
UBI: used_bytes      169
UBI: last_eb_bytes   169
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            test_data

UBI: volume information dump:
UBI: vol_id          2147479551
UBI: reserved_pebs   2
UBI: alignment       1
UBI: data_pad        0
UBI: vol_type        3
UBI: name_len        13
UBI: usable_leb_size 129024
UBI: used_ebs        2
UBI: used_bytes      258048
UBI: last_eb_bytes   2
UBI: corrupted       0
UBI: upd_marker      0
UBI: name            layout volume

Here's what I see via shell:

ubinfo -a
UBI version:                    1
Count of UBI devices:           1
UBI control device major/minor: 10:62
Present UBI devices:            ubi0

ubi0
Volumes count:                           10
Logical eraseblock size:                 129024 bytes, 126.0 KiB
Total amount of logical eraseblocks:     1020 (131604480 bytes, 125.5 MiB)
Amount of available logical eraseblocks: 848 (109412352 bytes, 104.3 MiB)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       0
Count of reserved physical eraseblocks:  10
Current maximum erase counter value:     257
Minimum input/output unit size:          2048 bytes
Character device major/minor:            252:0
Present volumes:                         1, 2, 3, 4, 5, 6, 7, 8, 9, 10

Volume ID:   1 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  4096 bytes (4.0 KiB)
State:       OK
Name:        Config-C
Character device major/minor: 252:2
-----------------------------------
Volume ID:   2 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  24576 bytes (24.0 KiB)
State:       OK
Name:        Config-A
Character device major/minor: 252:3
-----------------------------------
Volume ID:   3 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  24576 bytes (24.0 KiB)
State:       OK
Name:        Log
Character device major/minor: 252:4
-----------------------------------
Volume ID:   4 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  7936 bytes (7.8 KiB)
State:       OK
Name:        Environment
Character device major/minor: 252:5
-----------------------------------
Volume ID:   5 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  4096 bytes (4.0 KiB)
State:       OK
Name:        Misc-A
Character device major/minor: 252:6
-----------------------------------
Volume ID:   6 (on ubi0)
Type:        static
Alignment:   1
Size:        11 LEBs (1419264 bytes, 1.4 MiB)
Data bytes:  1407723 bytes (1.3 MiB)
State:       OK
Name:        kernel
Character device major/minor: 252:7
-----------------------------------
Volume ID:   7 (on ubi0)
Type:        static
Alignment:   1
Size:        139 LEBs (17934336 bytes, 17.1 MiB)
Data bytes:  17899520 bytes (17.1 MiB)
State:       OK
Name:        rootfs
Character device major/minor: 252:8
-----------------------------------
Volume ID:   8 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  4096 bytes (4.0 KiB)
State:       OK
Name:        appfs
Character device major/minor: 252:9
-----------------------------------
Volume ID:   9 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
State:       OK
Name:        caldata
Character device major/minor: 252:10
-----------------------------------
Volume ID:   10 (on ubi0)
Type:        static
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
Data bytes:  169 bytes
State:       OK
Name:        test_data
Character device major/minor: 252:11

and MTD (note differing erasesizes)

# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00080000 00020000 "U-Boot"
mtd1: 07f80000 00020000 "ubi"
mtd2: 00001000 0001f800 "Config-C"
mtd3: 00006000 0001f800 "Config-A"
mtd4: 00006000 0001f800 "Log"
mtd5: 00001f00 0001f800 "Environment"
mtd6: 00001000 0001f800 "Misc-A"
mtd7: 00157aeb 0001f800 "kernel"
mtd8: 01112000 0001f800 "rootfs"
mtd9: 00001000 0001f800 "appfs"
mtd10: 0001f800 0001f800 "caldata"
mtd11: 000000a9 0001f800 "test_data"

DTS (guess, based on MTD sizes):

...

	partition@0 {
		label = "U-Boot";
		reg = <0x000000 0x80000>;
		read-only;
	};

	partition@80000 {
		label = "ubi";
		reg = <0x80000 0x07f80000>;
	};

	/* this is the layout inside of UBI
	partition@80000 {
		label = "Config-C"; //mtdfactory from crm_factory_sw
		reg = <0x80000 0x1000>;
	};

	partition@81000 {
		label = "Config-A"; //mtddev squashfs /var/etc = crm stuff
		reg = <0x81000 0x6000>;
	};

	partition@87000 {
		label = "Log"; //squashfs /var/log
		reg = <0x87000 0x6000>;
	};

	partition@8D000 {
		//MAC addr, passwords, model names, numbers etc.
		label = "Environment"; //uboot env mounted as /var/identity.txt
		reg = <0x8D000 0x1f00>;
	};

	partition@8EF00 {
		//cal_wlan0.bin          dropbear_rsa_host_key  hostapd-wlan2.psk
		//dropbear_dss_host_key  hostapd-wlan0.psk
		label = "MiscA"; // squashfs /var/miscA
		reg = <0x8EF00 0x1000>;
	};
	partition@8FF00 {
		label = "kernel"; //uboot_image
		reg = <0x8FF00 0x157aeb>; //1,407,723 bytes
	};
	partition@1E79EB {
		label = "rootfs"; //squashfs
		reg = <0x1E79EB 0x1112000>; //17,899,520 bytes
	};
	partition@12F99EB {
		label = "appfs"; //squashfs /app
		reg = <0x12F99EB 0x1000>;
	};
	partition@12FA9EB {
		label "caldata"; //qualcomm calibration data(?) for the ar10
		// Read 2116 bytes (0x844) from /dev/caldata at position 0x00005000
		reg = <0x12FA9EB 0x1f800>;
	};
	partition@131A1EB {
		label "test_data";
		reg = <0x131A1EB 0xa9>;
	}; */

	/* Apparently there are 0x6C65D6C i.e. 113,663,340 bytes free after the above */
...

Question 1) is the DTS roughly correct (if we base directly on the /proc/mtd values)?
Question 2) how much 'reserved' space must one account for in the DTS? 172 * 128KiB = 172 * 0x20000 == 0x1580000 = 22,544,384 bytes?

UBI: total number of reserved PEBs: 172
UBI: number of PEBs reserved for bad PEB handling: 10

Question 3) Is caldata position fixed w.r.t. other partitions?

Question 4) caldata must be kept(?), but (can we?) how would we squeeze in some extra space for bigger kernel and root later? ubiconcatX? e.g.

<immovable partition1>|<ubiconcat0>|<immovable partition2>|<ubiconcat1>|<reserved>

...

<ubiconcat0>|<ubiconcat1>|<etc-->
<-----       ubi           ----->
<kernel>|<rootfs-------->|<extra>

Advantage: more space.
Problems: not possible to revert to OEM flash? We already have an MTD "ubi" partition?

The holy grail is maintaining backwards compat, so that if one finds a way, one could squeeze in a new firmware via the OEM GUI.

1 Like
2

ubi

Source: https://bootlin.com/blog/managing-flash-storage-with-linux/

3

This looks like they are using gluebi, which emulates MTD devices on top of UBI volumes... To see the real MTD flash layout of the stock firmware dmesg will be better, /proc/mtd is not a good source to find the layout.

4

Hmm, you think so?

Was gluebi a thing in 2016 (when this firmware was made)? dmesg reveals nothing which suggests something other than what I wrote in the first post. I checked and dmesg only shows what the bootlog does.

The bootlog shows basically the same things as what the uboot commands and proc/mtd reveal. :confused:

5

For people who are using legacy software and yet want to make use of NAND flash in a way which includes wear-leveling and all that, yes. As in this case they are even storing wifi calibration data inside an UBI volume (a very uncommon decision...), I guess using gluebi is actually the natural choice in this case if one doesn't want to modify the vendor driver which is made to read this data from MTD...

Not quite. dmesg reveals the start offset of MTD partitions, while /proc/mtd only exposes length and block size, also lacking any trace of the underlying device.

6

Sounds right. Could you briefly review the captured bootlogs and see if you find anything more interesting regarding the mtd? I don't think I see anything well described. I see memory offsets for some things, but those are once loaded into memory, no?

7

The erase-size of these (fake) MTD partitions is also a clear indicator that gluebi (or a similar vendor-specific MTD emulation on top of UBI) is being used. Only the MTD partitions listed in dmesg are actually real and should go to DTS:

0x000000000000-0x000000080000 : "U-Boot"
0x000000080000-0x000008000000 : "ubi"

That matches also with what you posted on github:

The only slightly strange thing here is that caldata is stored inside a UBI volume, hence we will need to extract it in userspace and supply it via owl-loader to ath10k driver (which should be fine and easy to do).
The more problematic issue are the MAC addresses assigned to the board which are stored in the Environment UBI volume. Here it will be needed to either assign them (a bit late) in userspace as well, or implement nvmem-cells on top of UBI, which will need dt-bindings for UBI and all that. @rmilecki may know what are the plans for that, if any...

8

Thank you @daniel - It's less of a rabbit hole now.

The MAC addrs in the OEM firmware are mounted to a text file, which is just a char device to read the MTD. But the MACs are stored in one of the MTD partitions. So those are also available via the same mechanism you described.

The wifi may also load its MACs in a similar way, by pointing to a memory offset, if I recall correctly.

Are there any openwrt examples in the wild you could point to?

9

Also, do you think the erase sizes are actually in line with being not a physical block size, but a logical one? (slightly less than the 131k, but 129k)

10

No, as this is the first device to be seen in the wild which uses UBI to store calibration data and MAC addresses. As I said, in case of the calibration data there is an exsiting solution (owl-loader) which allows supplying it from userspace, so that can easily be used to supply data read from that UBI volume. However, MAC addresses will have to set in user-space for now as nvmem-cells cannot (yet) be used on top of UBI volumes.

11

Yes, the size exactly corresponds to UBI logical erase block size (0x1F800 if used on NAND flash with 0x20000 block size, which is the case here).

1 Like
12

Perhaps, considering its age, OEMs figured out that this wasn't a great idea and stopped doing it.

Well, I don't rely on this device, so I'm OK with frying it if you have any creative ideas to get it up and running.

The owl-loader seems useful for the Atheros parts, but the rest of the device is Lantiq. Would that matter?

13

Well, I kinda think the opposite: This was someone with a very progressive mindset designing this firmware. A bit too progressive maybe...
Generally, using UBI for everything is a very good idea. Using legacy hacks like gluebi is not very clean though, but from a OEM perspective it provides a viable shortcut.
Having proper support for reading MAC addresses and calibration data from UBI volumes in the Linux kernel would be nice, actually Linux kernel documentation lists it as possible thing to support in future:

The AR10 SoC family is not sufficiently supported by OpenWrt, so this will need to fiddling around. All I could find when searching for GRX390 is this other device:

Maybe @hauke knows more about what is still missing there?

Are both 2.4GHz and 5GHz wifi provided by QCA chips? Or is one of them Lantiq WAV300 or similar? (Lantiq/Intel Wifi is not supported at all due to lack of decent drivers)

In case of the QCA radio, owl-loader will need to be accompanied by a user-space script which extracts the calibration data from the UBI volume.

14

Dev wiki shows
PXB 4389 EL V2.17 (GRX389) - AnyWAN GRX330 Network Processor (SLLF6)

Bootlogs show for 5GHz:

mtlk mtlk.0: firmware: requesting ProgModel_BG_CB_ar10.bin
mtlk mtlk.0: firmware: requesting ProgModel_BG_CB_40_RevI_ar10.bin

Bootlogs show for 2.4GHz:

mtlk mtlk.0: firmware: requesting ProgModel_BG_nCB_ar10.bin
mtlk mtlk.0: firmware: requesting ProgModel_BG_nCB_40_RevI_ar10.bin

and the mtlk stuff sets up wifi. The rest of the LAN stuff is Lantiq (PXB 4389).

If I am not mistaken, some openwrt Fritz!Boxes use these chipsets, and I found the GPL source for the D-Link DWR-966 which has a very similar board, and a(n old) DTS.

So Lantiq: switch ports, WAN port
Ath: both wifi?

Edit: the hardware list has:

Chips (2.4GHz):
Lantiq PSB 82312 M
v1.2
XWAY tm WRX312

Fully integrated 11n 2x2 RF/PA chip


Chips (5GHz): 
QCA9880-BR4A
FNSW920.U9
15

This still requires some work but it's a work in progress. First we need support for NVMEM layouts which is a matter of weeks. Meanwhile I'll try to get back to DT bindings for UBI volumes. Once we get those 2 things the rest should be simple.

16

Oh cool :slight_smile:

Does this mean one could simply use nvmem-cells in the above DTS, to describe the mtd partitions as found in /proc/mtd?

17

@rmilecki I've also started building bindings for UBI and also realized that UBI will have to be changed in the way it starts -- currently UBI is attached via late_initcall, and that's too late for other drivers to be using anything provided by UBI, such as nvmem-cells inside UBI volumes. Making UBI behave more like other MTD users improves that and allows to attach UBI volumes in parallel with other things happening during boot (my motivation here was mostly to reduce boot time, but I had the nvmem USE case in mind as well).

An earlier version of that patch is here, I've further improved it in the meantime and will post an updated version soon: https://patchwork.ozlabs.org/project/linux-mtd/patch/a547e14ac90af041e8135af5fed9272931dda463.1671738005.git.daniel@makrotopia.org/

1 Like
18

It's not that easy, because what you see there are not actually MTD partitions. Rather they are UBI volumes living on a UBI device which uses an actual MTD partition. New device-tree bindings will have to be introduced for that, both UBI device to MTD device (that's quite easy and @hauke has suggested that in the past and got rejected, I've also tried this more recently) and also UBI volumes (for which it will be even more tricky to convince upstream maintainers).

1 Like
19

OK, thank you again @daniel

20

Supporting 2.4 GHz WiFi will be hard if not impossible....