Overlay too small on RE450v2 snapshot

ZsZs73 · May 24, 2021, 10:26am

Hi,
First time poster but long time user here.
I've installed the latest available snapshot to TP-Link RE450 v2 but the available disk space under /overlay is way smaller (608kB) than on the stable 19.07.7 version (1.5M).
The goal would be to install LUCI on it.
Disk usage on 19.07.7:

  OpenWrt 19.07.7, r11306-c4a6851c72
 -----------------------------------------------------
root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 2.8M      2.8M         0 100% /rom
tmpfs                    29.1M     60.0K     29.0M   0% /tmp
/dev/mtdblock4            1.7M    224.0K      1.5M  13% /overlay
overlayfs:/overlay        1.7M    224.0K      1.5M  13% /
tmpfs                   512.0K         0    512.0K   0% /dev

Disk usage on snapshot:

 OpenWrt SNAPSHOT, r16794-d683175236
 -----------------------------------------------------
root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 3.0M      3.0M         0 100% /rom
tmpfs                    28.3M     56.0K     28.3M   0% /tmp
tmpfs                    28.3M     44.0K     28.3M   0% /tmp/root
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/mtdblock4          832.0K    224.0K    608.0K  27% /overlay
overlayfs:/overlay      832.0K    224.0K    608.0K  27% /

The flash size is 8MB if I understand the following kernel messages correctly:

[    0.341413] printk: bootconsole [early0] disabled
[    0.362738] spi-nor spi0.0: gd25q64 (8192 Kbytes)
[    0.367647] 6 fixed-partitions partitions found on MTD device spi0.0
[    0.374233] Creating 6 MTD partitions on "spi0.0":
[    0.379197] 0x000000000000-0x000000020000 : "u-boot"
[    0.385213] 0x000000020000-0x000000600000 : "firmware"
[    0.394094] 2 tplink-fw partitions found on MTD device firmware
[    0.400217] Creating 2 MTD partitions on "firmware":
[    0.405396] 0x000000000000-0x000000213b16 : "kernel"
[    0.411469] 0x000000213b16-0x0000005e0000 : "rootfs"
[    0.417355] mtd: device 3 (rootfs) set to be root filesystem
[    0.425005] 1 squashfs-split partitions found on MTD device rootfs
[    0.431447] 0x000000510000-0x0000005e0000 : "rootfs_data"
[    0.437890] 0x000000600000-0x000000610000 : "partition-table"
[    0.444725] 0x000000610000-0x000000630000 : "info"
[    0.450495] 0x000000630000-0x000000650000 : "config"
[    0.456514] 0x0000007f0000-0x000000800000 : "art"
[    0.463215] libphy: GPIO Bitbanged MDIO: probed

What is the reason of this reduced overlay size in snapshot?

Edit1: /proc/mtd on snapshot:

root@OpenWrt:~# cat /proc/mtd 
dev:    size   erasesize  name
mtd0: 00020000 00010000 "u-boot"
mtd1: 005e0000 00010000 "firmware"
mtd2: 00213b16 00010000 "kernel"
mtd3: 003cc4ea 00010000 "rootfs"
mtd4: 000d0000 00010000 "rootfs_data"
mtd5: 00010000 00010000 "partition-table"
mtd6: 00020000 00010000 "info"
mtd7: 00020000 00010000 "config"
mtd8: 00010000 00010000 "art"

@ffries has similar issue In this thread.

Edit2: I am aware of the 8/64 recommendation. My intention is to use this device as an bridged AP with different VLANs/SSIDs (no dnsmasq, no firewall and no other whistles). It would be still nice to have LUCI on it. Yeah, this might be already a whistle

slh · May 24, 2021, 10:00pm

0x5e0000 = 6'160'384 bytes, which is around the size needed for a current firmware image with WPA3/ SSL and luci. 8 MB flash isn't a whole lot to begin with, but with rather wasteful flash partitioning by the vendor, this quickly gets too tight.

ZsZs73 · May 26, 2021, 12:45pm

Thanks for your reply. Just realized that snapshots do not contain luci by design, but releases like 21.02-rc1 do.
I could install luci-ssl and still have some space left.

buzzLightbeer · October 12, 2021, 2:42pm

I have the same problem here with 21.02.0. The rootfs_data only has 348.2kB of free space.

root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 3.5M      3.5M         0 100% /rom
tmpfs                    28.6M      1.1M     27.5M   4% /tmp
tmpfs                    28.6M     68.0K     28.5M   0% /tmp/root
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/mtdblock4          576.0K    236.0K    340.0K  41% /overlay
overlayfs:/overlay      576.0K    236.0K    340.0K  41% /

root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00020000 00010000 "u-boot"
mtd1: 005e0000 00010000 "firmware"
mtd2: 001f3f86 00010000 "kernel"
mtd3: 003ec07a 00010000 "rootfs"
mtd4: 00090000 00010000 "rootfs_data"
mtd5: 00010000 00010000 "partition-table"
mtd6: 00020000 00010000 "info"
mtd7: 00020000 00010000 "config"
mtd8: 00010000 00010000 "art"

[    0.346935] Creating 6 MTD partitions on "spi0.0":
[    0.351906] 0x000000000000-0x000000020000 : "u-boot"
[    0.357979] 0x000000020000-0x000000600000 : "firmware"
[    0.366693] 2 tplink-fw partitions found on MTD device firmware
[    0.372856] Creating 2 MTD partitions on "firmware":
[    0.378003] 0x000000000000-0x0000001f3f86 : "kernel"
[    0.383985] 0x0000001f3f86-0x0000005e0000 : "rootfs"
[    0.390007] mtd: device 3 (rootfs) set to be root filesystem
[    0.397531] 1 squashfs-split partitions found on MTD device rootfs
[    0.403983] 0x000000550000-0x0000005e0000 : "rootfs_data"
[    0.410497] 0x000000600000-0x000000610000 : "partition-table"
[    0.417315] 0x000000610000-0x000000630000 : "info"
[    0.423146] 0x000000630000-0x000000650000 : "config"
[    0.429171] 0x0000007f0000-0x000000800000 : "art"

I think one problem lies in the unused address range from 0x5e0000 to 0x60000 and essentially 0x65000 to 0x7f0000. Is there any way that this can be made usable?

leeandy · October 13, 2021, 10:31pm

Yep, you're corrected. From this commit we see wrong partition size & mapping defined:

+                       partition@20000 {
+                               label = "firmware";
+                               reg = <0x020000 0x5e0000>;
+                       };
+
+                       partition@600000 {
+                               label = "partition-table";
+                               reg = <0x600000 0x010000>;
+                               read-only;
+                       };
+
+                       info: partition@610000 {
+                               label = "product-info";
+                               reg = <0x610000 0x020000>;
+                               read-only;
+                       };
+
+                       partition@630000 {
+                               label = "config";
+                               reg = <0x630000 0x020000>;
+                               read-only;
+                       };
+
+                       art: partition@7f0000 {
+                               label = "art";
+                               reg = <0x7f0000 0x010000>;
+                               read-only;

buzzLightbeer · October 14, 2021, 4:53am

I had a look in the original TP-Link .bin-File. The current flash layout seems to be just copied from the original which also lacks the same area after 0x650000. I'm unfamiliar with creating OpenWRT images, but is there any way for a sysupgrade image to keep the data at 0x600000 and put the rootfs_data after 0x650000? Is it even safe to use that space und is there any 'hidden' data in the flash? It would be ideal if both areas, the 576kb plus 1,6MB could be joined.

leeandy · October 14, 2021, 5:57am

Ping author or commiter to fix this issue.

rhododendron · January 28, 2022, 6:50pm

Is the author still active, I only see two commits from them?

xman · January 29, 2022, 10:18pm

Does not look like this issue can or will be resolved. I was able to use 19.X sysupgrade images, but with 21.X to create a dumb AP I have to use the custom image builder - with few packages stripped and ssl hostapd added I have about 500K of overlay space left ( with a full luci interface ).