Wrt1900acs Version 2 Low space on overlay

davidc502 · July 18, 2019, 9:17pm

See below... We have a user @slim0287 who owns the 1900acs V2, and his / overlay has 9 MB of total space. This seems strange to me because the acs V2 has 128MB of flash RAM. I realize the 128 is kind of split between the 2 partitions, but 9MB for / seems small allocation. I think the size allocations have been like this for a while, but what you are seeing below is for r10525.

Filesystem Size Used Available Use% Mounted on
/dev/root 18.5M 18.5M 0 100% /rom
tmpfs 249.5M 3.3M 246.2M 1% /tmp
/dev/ubi0_1 9.0M 4.8M 3.7M 57% /overlay
overlayfs:/overlay 9.0M 4.8M 3.7M 57% /
ubi1:syscfg 29.6M 368.0K 27.7M 1% /tmp/syscfg
tmpfs 512.0K 0 512.0K 0% /dev
/dev/sda1 28.1G 6.7G 19.9G 25% /mnt/sda1
/dev/ubi1_0 29.6M 368.0K 27.7M 1% /mnt/ubi1_0

Comparatively speaking the 3200ACM has 256MB of Flash, but on my copy I have 3 times the space on / overlay.

3200ACM
overlayfs:/overlay 42.6M 4.6M 35.8M 11% /

Anyhow, is there any reason why the V2 would have only 1/3 the space of a 3200acm, on overlay?

Thanks,

David

slh · July 18, 2019, 9:38pm

github.com

torvalds/linux/blob/master/arch/arm/boot/dts/armada-385-linksys-cobra.dts#L74


&gpio_leds {
	power {
		label = "cobra:white:power";
	};


	sata {
		label = "cobra:white:sata";
	};
};


&nand {
	/* 128MiB */
	partitions {
		compatible = "fixed-partitions";
		#address-cells = <1>;
		#size-cells = <1>;


		partition@0 {
			label = "u-boot";
			reg = <0x0000000 0x200000>;  /* 2MiB */
			read-only;

Suggests a similar partitioning scheme as for the other Linksys mvebu devices, 2*(6+34) MB for kernel and rootfs, but the wrt3200 offers twice that much space:

github.com

torvalds/linux/blob/master/arch/arm/boot/dts/armada-385-linksys-rango.dts#L82


	wlan_5g {
		gpios = <&gpio1 14 GPIO_ACTIVE_LOW>;
		label = "rango:white:wlan_5g";
	};
};


&gpio_leds_pins {
	marvell,pins = "mpp21", "mpp45", "mpp46", "mpp56";
};


&nand {
	/* AMD/Spansion S34ML02G2 256MiB, OEM Layout */
	partitions {
		compatible = "fixed-partitions";
		#address-cells = <1>;
		#size-cells = <1>;


		partition@0 {
			label = "u-boot";
			reg = <0x0000000 0x200000>;  /* 2MiB */
			read-only;

namely 2*(6+74) MB.

Looking into your figures, you currently use 18.5 MB for the rootfs (which is quite a lot for a default'ish image) leaving 9 MB for the overlay, amounting to 27.5 MB, while I'm not that familiar with the overhead and reserved sectors for wear leveling under ubi/ ubifs, that sounds about right (albeit slightly much).

davidc502 · July 18, 2019, 10:08pm

Much appreciated... It just seemed weird that I have more than 3 times the overlay space but only twice the total flash. If using the same partitioning scheme, it just seemed to me the V2 would have 18MB-ish .

I guess that's what I get for thinking again

slh · July 18, 2019, 10:11pm

ubinfo -a might provide closer insight.

tapper · July 18, 2019, 11:22pm

Hi if people are running out of space why don't you remove some packages from your builds. That is why I stopped using them. They are good, but you cant use SQM and QOS at the same time and why do you ship with both iperf - 2.0.13-1
and iperf3 - 3.7-1?
If you have a packages folder on your server you could let people install extras that they mite need at a later date like IGMP.proxy and samba. I did injoy your builds and you do a lot for the community. I thank you for that! I am not trying to be a ass.

davidc502 · July 18, 2019, 11:40pm

Much appreciated @tapper. Who knows maybe this is a reason to cut back on packages, and I didn't realize 2 iperf's were included, so there is a start. Over the years you add a package here and a package there, and before you know it........ Still to this day there are people asking for more packages to be included, but I've had to hold firm and give a diplomatic answer. First I'm going to go down the road to figure out why V2 is different.

slh · July 18, 2019, 11:47pm

If you'd set CONFIG_TARGET_PER_DEVICE_ROOTFS=y, you could customize your package lists on a per-device level, distinguishing between devices with large- or small flash.

E.g.: (this is only a partial config snippet, based heavily around custom meta-packages):

### ath79 specifics
CONFIG_TARGET_ath79=y
CONFIG_TARGET_ath79_generic=y
CONFIG_TARGET_DEVICE_ath79_generic_DEVICE_buffalo_wzr-hp-ag300h=y
CONFIG_TARGET_DEVICE_PACKAGES_ath79_generic_DEVICE_buffalo_wzr-hp-ag300h="slh-misc-16m roen-misc-meta kmod-usb-ohci strongswan-slh uboot-envtools -wpad-basic -wpad-mini -wpad wpad-openssl"
CONFIG_TARGET_DEVICE_ath79_generic_DEVICE_tplink_tl-wdr3600-v1=y
CONFIG_TARGET_DEVICE_PACKAGES_ath79_generic_DEVICE_tplink_tl-wdr3600-v1="slh-misc-8m slh-misc-qos strongswan-slh -wpad-basic -wpad-mini -wpad wpad-openssl"
CONFIG_TARGET_DEVICE_ath79_generic_DEVICE_tplink_tl-wdr4300-v1=y
CONFIG_TARGET_DEVICE_PACKAGES_ath79_generic_DEVICE_tplink_tl-wdr4300-v1="slh-misc-8m slh-misc-qos strongswan-slh -wpad-basic -wpad-mini -wpad wpad-openssl"
CONFIG_TARGET_DEVICE_ath79_generic_DEVICE_tplink_tl-wr1043nd-v1=y
CONFIG_TARGET_DEVICE_PACKAGES_ath79_generic_DEVICE_tplink_tl-wr1043nd-v1="slh-misc-8m strongswan-slh -wpad-basic -wpad-mini -wpad wpad-openssl"
CONFIG_TARGET_DEVICE_ath79_generic_DEVICE_tplink_tl-wr941-v2=y
CONFIG_TARGET_DEVICE_PACKAGES_ath79_generic_DEVICE_tplink_tl-wr941-v2="-opkg -wpad-basic -wpad-mini wpad -wpad-openssl"

### enable strongswan (IPsec)
CONFIG_PACKAGE_strongswan-slh=m

### don't build deprecated ciphers for openssl
# CONFIG_OPENSSL_WITH_DEPRECATED is not set

### Enable per device rootfs
CONFIG_TARGET_MULTI_PROFILE=y
CONFIG_TARGET_PER_DEVICE_ROOTFS=y

### custom meta packages, https://github.com/pkgadd/owrt-feed-pkgadd/
CONFIG_PACKAGE_slh-misc-16m=m
CONFIG_PACKAGE_slh-misc-8m=m
CONFIG_PACKAGE_slh-misc-luci-statistics=m
CONFIG_PACKAGE_slh-misc-qos=m
CONFIG_PACKAGE_roen-misc-meta=m
[…]

(you need to build all packages you might want to include on a per-device basis with =m, they are not implicitly being marked for building just by including them into TARGET_DEVICE_PACKAGES).

davidc502 · July 18, 2019, 11:54pm

I was thinking that if this potential issue could be fixed then I wouldn't want to just provide it on my builds. The community should be aware and the change should be made so that all who download Openwrt with V2 hardware gets the new configuration.

Right now I just want to get to the root of why there is 9M of overlay allocated, and can or should it be shuffled around a bit to provide more space so owners can install more packages if they want to.

1st, I've never peered deeply into the code and how it is configured to build out space like it does, so I will be looking to learn more about how that happens.

slim0287 · July 19, 2019, 9:07pm

All,
I'm the user david was referring to in the original post regarding/overlay space. I don't know if it matters, however my router is a WRT1900ACS V2, not WRT1900AC V2 as David wrote in the original post. So, if this changes any of this conclusions here please let us know.

Eric

slim0287 · July 19, 2019, 9:19pm

All

David also asked me to run ubinfo -a and post the results. Here they are:

root@OpenWrt:~# ubinfo -a
UBI version:                    1
Count of UBI devices:           2
UBI control device major/minor: 10:59
Present UBI devices:            ubi0, ubi1

ubi0
Volumes count:                           2
Logical eraseblock size:                 126976 bytes, 124.0 KiB
Total amount of logical eraseblocks:     272 (34537472 bytes, 32.9 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       0
Count of reserved physical eraseblocks:  20
Current maximum erase counter value:     24
Minimum input/output unit size:          2048 bytes
Character device major/minor:            249:0
Present volumes:                         0, 1

Volume ID:   0 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        129 LEBs (16379904 bytes, 15.6 MiB)
State:       OK
Name:        rootfs
Character device major/minor: 249:1
-----------------------------------
Volume ID:   1 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        119 LEBs (15110144 bytes, 14.4 MiB)
State:       OK
Name:        rootfs_data
Character device major/minor: 249:2

===================================

ubi1
Volumes count:                           1
Logical eraseblock size:                 126976 bytes, 124.0 KiB
Total amount of logical eraseblocks:     296 (37584896 bytes, 35.8 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       8
Count of reserved physical eraseblocks:  12
Current maximum erase counter value:     74
Minimum input/output unit size:          2048 bytes
Character device major/minor:            248:0
Present volumes:                         0

Volume ID:   0 (on ubi1)
Type:        dynamic
Alignment:   1
Size:        280 LEBs (35553280 bytes, 33.9 MiB)
State:       OK
Name:        syscfg
Character device major/minor: 248:1
root@OpenWrt:~#

davidc502 · July 19, 2019, 9:28pm

Much appreciated for the correction... I made the correction in the original description and title. Still 128MB flash, but important to get right, so my fault on that.

slh · July 21, 2019, 7:30am

Trying to shed some light into it, as I have only limited experience with NAND myself, I'm taking the BT Business Hub 5 Type A (lantiq) for comparison:

github.com

openwrt/openwrt/blob/master/target/linux/lantiq/files/arch/mips/boot/dts/BTHOMEHUBV5A.dts#L266


			};
			partition@a0000 {
				label = "uboot-env";
				reg = <0xa0000 0x20000>;
				read-only;
			};
			partition@c0000 {
				label = "unused";
				reg = <0xc0000 0x40000>;
			};
			partition@100000 {
				label = "ubi";
				reg = <0x100000 0x7e80000>;
			};
			/*
			 * last 512 KiB are for the bad block table, not writable
			 */
		};
	};
};

[    0.518283] nand: device found, Manufacturer ID: 0x01, Chip ID: 0xf1
[    0.523280] nand: AMD/Spansion S34ML01G1
[    0.527187] nand: 128 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
[    0.535262] Bad block table found at page 65472, version 0x01
[    0.541274] Bad block table found at page 65408, version 0x01
[    0.546349] nand_read_bbt: bad block at 0x000001fc0000
[    0.551361] nand_read_bbt: bad block at 0x000002600000
[    0.556531] nand_read_bbt: bad block at 0x000003c60000
[    0.561764] 4 fixed-partitions partitions found on MTD device 14000000.flash
[    0.568686] Creating 4 MTD partitions on "14000000.flash":
[    0.574178] 0x000000000000-0x0000000a0000 : "u-boot"
[    0.581290] 0x0000000a0000-0x0000000c0000 : "uboot-env"
[    0.587246] 0x0000000c0000-0x000000100000 : "unused"
[    0.592932] 0x000000100000-0x000007f80000 : "ubi"


root@bthub5:~# cat /proc/mtd 
dev:    size   erasesize  name
mtd0: 000a0000 00020000 "u-boot"
mtd1: 00020000 00020000 "uboot-env"
mtd2: 00040000 00020000 "unused"
mtd3: 07e80000 00020000 "ubi"

(yes, I do have 3 bad blocks, at 128 KB each.)

0x7e80000 bytes = 126.5 MB

Let's take a look at the usable space (kernel 2.1 MB, rootfs 6.7 MB):

root@bthub5:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 6.8M      6.8M         0 100% /rom
tmpfs                    60.0M     84.0K     60.0M   0% /tmp
/dev/ubi0_2             104.0M    180.0K     99.1M   0% /overlay
overlayfs:/overlay      104.0M    180.0K     99.1M   0% /
tmpfs                   512.0K         0    512.0K   0% /dev

 126.5   MB (ubi)
-  0.768 MB (bad blocks)
-  2.1   MB (kernel)
-  6.7   MB (rootfs)
-  0.180 MB (overlay contents)
 ==========
 116.752 MB (free space, at least mathematically, ignoring padding, overhead, etc.)

actually free space as reported by df -h, 99.1 MB - so ~17.652 MB (~13,95%) 'unaccounted' for, probably 'lost' for overhead and wear leveling.

ubinfo -a seems to confirm these rough calculations:

root@bthub5:~# ubinfo -a
UBI version:                    1
Count of UBI devices:           1
UBI control device major/minor: 10:58
Present UBI devices:            ubi0

ubi0
Volumes count:                           4
Logical eraseblock size:                 129024 bytes, 126.0 KiB
Total amount of logical eraseblocks:     1009 (130185216 bytes, 124.1 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       3
Count of reserved physical eraseblocks:  17
Current maximum erase counter value:     21
Minimum input/output unit size:          2048 bytes
Character device major/minor:            252:0
Present volumes:                         0, 1, 2, 3

Volume ID:   0 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        17 LEBs (2193408 bytes, 2.0 MiB)
State:       OK
Name:        kernel
Character device major/minor: 252:1
-----------------------------------
Volume ID:   1 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        54 LEBs (6967296 bytes, 6.6 MiB)
State:       OK
Name:        rootfs
Character device major/minor: 252:2
-----------------------------------
Volume ID:   2 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        916 LEBs (118185984 bytes, 112.7 MiB)
State:       OK
Name:        rootfs_data
Character device major/minor: 252:3
-----------------------------------
Volume ID:   3 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        1 LEBs (129024 bytes, 126.0 KiB)
State:       OK
Name:        caldata
Character device major/minor: 252:4

Taking padding into account, the overhead delta changes only slightly from ~17.652 MB (~13,95%) to 13,42 MB (~11,93%).

Looking at this for comparisons, the values you see for the wrt1900acs v2 might be reasonable.