Expanding OpenWrt squashfs image (SDcard)

Hello,
The thread Expanding OpenWrt squashfs image? (SDcard) by @hadmut1 was closed, so I started this similar topic for expanding the ext4 overlayFS for a Raspberry Pi.

What works for me is the following outline:

  1. Transfer OpenWRT to SD-card, boot from that SD-card, log into the RaspberryPI
  2. Install additional packages
  3. Delete the second partition and create a new one with the same start sector and the suggested full length of the SD-card
  4. reboot and log in again
  5. resize the ext4 partition while it is mounted
  6. reboot again just to check if everything went well

A bit more in detail:

  1. At the PC:
#unzip the Image
gzip -d openwrt-21.02.0-bcm27xx-bcm2708-rpi-squashfs-factory.img.gz
# insert SD-card and check the name, alternatively check dmesg
lsblk
# copy the Image 1:1 to the SD-card
sudo dd if=openwrt-21.02.0-bcm27xx-bcm2708-rpi-squashfs-factory.img of=/dev/mmcblk0 bs=1M conv=fsync
  1. SSH into the Raspberry Pi:
opkg install kmod-usb-storage kmod-usb-ohci kmod-usb-uhci e2fsprogs fdisk resize2fs
  1. fdisk /dev/mmcblk0 (also see the less finicky way by @SemperEnim below):
    To make it clear what I entered, here the complete session of fdisk. Please note how the first sector remains unaltered:
Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/mmcblk0: 14.99 GiB, 16096690176 bytes, 31438848 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5452574f

Device         Boot  Start    End Sectors  Size Id Type
/dev/mmcblk0p1 *      8192 139263  131072   64M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      147456 360447  212992  104M 83 Linux

Command (m for help): d
Partition number (1,2, default 2): 2

Partition 2 has been deleted.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2): 2
First sector (2048-31438847, default 2048): 147456
Last sector, +/-sectors or +/-size{K,M,G,T,P} (147456-31438847, default 31438847): 

Created a new partition 2 of type 'Linux' and of size 14.9 GiB.
Partition #2 contains a squashfs signature.

Do you want to remove the signature? [Y]es/[N]o: n

Command (m for help): p

Disk /dev/mmcblk0: 14.99 GiB, 16096690176 bytes, 31438848 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5452574f

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1 *      8192   139263   131072   64M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      147456 31438847 31291392 14.9G 83 Linux

Command (m for help): w
The partition table has been altered.
Syncing disks.
  1. reboot and log in again
  2. The device usage and sizes are now as follows (my SD-card is 16 GB in size). The size needs still needs to be adjusted:
df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 4.3M      4.3M         0 100% /rom
tmpfs                   217.9M     52.0K    217.9M   0% /tmp
/dev/loop0               92.7M      2.6M     83.1M   3% /overlay
overlayfs:/overlay       92.7M      2.6M     83.1M   3% /
/dev/mmcblk0p1           63.9M     17.3M     46.6M  27% /boot
tmpfs                   512.0K         0    512.0K   0% /dev

Actually this was easier than expected, simply use resize2fs:

resize2fs /dev/loop0

resize2fs 1.45.6 (20-Mar-2020)
Filesystem at /dev/loop0 is mounted on /overlay; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 120
The filesystem on /dev/loop0 is now 15641408 (1k) blocks long.

And now df -h returns the adjusted size:

df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 4.3M      4.3M         0 100% /rom
tmpfs                   217.9M     56.0K    217.9M   0% /tmp
/dev/loop0               14.5G      3.1M     13.9G   0% /overlay
overlayfs:/overlay       14.5G      3.1M     13.9G   0% /
/dev/mmcblk0p1           63.9M     17.3M     46.6M  27% /boot
tmpfs                   512.0K         0    512.0K   0% /dev
  1. Just reboot and check if everything went well. Done :smiley:

Edit #1: @SemperEnim pointed out below, that there is a less finicky alternative to using fdisk for step 3 ff:

2 Likes

Alternatively instruct image builder how large the partitions are made.

The BASH-script below is also generating SquashFS & F2FS images, which should make the RPi less susceptible to power-loss induced filesystem corruption. Please consider using those images if your RPi does not have a uninterruptible-power-supply:

#!/bin/bash

#Documentation: https://openwrt.org/docs/guide-user/additional-software/imagebuilder

OUTPUT="$(pwd)/images"
BUILDER="https://downloads.openwrt.org/releases/21.02.0/targets/bcm27xx/bcm2708/openwrt-imagebuilder-21.02.0-bcm27xx-bcm2708.Linux-x86_64.tar.xz"
KERNEL_PARTSIZE=128 #Kernel-Partitionsize in MB
ROOTFS_PARTSIZE=4096 #Rootfs-Partitionsize in MB

# download image builder
if [ ! -f "${BUILDER##*/}" ]; then
	wget "$BUILDER"
	tar xJvf "${BUILDER##*/}"
fi

mkdir "$OUTPUT"
cd openwrt-*/

# list all targets for this image builder, consider 'make help' as well
# make info

# clean previous images
make clean

# Packages are added if no prefix is given, '-packaganame' does not integrate a package
sed -i "s/CONFIG_TARGET_KERNEL_PARTSIZE=.*/CONFIG_TARGET_KERNEL_PARTSIZE=$KERNEL_PARTSIZE/g" .config
sed -i "s/CONFIG_TARGET_ROOTFS_PARTSIZE=.*/CONFIG_TARGET_ROOTFS_PARTSIZE=$ROOTFS_PARTSIZE/g" .config

make image  PROFILE="rpi" \
           PACKAGES="kmod-rt2800-usb rt2800-usb-firmware kmod-cfg80211 kmod-lib80211 kmod-mac80211 kmod-rtl8192cu \
                     luci-base luci-ssl luci-mod-admin-full luci-theme-bootstrap \
                     kmod-usb-storage kmod-usb-ohci kmod-usb-uhci e2fsprogs fdisk resize2fs \
                     htop debootstrap" \
            BIN_DIR="$OUTPUT"

Edit #1: Suggest using SquashFS/F2FS. This combination is less susceptible to power-loss and subsequent filesystem-corruption.

I might have simplified the process a bit.

I find using fdisk is a little finicky, i got it to work following your instructions once but i failed multiple times after i deleted the partition and rebooted, it'd go into a boot loop.

However, i find not deleting the partition that comes with it, and expanding it using cfdisk works easier and is simplier. my install step looked more like this

opkg update && opkg install cfdisk resize2f

cfdisk /dev/mmcblk0

resize the bottom-most partition (it typically goes to max size automatically for you)

reboot and just run resize2fs

resize2fs /dev/loop0

Feel free to edit your instructions if you like.

EDIT: small edit needed following my own instructions and it didn't work.

7 Likes

Thank you, I will give it a try myself if I need to reinstall or upgrade that RPi. I edited the post above so others, looking for help might see it immediately.

Take care that crashes and "brutal" power-off may broke your system !
I have this issue for long times and get it work in another way...

I advice that a third partition for overlay may be a better way...
I get some success to automate and also get auto-upgrade : Request for best practice for eMMC devices - #16 by erdoukki

This method still need some tweaks and also a few more tests, but I think it can be generally proposed and will help OpenWrt devices management to be more efficient and simpler !

1 Like

Thanks for the hint. I think the second method I posted, using image builder and squashfs/f2fs with large partitions is a quite power-loss-robust-way. What's your opinion on that one?

The rootfs will be live and any crash will broke it...
Next boot will not get it autorepaired and will give you in a unbootable state !
I have this issue too often on RPi.
You must then repair the uSD from another computer !

The good way is to have a ReadOnly rootfs, then block-mount and overlay will help to expand the storage.
In case of crash or power-fault, the rootfs will work and the overlay will be checked and auto-repaired...
The worst will only be an auto-reboot !

The two methods can be "mixed".
An image customized with image builder, which integrate block-mount and a f2fs overlay as a third partition (which can be resized at first boot) will be a great idea !

1 Like

F2FS is an overlay to the SquashFS. For EXT4 based images you are certainly right and likely that's vulnerable to powerloss-triggered-corruptions.

My understanding is, in case the F2FS-Overlay is damaged it will be checked (and fixed) at startup. Even if that fails, I expected the startup scripts to drop me to failsafe or at least run with just the SquashFS based read-only system.

I am not keen on intentionally corrupting the F2FS just to test it now, but perhaps that would be the way to clarify the behavior / confirm that assumption.

1 Like

You're right...
I do not seen the F2FS overlay from the given link.
But I agree with you it is the best way to handle crashes !

1 Like

I will make it more obvious and edit the post above.

Doesn't work for me unfortunately. Stuck at the resize2fs step.

Tried:

root@OpenWrt:~# resize2fs /dev/loop0
resize2fs 1.45.6 (20-Mar-2020)
resize2fs: Invalid argument while trying to open /dev/loop0
Couldn't find valid filesystem superblock.

Also tried:

root@OpenWrt:~# resize2fs /dev/mmcblk0
resize2fs 1.45.6 (20-Mar-2020)
resize2fs: Resource busy while trying to open /dev/mmcblk0
Couldn't find valid filesystem superblock.

Was able to expand using cfdisk, but stuck at the resize2fs and not able to move forward at this point. Using the ext4 image.

1 Like

Thank you for your post! :slight_smile:

I am using this image for RaspberryPi 4B and I had been trying to expand the /dev/mmcblk0p2 partition using this guide but without much luck.
I would like to just add my experience on resizing using cfdisk, since the Resize option -for some reason- was not available for /dev/mmcblk0p2.

So, these are the steps that I followed based on your post:


  1. ssh in the openwrt device

  2. Install the required packages

    opkg update
    opkg install cfdisk resize2fs
    
  3. Resize the partition

    cfdisk /dev/mmcblk0
    
    • To unlock the Resize option for the last (/dev/mmcblk0p2) partition:
      • Create a New - primary partition in the large free space after the last (/dev/mmcblk0p2) partition (leave default values)
      • Delete this partition you just created
    • Resize the /dev/mmcblk0p2 partition (leave the default values which will use all the unallocated space)
    • Write the changes
    • Quit
  4. reboot

  5. Resize the f2fs file-system

    resize2fs /dev/loop0
    
4 Likes

Wow these instructions were so good.

You're a legend

I've also had luck on some ext4 installs as well with these modified steps:

1. ssh in the openwrt device

2. Install the required packages

	opkg update
	opkg install cfdisk resize2fs tune2fs
	
3. Resize the partition

	cfdisk /dev/mmcblk0
	Resize the /dev/mmcblk0p2 partition (enter desired space)
	Write the changes
	Quit
	reboot

4. remount root as RO (if fails, reboot and remount as ro)

	mount -o remount,ro /

5. Remove reserved GDT blocks

	tune2fs -O^resize_inode /dev/mmcblk0p2
	fsck.ext4 /dev/mmcblk0p2                   (this will probably fail, doesn't seem to effect anything though)
	reboot
	
6. Go to System > File System Check > Click "File System Check"
	This will do some things and then ask to reboot

7. Resize the f2fs file-system

	resize2fs /dev/mmcblk0p2
	
8. Check new root partition size with:

	df -h
	
9. Reboot

Some of these steps might not be needed, but this has worked on x3 CM4 units of mine

2 Likes

Here's what worked for me on OpenWRT 22.03.2 with SquashFS.

1. Flash image to SD as normal with the firmware selector (I added packages to mine)
2. ssh into router
3. "opkg update && opkg install cfdisk resize2fs"
4. "cfdisk /dev/mmcblk0"
5. Create a NEW PRIMARY partition in the free space at the end. Delete this new partition. Select /dev/mmcblk0p2 and RESIZE it. The default entries will take the whole space.
6. Reboot
7. ssh into router
8. "resize2fs /dev/loop0"
9. Success!

I tried to apply the steps for the BPI-R3 but since the SD-Card partition layout significantly differs, I struggle to success. I used the SDCARD.IMG.GZ from

https://firmware-selector.openwrt.org/?version=SNAPSHOT&target=mediatek%2Ffilogic&id=bananapi_bpi-r3

The partition layout is as follows:

/dev/disk3 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *255.9 GB   disk3
   1:           Linux Filesystem                         4.2 MB     disk3s1
   2:           Linux Filesystem                         524.3 KB   disk3s2
   3:           Linux Filesystem                         2.1 MB     disk3s3
   4:                        EFI                         4.2 MB     disk3s4
   5:                        EFI                         33.6 MB    disk3s5
   6:                        EFI                         21.0 MB    disk3s6
   7: CAE9BE83-B15F-49CC-863F-081B744A2D93               109.1 MB   disk3s7

After booting OpenWRT intro recovery from SD and installing cfdisk, expanding the partition mmcblk0p7 and rebooting from SD

opkg update && opkg install fdisk resize2fs cfdisk
cfdisk /dev/mmcblk0

                               Disk: /dev/mmcblk0
             Size: 238.3 GiB, 255869321216 bytes, 499744768 sectors
          Label: gpt, identifier: 5452574F-2211-4433-5566-778899AABB00

    Device              Start         End     Sectors     Size Type
>>  /dev/mmcblk0p1         34        8191        8158       4M Linux filesystem 
    /dev/mmcblk0p2       8192        9215        1024     512K Linux filesystem
    /dev/mmcblk0p3       9216       13311        4096       2M Linux filesystem
    /dev/mmcblk0p4      13312       21503        8192       4M EFI System
    Free space          21504       24575        3072     1.5M
    /dev/mmcblk0p5      24576       90111       65536      32M EFI System
    /dev/mmcblk0p6      90112      131071       40960      20M EFI System
    /dev/mmcblk0p7     131072   499744734   499613663   238.2G unknown
 ┌────────────────────────────────────────────────────────────────────────────┐
 │Partition name: bl2                                                         │
 │Partition UUID: 5452574F-2211-4433-5566-778899AABB01                        │
 │Partition type: Linux filesystem (0FC63DAF-8483-4772-8E79-3D69D8477DE4)     │
 │    Attributes: RequiredPartition LegacyBIOSBootable                        │
 └────────────────────────────────────────────────────────────────────────────┘

     [ Delete ]  [ Resize ]  [  Quit  ]  [  Type  ]  [  Help  ]  [  Write ]
     [  Dump  ]
       Device is currently in use, repartitioning is probably a bad idea.
                      Quit program without writing changes

I checked the storage and nothing seems to make any sense:

df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 5.0M      5.0M         0 100% /rom
tmpfs                   997.9M     64.0K    997.8M   0% /tmp
/dev/mmcblk0p66          91.9M     60.3M     31.7M  66% /overlay
overlayfs:/overlay       91.9M     60.3M     31.7M  66% /
tmpfs                   512.0K         0    512.0K   0% /dev

Though, p6 and p7 are said to be (source) virtual partitions of mmcblk0p5 I am curios as to how to expand the storage properly.

Is this something unique to BananaPis image or indeed common practiced? I tried various attempts over the weekend and nothing seems to make any sense to me. Thanks in advance for you help!

EDIT
I believe I am booting form the wrong device. When I check the SD-Card via lsblk the partition seems to have been expanded correctly:

lsblk 
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
mtdblock0     31:0    0 122.5M  0 disk 
mtdblock1     31:1    0     2M  1 disk 
mtdblock2     31:2    0     3M  0 disk 
mtdblock3     31:3    0   512K  1 disk 
mmcblk0      179:0    0 238.3G  0 disk 
├─mmcblk0p1  179:1    0     4M  0 part 
├─mmcblk0p2  179:2    0   512K  0 part 
├─mmcblk0p3  179:3    0     2M  0 part 
├─mmcblk0p4  179:4    0     4M  0 part 
├─mmcblk0p5  179:5    0    32M  0 part 
├─mmcblk0p6  179:6    0    20M  0 part 
├─mmcblk0p7  179:7    0 238.2G  0 part 
├─mmcblk0p65 259:0    0   4.8M  1 part /rom
└─mmcblk0p66 259:1    0 238.2G  0 part /overlay
ubiblock0_1  254:0    0    28M  0 disk

Best
Mike

so close. stuck at resize2fs. nanopi R2S 22.03.3, using fdisk method here.

resize2fs: Bad magic number in super-block while trying to open /dev/loop0
Couldn't find valid filesystem superblock.

Will format and try again. This time cfdisk route. Will see.

edit: at the end, didn't work for me. tried another card, charger, too.
Will put this aside for now until next time. have fun everyone.

I have problems on my r2s too. Did You solve the problem ?
Did anyone help me with squashfs on sdcard on nanopi r2s ?
Did anyone can write step by step what i need to install ?

nope.
However, I came across another post, about the squashfs overlay was ext4. Something liket that. I'll try to dig that other post here for you.
Planning to try that next time myself.
Hey, report back if that worked for you. :slight_smile:

edit: there you go

edit: I've gone the online firmware selector route. Not exactly what I desire, but achieved similar purpose (more space).

edit: I've successfully expanded my sdcard (R4S) using the above link from @spence.

1 Like