Head Scratch - EXT4 vs. Squashfs expansion & RAM use

Back in 2020, I bought this NanoPI R2S as a curiosity and had serious issues with this device slowing down to a crawl upon a couple of reboots after a fresh snapshot install, no matter what chip I used, so I ended up putting it aside in the junk pile, thinking it was defective hardware.

Then on Easter Friday, I was on holiday and was wondering if maybe I could take a fresh look at this device. I installed current SNAPSHOT r16383-ec6293febc ext4-sysupgrade and I am happy to say the device seems quite stable, not showing any issues I had previously, so I guess the many firmware versions at the time were the issue.

So basically, this thing was raised out of the dead junk pile brought back to life and resurrected on Good Friday, things that make you go hummm...

I do however have another issues, kind of a head scratcher...

The first chip I used EXT4 version, installed basic stuff, leaving 85% or 91.8MB free storage and roughly between 50-55MB RAM used.

Now, I was wondering why 2 file systems were available for this device, surely there would be reasons for that. So I did some research and found that squashfs can be sysupgraded, while ext4 cannot. Also, squashfs partitions can not be expanded, while ext4 can. Maybe I am wrong about both assertions, please correct me if so.

So I installed SNAPSHOT r16383-ec6293febc squashfs on another chip and here's where I start scratching my head, this one has 34% or 35.5MB free storage and roughly 75-80MB RAM used. There seems to be about 25MB total RAM more cached and buffed here.

The extra used RAM doesn't bother me here, although would appreciate knowing why, however the 35MB free software storage really bothers me, as I intent to push this device way more than the EA8300 and MR8300 I usually use that have close to 65MB free storage space.

Any input as to why the difference between both basic setups on different fs and how to make more free space on squashfs or expand partition, as ext4 expansion not the issue here.

Also, about sysupgrade on ext4 not being possible, is it still an issue ?

forgive my assumptions, but i assume 'chip' is referring to some nanopi r2s?

the reason you have less free ram/storage on a squashfs with a mounted file system that is read write (in this case ext4) is because the mounted system caches some things in your ram-- this is normal.

on my DIR 882 there is approximately a few MBs of ram consumed for a mounted ext4 usb key that is mounted (small partitions).

if you type

echo 3 > /proc/sys/vm/drop_caches

and then use the device as normal, you should be able to compare both 'chips' and, if the installations are identical, their consumption should be the same. if you use the mounted file system, expect there to be less 'free' ram as it will used to cache and buffer programs you are running off it.

squashfs is a read-only file system by design. it is a fantastic and ingenious solution for embedded devices. it is supposed to offer high compression and performance. when programs are executed off the squashfs, it is specifically designed to efficiently page memory to run the program. i haven't read too deeply into it, but i have a respect for what it does and its purposes.

you do not compare squashfs to a 'typical' file system that is deployed on machines with larger resources. ext4 isn't designed for the same purposes as squashfs.

1 Like

By Chip I mean a microSD non-volatile storage installed as boot, since R2S no onboard storage other than uSD slot. As such if I take one uSD with freshly loaded EXT4 and another with freshly loaded squashfs, these are the differences I see once basic identical config is performed.

Also, considering huge uSD size, we're talking 32GB, I'm surprised there is no attempt at expanding fs on initial install boot, hence my questions about expanding squashfs...

squashfs isn't made to expand. it may be loaded through a non-volatile storage, but it's designed to use memory (efficiently).

when you squash sits on your non-volatile storage, it is compressed. at boot the kernel will unpack and 'mount' this 'blob' (hence the squashfs kernel option), where it then sits in memory. since it is a read-only file system, this reduces any kind of cache thrashing or fragmentation as you're not able to write to the system.

looking here, i note you have 1gb. it makes sense that openwrt offers an ext4 version because even if the ext4 needs memory for buffering, 1gb is more than sufficient.

in your situation, ext4 vs squashfs for 1gb ram is purely a choice. i am curious if the ext4 system is read/write after boot? i assume so?

many devices do not have the luxury of 1gb ram or even 512. 256 is becoming popular but mine only has 128, and for both i'd use squash. maybe even for 512. for 1gb i could see ext4 being realistic.

tldr: it's a choice, you have 1gb ram and as you observed there are tradeoffs: the squashfs is read-only whereas the ext4 seems to be read/write and also expandable. the performance penalty will be negligible (if it is even observable, at which point i'd argue the performance margin is a random outcome)

1 Like

I've decided to go with EXT4 because it's expandable, therefore the default 92MB free storage can always be expanded if need be. I just backup, setup another uSD, load modules, restore backup and voilà, fully upgraded to newest version.

As I do have huge RAM size, I really don't care about extra RAM usage with squashfs, my problem is I would like to use squashfs for it's R/O benefits, however it only leaves 35MB free storage on a large 32GB uSD. Plus, I could syupgrade without new uSD setup, which can't be done with EXT4.

So how do I get more free storage with squashfs ?

that's a good question, and i don't have an answer for that.

personally i'm surprised to learn that the squash is expanding that much. it shouldn't be that big.

maybe you spotted a bug. i know when i use a squashfs for the 256mb dir2640, it used the exact same amount of ram as my DIR882, so it had an extra 128MB free. and for the nand, it was only using however much was written.

the squash shouldn't be that big. are you sure you aren't mistaken in reading the free space? could you post a screenshot? if you are observing this, it's possible you may have found a bug (congratulations!) and have to file a ticket.

it may be that the squash on your sd card is being expanded into memory and then, for some reason, is being resized to consume your entire SD card

It's not about the squash expanding to only leave that little free.

The partitions are 32MB unallocated then 16MB healthy then 16MB unallocated then 104Mb healthy Primary then 29.56GB unallocated, as per Windows partition manager.

With all this free space........

Works fine for me, your info source is likely outdated.

https://openwrt.org/docs/guide-user/additional-software/extroot_configuration

2 Likes