I want to calculate how much space does a file take up in a compressed filesystem.
The problem is a bit confusing but I will try to explain it with an example:
I have a device with around 3MB of physical flash space.
Checking the filesystem space usage with df outputs that the overlayfs:/overlay filesystem has around 3MB of available space.
root@XXX:/usr/bin# df
Filesystem 1K-blocks Used Available Use% Mounted on
overlayfs:/overlay 3264 2136 1128 65% /
I can have a file of size 10MB inside this filesystem because it is compressed, using du <FILENAME> outputs that it takes 10MB of space.
root@XXX:/# du -sh 10MB_FILE
10.0M 10MB_FILE
Question: is there a way to see how much space does this 10MB file take up in relation to 3MB compressed filesystem? I can get an approximation of this number by checking how much space is left on this compressed filesystem before and after adding this file but this method is not feasible for me.
That's a wrong assumption. The overlay is uncompressed, hence the standard advice to include any extra packages you need into the image you flash. Only the read-only Squashfs root is compressed.
Nevermind me, overlay is still compressed, just not as much as the read-only SquashFS. Thanks @jow.
In that case, how is it possible to have files of bigger size than shown with df? I understand that it is best to include extra packages into the image but my question is related to installing additional packages with opkg. Lastly, I'm just trying to get a better understanding of the filesystems on OpenWRT.
I will try to show it with another example. Here is a sample filesystem of my device:
Now I check again the device's filesystem and size of the file:
root@device:/usr/bin# df 10MB_FILE
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mtdblock6 3264 2456 808 75% /overlay
root@device:/usr/bin# du -sh 10MB_FILE
10.0M 10MB_FILE
root@device:/usr/bin# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 10688 10688 0 100% /rom
tmpfs 62416 172 62244 0% /tmp
/dev/mtdblock6 3264 2456 808 75% /overlay
overlayfs:/overlay 3264 2456 808 75% /
tmpfs 512 0 512 0% /dev
/dev/mtdblock7 576 252 324 44% /log
Now du shows that it still is of size 10MB yet the filesystems /dev/mtdblock6 and overlayfs:/overlay used space increased around 28% or around 272KB(2456-2184=272). I'm assuming that the new file takes up around 272KB of the compressed filesystem's space. This confuses me and that is the reason I have asked this question. Maybe I am missing some point here?
I have tried reading about filesystems on OpenWRT wiki but I still don't quite understand how this works.
There is basically the same question asked elsewhere https://stackoverflow.com/a/31622594/14984008. I wanted to ask if there is any way to calculate an approximate size a file takes up in relation to the compressed filesystem or is it pretty much impossible to do that?
How much space the file takes depends on the file's contents. There is no one, single rule for that. If you don't know the contents of the file, then you can't determine the space it will use, either.
Afair the overlayfs is compressed, but not as thoroughly as the readonly squashfs one.
A very very rough approximation would be something like cat file | xz | wc -c on your desktop. The actual storage footprint depends on further factors (data in adjacent sectors, squashfs vs. jffs, specific compression parameters and dictionary size chosen by the FS etc.)