Minimizing flash writes with overlayfs

Hi all

I'm working on a hotplug script for adblock-lean and I'm thinking how to minimize flash writes when that script needs to later be removed and/or re-added.

My current assumption is that moving that file (say from /etc/hotplug.d/block/99-abl-hotplug.sh to /etc/hotplug.d/block/.99-abl-hotplug.sh) and later moving it back writes less bytes to flash than deleting the file and later creating it again.

Is this assumption correct?

The question is mainly addressed to people who know the inner workings of overlayfs. Please only informed answers.

Thank you!

Like here: https://github.com/search?q=repo%3Atorvalds%2Flinux+path%3A%2F^fs\%2Foverlayfs\%2F%2F++write+OR+sync+OR+update&type=code

Thanks for the link but I can't really analyze that code.

Well I'm not deeply steeped in the inner workings of overlayfs, but I have been doing similar and not encountered flash wear issues.

In my opinion, the answer to your question is yes, your assumption is correct IF:

  1. You created the file, in the first place, directly onto overlay (ie you did not build it into rom with Imagebuilder, firmware-selector et al)
  2. You use the mv command ( NOT the cp command followed by a delete - yes, obvious for Linux-ers, but not necessarily for Windows-ers).

mv just updates the directory entries as long as the original is on overlay and not rom. If it is on rom, it will up-write the entire file from rom to overlay... I think.

1 Like

I read a doc about overlayfs in the meantime and looks like this is correct. Moreover, to my understanding, once the file migrates from the 'lower' into the 'upper' filesystem, further mvs will only update the filesystem entry as well. Still not 100% sure as none of that is written explicitly.

Neither am I, but I have not seen an issue - at least not yet (some old systems still running that do the mv once every restart, so very little writing, 3 or 4 times per year most likely.

Later versions keep the file on /tmp using ln to create a link, so no longer an issue for that use case.

1 Like

Do you mean something like

ln -s /etc/hotplug.d/block/99-smth.sh /tmp/actual-99-smth-file.sh

?
What happens if the file on /tmp/ is removed but the symlink remains? Does the hotplug system tolerate this? Does it just skip the symlink file when it can't be resolved?

Yes, pretty much.

I've not tried it with a hotplug script. It will probably syslog an error...

I'll maybe try a few things over a coffee if I get a minute....

It would be the other way round:
ln -s /tmp/actual-99-smth-file.sh /etc/hotplug.d/block/99-smth.sh

ie it makes a link from /tmp/actual-99-smth-file.sh to /etc/hotplug.d/block/99-smth.sh

What I was doing was creating the file on /tmp with the package startup script.
The symlink stays as a link file on flash. The link is valid if the /tmp file is present, or gives "No such file or directory" if it doesn't.

I don't know how this will fit into your use case.... but once the link file is created, there are no more writes to flash.

1 Like

Ye, I messed up the order.