How to setup filesystem cache to make it less frequently flushing data to Flash?

For Developers
1

Hi,

I have several programs calling write file operation to the storage (Flash), some in every 5 seconds, some randomly in log writing. I believe normally, the flush operation simply invokes the OS write operation, which places the data in the filesystem cache to write it to the Flash later. The OS may combine multiple such write operations in the cache before the data is actually written to the Flash.

I am concerning the the flash media health, too frequently writing will reduce Flash life span.

I think either I can control the write operation in each of my application in buffer then flushing to filesystem write operation manually, or it can be controlled in file system if I can tune the filesystem cache larger enough to make it less frequently flushing data to the Flash which is a much desire way.

Appreciate any tips how frequently the OpenWrt file system default flushing write operation to the Flash and how to increase filesystem cache to reduce flushing rate to protect Flash health.

Thank you.

Kind regards.

JHH

2

You're right, flash isn't designed for that kind of repeated writing.

Several options come to mind:

  1. Use a USB "stick" which typically are a bit better for repeated writes and the better ones manage "wear leveling"
  2. Use an SSD in a USB enclosure. These days a good quality, 250 GB SSD can be had for about US$50, smaller cheap ones for around US$20, and US$10-20 for an enclosure
  3. Use a network filesystem
  4. Use syslog or MQTT for remote logging of the information
  5. Write to a local, memory-backed file system, then periodically write that data to flash (allowing you "full control" of the balance between data loss and flash write rate)
3

Thanks Jeff, I guess the OpenWrt filesystem does not have the mechanism for write operation to hold the data in filesystem cache, then flushing to the flash when the cache is full. It is very low budget device, I'll take the option 5 as you suggested.

Thank you very much.

4

My two cents:

  • Use a USB memory, the larger the better (more space for the wear leveling mechanism).
  • Use a specific filesystem such as F2FS.
5

One question in this context: is anyone aware of USB sticks that definitely do wear levelling? AFAIK most sticks don't. f2fs has shown unreliable when using as a boot medium, so I went back to ext4 for boot medium and for things like "asterisk" which frequently write to a database I decided to put the database on an additional USB hdd (rotational version, not SSD). USB stick with built-in wear levelling would be better though, as it's quieter and consumes less energy...

with curious regards

codefish

6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.