Is there a way to "commit" the overlay state to firmware?

My MT76x8-based device has 16M of NOR flash storage. During operation, I expect to install updates via opkg regularly. I would like to be able to, at certain points, "commit" the full state of the overlay to the firmware so that

  • a factory reset takes it back to that point
  • the upgraded packages don't take up extra space

I realise that the point of a factory reset might be to get out of a bad state caused by a bad package upgrade, so I am not criticising the existing design, but in this specific case there may well be cases where I want to snapshot a known-to-be-good point.

There is the sysupgrade tool, but it requires me to build the full image with the upgraded packages and send the relatively large image (compared to the size of single packages) to each device.

I understand the theory enough to build my own tool that, perhaps, builds a combined firmware image in RAM (eg. on /tmp) and flashes it back... but I can also see about a hundred edge cases and things I could get wrong. (And also I don't know how wise it is to do this on the MT76x8 itself.)

So if there's a tool that already does this ie. an in-place, incremental "commit" of the overlay to the flash, and it's been tried and tested, I'd love to know about it!

Build your own image.

5 Likes

In addition to what tmomas said, you can use UCI defaults to preconfigure so it will reset to the settings you'd like. Basically, it will program the settings just like you have them now (check uci show output).

2 Likes

But there is no incremental snapshot capability.
There is just overlayfs with /rom + /overlay.
And they are usually compressed differently, so a modification of the flashed /rom would be a rather large task.

In theory you might be able to "stack" overlayfs to at least to two layers, but that would require some research, manual settings etc., and finally a revamp of the OpenWrt reset procedure itself.

2 Likes

I am. And then I update the packages on it. And then I want to not have to send a fully-built image with the same upgrades that already exist on the device.

Is it possible to set up extroot?
https://openwrt.org/docs/guide-user/additional-software/extroot_configuration
That is, if you have a working USB port, there should be no more problems with free space.

It's possible, but I don't think it'll be cost-effective for this project. The device already has the ability to use an SD card, but saving space is only part of the point; another part is making sure factory resets preserve the package upgrades.

1 Like

Then don't upgrade packages. Always rebuild your image and sysupgrade

3 Likes

Why do you expect to be doing this?

The project I'm working on involves many such devices in operation, in bandwidth-constrained environments. The software running on it will still need to be upgraded from time to time though, so the tradeoff is to upgrade individual packages instead of doing everything via full image reflashes. (The time to actually do the upgrade is also an issue: a couple of minutes for a sysupgrade and reboot vs. a few seconds for a package install, each time.)

The difference in bandwidth usage is significant though, we're talking a few kB vs 10-ish MB per device, per upgrade. (Sorry, I know I didn't make this constraint clear when I asked.)

Isn't factory reset supposed to clear only jffs2@rootfs_data and leave extroot?
https://git.openwrt.org/?p=project/fstools.git;a=blob;f=jffs2reset.c;hb=HEAD#l125
In addition, a small USB/SD flash should cost less than $1.

1 Like

I just came back to this after leaving it for a few weeks, and found this post on the Unix/Linux Stackexchange: Merge changes to upper filesystem to lower filesystem in Linux Overlay (OverlayFS) mount

Specifically it links to some useful OverlayFS tools here. I have not yet had time to test this but it looks promising.

Here is an idea for supporting "snapshotting" or at least some level of "rollback":

  1. Create a seperate but persistent partition with a writable filesystem when you make your firmware, one that will not be erased on reset
  2. When you create a snapshot, all you will do is use uci export for every file you want to support.
  3. In your baked-in UCI defaults, make your last script copy the files from this persistent storage into their corresponding locations in the overlay (you will need to mount that partition first, of course)
    3a. you can even put a flag in the persistent storage, e.g. "restore from backup on reset" that indicates whether or not to perform the copy, for greater control.

If you want to get more sophisticated, instead of using uci export, you could come up with a way to create uci-defaults scripts based on the current config instead, so you can have more granular control over what gets restored, but uci will probably need some support for that added first.

Mount a USB stick as the default overlay, replacing the router's ubi/jffs. Mount the router's ubi/jffs into a new directory such as /overlay2. To make the snapshot, just empty /overlay2 (router) and copy all files from /overlay (stick). When you need to revert to the backup, just remove the stick.
Or make a backup of that stick on a PC.

This is (possibly) for a commercial product, so we can't just add components without a substantial cost. Also...

...for similar reasons, we can't rely on manual operations.

Will this work for things like upgraded opkg packages and the like? Or is it just for UCI's config structure?