Looking around on the internet, it looks like this would have badly broken in the past; however, it sounds like ASU is much more exroot aware now, is that true?
(cat /etc/config/fstab; block info; mount; wasn’t quite as easy to read but I can attach those or other bits instead if needed?)
I’m fairly sure I’ve got everything that I believe needs to be included in sysupgrade.conf (including an automatically daily generated /etc/installed-packages.txt just in case.)
I have downloaded the generated archive tgz and the ASU custom sysupgrade ready to do a “reinstall” as a test of the process.
Any pointers, advice, or even just reassurance would be gratefully appreciated!
- Julie
PS The OpenWRT is AWESOME! What lovely, geeky, tinker-friendly hardware. I’m loving the classy full aluminium case. And I love that the purchase results in a contribution to OpenWRT too. (And no, I’m not a plant, and they’re not paying me, lol.)
Not as you expect it to, sysupgrade (attended or not) will disable the extroot - and the oversized extroot install should make owut (which will try to build everything into the squashfs) fail.
Do not get too ambituous here, config files - yes, binaries - no (there is no binary compatibility, nor dependency checking, it will explode in your face).
Extroot is a hack, it never works as well as a normal install and all the convenience options are lost. It's a method of last resort, for desperate users - not something to undertake willingly or recommend.
These days you can find cheap (and/or used) devices with 'enough' flash, just say no to extroot and live your life
You can have extra mounted partition in m2 ssd and keep the sysupgrade rolling. I have this setup on raspberry sdcard, maybe to extreme to push it to last half of 16gb SSD, but works with sysupgrade since v19....
Size wise, it’s currently still very small so that bit shouldn’t be an issue? Cross fingers.
The backup tgz plus ASU built image:
julie@Computer:/mnt/router/Other/Backups/Routers and Extenders/Router/OpenWRT One/2025-12-30$ ls -al
total 26336
drwxr-xr-x 2 root root 32768 Dec 30 12:49 .
drwxr-xr-x 5 root root 32768 Dec 30 12:50 ..
-rwxr-xr-x 1 root root 1032354 Dec 30 12:43 backup-Router-2025-12-30.tar.gz
-rwxr-xr-x 1 root root 25821982 Dec 30 12:48 openwrt-24.10.5-565dec8fff56-mediatek-filogic-openwrt_one-squashfs-sysupgrade.itb
With regards to the backup tgz, I’m not daft, no binaries, just the config files where needed, so still nice and small.
(I’m actually familiar with the rough concept, as it’s not totally unlike how I do my minimal-backup-for-future-rebuilding Raspberry Pi system backups – e.g. tracking added packages, and a hand written list of config files that’s then used to turn it into a tarball, plus a self-restoring script which mostly works – nice for refreshing the setup on to a newer OS version.)
So! It looks like by using Exroot I’ve been going about using the NVMe for permanence and extra storage the wrong way?
If so, then I best revert! That’ll be interesting. LOL. In theory, I’ve not done enough yet that the flash would be filled.
Going forward…
It sounds like I’d be best sticking with the normal flash /overlay, and then mounting the spare extra NVMe partition like any other extra filesystem at e.g. /mnt/other?
And then putting symbolic links from the /overlay filesystem into /mnt/other/… for the directories where I want to use that extra storage (e.g. log files and things)?
Ironically, the OpenWRT is actually more than adequate for me – a massive upgrade on what I’ve been using (a firmware hacked DGND3700V1 that would lockup if I let Steam downloads hit the max fibre download rate.)
I only did the wretched Exroot because I could, because I already had a decent 2230 128GB NVMe in the OpenWRT One, of which only 6GB is otherwise in use (one partition for a minidlna database, and a very low priority 4GB swap partition for “just in case.”)
Thanks again.
Wish me luck undoing it all (a job for tomorrow now, I think – pretty sure I’m going to end u get the opportunity to “test” the built in USB C console port, sigh!)
spending quality time using binwalk on the custom build sysupgrade…
which in turn required finding sasquatch sources that would compile on modern gcc versions…
and having an informative rummage throught the initial OpenWRT filesystem and boot scripts and getting a bit of a better idea how the boot works and where the the packaged upgrades live and so on (from mostly zero, my linux knowledge is mostly back in kernel v2.6 days, and it’s not grown much from the rummaging)…
and then having a good old read about owut to learn more about the process…
Note that if your device is configured for Extroot, then you will need to reboot twice after any type of sysupgrade. The external root filesystem does not remount after the first boot. There is no need to modify or recreate extroot settings, just reboot again and the configuration will work as before and continue to do so.
It sounds like it should work after two boots then, if it fits? Which is what I gathered from other sources?
I feel like I should write a post about why extroot fails (in terms of technical issues) to deliver what folks actually want (which is about the same as what I wanted when I wrote the first version of extroot for OpenWrt that made it into the repo).
It's not so much that it is a hack, but that extroot is a thin layer, while to achieve what most folks want from it would need quite a different approach to the OpenWrt userspace. To really have the kind of system I think most folks image with extroot, one would need to create a distro that used the OpenWrt kernels, but otherwise was build more like a standard Linux distro.
sysupgrade for extroot only works if you are able to temporarily disable the extroot, do a normal sysupgrade, and then add the extroot back (and do not try to upgrade and use the packages stored on an old extroot; that is almost certain to go badly).
Actually you can use alt location for opkg packages (kmods for sure and libraries preferably still stay in default place) - see opkg.conf for temp, web package manager does not "see" those packages, you just opkg them after sysupgrade.
Sorry but thats lost in a month towards apk....
You can. And it might even not blow up for you. I wouldn't want to count on it for anything important. 'Tis moot soon anyway, as you write, and just as well.
Exactly this, not so much a fan of excessive symlink farms though.
However, with the OpenWrt One I don't really understand the problem, nor the desire for extroot.
You have 255 MB NAND flash, while I don't really how big the usable firmware space is (255 MB for the ubi container), I would expect it to be >100 MB - it's pretty hard to fill that up with OpenWrt packages (at least those commonly installed). So why not just keep it simple, leave the 'firmware' to itself on the NAND (no extroot necessary) and the volatile data (or containers, VMs, etc.) on NVMe? Easy, upgradeable, wear-a-dn-tear on NVMe, not NAND.
For the devices with removable media (around 50% of the devices on the market at best, declining), a 'better extroot' would be imaginable (only a chainloading bootloader on the flash, just enough to initialize USB (or alternative media) and boot the whole firmware (kernel && rootfs) from there)). The devil however is in the details, as devices differ too much from each other to make this really generic (very few OEM bootloaders can boot from USB, off hand I only remember 2, UEFI support is still rare/ basically non-existent, storage concepts differ, only ARMv8 (and x86_64) really has something equivalent to ARMv8 SBSA/ ARM system ready). A lot of potential, but very little to profit from others taking part in the development (and again, >50% of new devices don't come with USB or similar, so there'd always remain a need for two-tiered development for the same targets, losing synergy effects). kexec might come to mind (which isn't necessarily realistic on 'exotic' platforms either).
The pragmatic choice these days is to buy the hardware that meets your flash/ RAM size requirements and to keep it easy. We're no longer in the WRT-54G days, where the choice was between 4- or 8 MB flash and 16- or 32 MB RAM, devices with 128/ 256 MB flash (careful, not in all cases all of that is usable; usually only half at best, often less (20-40 MB is common)) and 512-2048 MB are among your choices these days (and not all particularly expensive, especially if you include the used markets into your search).
Got a Sysupgrade image with all the current packages via Attended Sysupgrade → Download (and made sure it was going to be small enough to still fit in the flash)
Got a backup tar.gz of the config
Processed the backup tarball to remove etc/config/fstab (hit a GNU tar bug that's still present in v1.35, –delete on uncompressed tar files still corrupts the tarball, it works fine in a pipe though)
Did a sysupgrade using the downloaded custom image with the extra packages without preserving the settings (so there would be no attempt to mount the exroot)
Rebooted to clean, reset config, now running without exroot
Reformatted my old exroot partition, now to be used as general persistent storage
Restored my settings from the hacked tarball (i.e. sans etc/config/fstab)
Reconfigured my mounts
Finally-
Set-up logging to the external partition again, etc
Reviewed which extra config files need to be added to the backup list
Did a backup
Rebooted
Tested everything was working
Fortunately, it all went completely smoothly.
But… why did I exroot in the first place?
Because I could.
The OpenWRT One is way more powerful than I'm used to.
I had a spare 128GB NVMe in there that was mostly unused and I wanted to play with all the bells and whistles.
I wanted to learn, and I made a mistake and went down a path where there be dragons.
Thanks all!
Really informative and helpful responses. Has been a great learning experience.
What lovely, geeky, tinker-friendly hardware. I’m loving the classy full aluminium case. And I love that the purchase results in a contribution to OpenWRT too. (And no, I’m not a plant, and they’re not paying me, lol.)
Urgh.