I finally left snapshot and installed a release build, tnx for the hard work developing OpenWRT!
But something unexpected happened. I was on openwrt-21.02-snapshot-r15986-cc51d97200-x86-64-generic-ext4-combined.img and upgraded to openwrt-21.02.0-rc1-x86-64-generic-ext4-combined.img.
sysupgrade recognized the file and made the upgrade and rebooted. For a short time I saw a msg, it seems that because I had increased both partition sizes it decided to rewrite the image into SSD instead of "extracting" the files and saving them.
All my configs were gone, including the hostname. I edited LAN subnet so I could recover access to it using SSH, copied back all config files and installed all softwares again.
It's now back working and this time I didn't resize the partitions, rootfs has 40MB free. I also tested restoring the new backup and it works on VM.
My question is if it's possible avoiding to lose everything on each sysupgrade. Is it inevitable or was it indeed caused by resizing the partitions? If I can't resize them, how should I proceed to use more storage?
Hello.
This is the normal behavior with x64 combined : everything is stored in a single partition (kernel, packages, and settings). If you flash another version, the partition is rewritten, and so settings are erased. To keep the settings, you must use the squashfs version intead of the combined.
I suggest that you use the image builder to create an image containing all the packages you need. Flash it, setup the router, than save the config into a file. After the next flash, you just need to restore the config from the file. It's fast and effective.
Is there a tutorialt eaching how to do it properly?
Will I need to dd the file into HD, then create a third partition to store installed packages and config files? Then when sysupgrade it will rewrite those 2 partitions and leave the 3rd intact?
After flashing on the SSD, you can resize the partition. Reinstall the SSD into the router, start. Restore the config file and reboot. All should be working immediatly.
You can even do this with a USB stick in order to test : flashing, resizing, restoring config. In case of error, just reflash the USB stick. I have tested many snapshots this way.
I have tested both squashfs and combined, and found combined more interesting.
Here is small tutorial to make tests.
dd the x64 combined to an USB stick, resize the kernel partition. Boot on the USB stick. Configure the router to your will (packages, config), than save the config. You can do this each day with a snapshot if you like !
Once you have get satisfaction (packages, config), dd the image on the SSD, restore the config file for a definitive use.
I would recommend that you use the image builder to create your own image, so the image will directly includes the packages. After dd and resizing, you will only need the restore to config and reboot.
What do u mean? Extract an image from the USB and write it to HD?
Once I have installed a release version on SSD, I only need to sysupgrade when a new release version is released. I don't wanna keep using snapshots.
The problem is that when I sysupgrade it will delete all installed apps and reset configs. Isn't there a way to do sysupgrade and keep them?
For testing I use VMware. For installing I boot with Parted Magic, cp the img from network and install it. SSD is NVMe, it's not practical nor needed to keep taking it off.
If sysupgrade writes the image directly to SSD, maybe squashfs is better, if it keeps rootfs as readonly and lets us install apps and save configs on a third partition. But then, do we need to create this partition after dding it? And won't sysupgrade mess with this partition?
There is no way to keep installed packages with default images. There are scripts or commands floating around on the forum to remedy that. You could also use the image builder to repackage your own image and include all the packages you use. Similar result, but a cleaner approach if you ask me: you don't need your devices to have internet access and it's a one-time operation (to be repeated with each new image you install of course, but installing packages afterwards would need that as well).
Sysupgrade can be told to backup all the files you'd like to keep. That's in /etc/sysupgrade.conf.
So, on every new release, instead of downloading the file openwrt-21.02.0-rc1-x86-64-generic-ext4-combined.img.gz, I build the image adding to it all packages I want.
Then, how to properly sysupgrade it? If it has all the packages, then I can keep my configs. Or is it easier to just give up on them and copy them back after the sysupgrade?
If we have to resize rootfs and copy configs back after each sysupgrade, then building the image is the same as sysupgrading default image, resizing rootfs and installing the packages from opkg, right?
Is it possible to save configs on a 3rd partition, in example mounting it on /etc? If there's a 3rd partition, is there any risk of sysupgrade destroying it?
Sysupgrade doesn't care if your image is vanilla or not. So you can still use it with your custom images. Either you include the configuration files there as well, or you just add all the configs that you'd like to keep (but aren't kept now) into /etc/sysupgrade.conf. Then sysupgrade will preserve them (unless you use -n of course).
It's less intensive to configure sysupgrade to retain the configs than to copy them back and forth every time you change them. Of course, regardless of the option you pick, it makes sense to have a separate backup anyway, in case stuff goes wrong somehow.
Resizing rootfs means doing it yourself post-installation (manually, I reckon, which you probably do now already?) or compiling an image (using the buildroot, not the image builder).
It sure is possible to add a third partition, but that is again manual work. If told to do so sysupgrade will back up your configuration files and restore them in the sysupgrade process. So a separate partition just sounds like extra work. I'm not sure about the sysupgrade internals, and how it manages x86 installations.
Yes, building your own image is the fastest way to obtain an image with the packages you need. Than forget about sysupgrade, use dd.
Considering that the x64 build has only one profile, just use something like ... make image PACKAGES="pkg1 pkg2 pkg3 -pkg4 -pkg5 -pkg6"
You will obtain several images, use the x64 combined one (EFI if you like)
As an example, here is what I use. You can see that it is highly configurable ! make image PACKAGES="luci luci-theme-material intel-microcode kmod-ata-ahci kmod-usb2 kmod-usb3 -ppp -ppp-mod-pppoe -bnx2-firmware -kmod-bnx2 -kmod-e1000 -kmod-e1000e -kmod-ixgbe -r8169-firmware -kmod-r8169 -kmod-forcedeth kmod-ath10k-ct ath10k-firmware-qca988x-ct wpad-wolfssl openvpn-openssl openvpn-easy-rsa luci-app-openvpn openssh-sftp-server"
Ok, so I add to /etc/sysupgrade.conf all files and folders I wanna preserve. This avoids having to bring them back manually. This includes /usr/script/ where I'm saving my custom python scripts. The most troublesome case is network which takes the router down from LAN subnet.
Then there's the issue of rootfs size. It was enough after I installed everything I use ATM, it has around 100MB and has 40MB free. But the SSD is 256GB, so there's a lot of wasted space.
I'd like to either increase rootfs size, or save installed apps on another partition. Which is the most compatible and reliable?
Backup isn't an issue for me. I'm using a Subversion repo to keep track of all my settings and scripts. OpenWRT's subversion-client doesn't support HTTPS and I'm unable to find how to compile it myself, so I'm keeping the working copy on my server and using a script full of rsync to sync them.
Does it support setting the partition size? Then I could build it having boot partition with 512MB and rootfs partition with 5GB. That's big enough to hold it on Parted Magic on /tmp. Then I add network, dhcp, sync script and authorized_keys files to it, dd it into SSD. It'd have LAN access and be able to easily sync all other configs and scripts back.
This command copy every files and directories that are inside the files/ directory.
Create a directory files/
Put every files and other directories you need inside, such as etc/, etc/config/ ...
using FILES=files/ will copy everything
Beware of the path
Note: The files/ folder is best in the imagebuilder root folder ( where you issue the make command ) otherwise it is best to use an absolute ( full ) path.
(packages, config), dd the image on the SSD, restore the config file for a definitive use.
