I need to perform specific file backup tasks before sysupgrade starts erasing stuff. Looked into the related scripts, can't find how it could be achieved. I tried to put that into the platform.sh script, but it appeared to be too late.
Thanks!
Do (/rom/etc/)uci-defauts scripts fit the bill?
Look this: https://github.com/openwrt-xiaomi/facinstall/tree/main/files
it is very likely that you will find something useful there. Yes, these are dirty hacks, but they work stably without changing the code (you need to patch in those places that are rarely changed by maintainers).
Do you need more than just keeping files between upgrades? Otherwise /etc/sysupgrade.conf could also work.
1 Like
Will they be anyway preserved if I put -n option for sysupgrade?
And stupid but smart question: the template says
## This file contains files and directories that should
## be preserved during an upgrade.
# /etc/example.conf
# /etc/openvpn/
but not clear if I can point to something out of /etc/, and if I can put wildcards in there. What if I put whole root into the list?
And the last one - where does it "save" those files? As I see the message /dev/ubi0_4 is mounted as /overlay, only erasing files and maybe it does not "format" the volume, but erases all the files which are not in the set matching of /etc/sysupgrade.conf...
It was worth trying and preserving from other locations seems to work, but I've got a problem.
-n option erases everything, and does not keep even those in sysupgrade.conf.
no option also copies a ton of other files which I do not need and actually need to be loaded from the new image.
Is there a way to tell sysupgrade to keep only those files I need?
Well,
1 )
one option might be to exclude all files that are identical with the old image, with the sysupgrade -u option :
-u skip from backup files that are equal to those in /rom
That should exclude all those unchanged config files that "you should be loaded from the new image"
2 )
Other option: create a service (just init script) that has a stop function, which then automatically runs in sysupgrade before the sysupgrade preserved archive is generated.
LuCI statistics backup uses that, and creates a .tar.gz backup archive of all the stats (dozens of files), and places it into /etc/luci_statistics directory, which is marked to be preserved in sysupgrade.
See init script of the luci-app-statistics:
Key lines:
The other logic there is likely too complex to your needs, as the logic aims to also combat unexpected power-offs etc.
3 )
Before the automatic LuCI stats backup was developed, I used a more manual approach, where I manually triggered a script from LuCI (via luci-app-commands) to collectd an archive to a tar.gz file that was marked to be preserved.
I even used cron to make daily backups just in case.
I have taken second approach, created script in /etc/init.d, but it appears sysupgrade does NOT perform stops, it just kills running processes. Thus /etc/init.d/myscript stop is not being run.
Just show your script....
And have you created the service symlink into /etc/rc.d ?
Please show the symlinks there...
(You need to do "/etc/init.d/yourservice enable" to create the symlink.)
When I perform /etc/init.d/myservice stop it works properly copying data into the backup location. Start section also works properly on (first) startup. It is just start/stop standard script installed as separate package.
START=90
STOP=11
USE_PROCD=1
start_service()
{
local SERVICE_CONFIGURED
SERVICE_CONFIGURED=$(uci get myservice.@myservice_options[0].service_enabled)
if [ "$SERVICE_CONFIGURED" = "1" ]; then
...
uci set myservice.@myservice_options[0].service_enabled='0'
uci commit myservice
fi
}
stop_service()
{
echo something
# copy files
}
config myservice
option service_enabled '1'
I do not see anything being echoed onto the console/log (which persists on the separate media), and no files are copied...
I must say I may be using non-standard build with some scripts modified (if that even possible), at least it will not be a big surprise for me.
I have made my own patch for sysupgrade, so it exclusively respects the files and folders listed in /etc/sysupgrade.conf.
It was so annoying to me, that it created a list of so many files from various sources and no way to control them, as I only wanted to keep wireguard keys and dropbear keys.
I could add that as a PR for sysupgrade if more people are interested, though then some more brainwork needs to be put into the solution and goals.
By the default -e is enabled, so my upgrades via Luci always use this mode. Because I can't set that flag manually in Luci. Or didn't want to try to figure out what script to change.
# sysupgrade: exclusively consider files in /etc/sysupgrade.conf and no other config files
patch -l /sbin/sysupgrade << 'EOF'
@@ -20,6 +20,7 @@
CONF_BACKUP_LIST=0
CONF_BACKUP=
CONF_RESTORE=
+CONF_EXCLUSIVE=1
NEED_IMAGE=
HELP=0
TEST=0
@@ -50,6 +51,7 @@
-r|--restore-backup) CONF_RESTORE="$2" NEED_IMAGE=1; shift;;
-l|--list-backup) CONF_BACKUP_LIST=1;;
-f) CONF_IMAGE="$2"; shift;;
+ -e) CONF_EXCLUSIVE=0;;
-F|--force) export FORCE=1;;
-T|--test) TEST=1;;
-h|--help) HELP=1; break;;
@@ -221,12 +223,30 @@
return 0
}
+build_list_of_exclusive_config_files() {
+ local file="$1"
+
+ local paths
+ paths=$(sed -ne '/^[[:space:]]*$/d; /^#/d; p' /etc/sysupgrade.conf 2>/dev/null)
+
+ if [ -z "$paths" ]; then
+ touch "$file"
+ return 0
+ fi
+
+ find $paths \( -type f -o -type l \) $find_filter 2>/dev/null | sort -u > "$file"
+
+ return 0
+}
+
if [ $SAVE_OVERLAY = 1 ]; then
[ ! -d /overlay/upper/etc ] && {
echo "Cannot find '/overlay/upper/etc', required for '-c' or '-o'" >&2
exit 1
}
sysupgrade_init_conffiles="build_list_of_backup_overlay_files"
+elif [ "$CONF_EXCLUSIVE" = 1 ]; then
+ sysupgrade_init_conffiles="build_list_of_exclusive_config_files"
else
sysupgrade_init_conffiles="build_list_of_backup_config_files"
fi
EOF
Please show that you have created the correct symlinks in /etc/rc.d (by using the enable command)
They are the ones that run start/stop automatically.
And the startup and stop automatics run headless, without console, so echo does not work.
You need to use "logger" command to log something into system log.
lrwxrwxrwx 1 root root 28 Dec 6 10:46 /etc/rc.d/K11myservice -> ../init.d/myservice
lrwxrwxrwx 1 root root 28 Dec 6 10:46 /etc/rc.d/S90myservice -> ../init.d/myservice
I thought if it runs that way its output goes to system log (as with any service running in background) or to the serial console I am operating. At least when applications are being terminated by the sysupgrade, I am certain their termination output appears on the serial console. but I am not sure if that application may write to logger internally.
Symlinks look ok.
Remove the echo commands, it may help.
I can't try right now, but curious how it may help - what are your thoughts? Can I replace with logger? I need to keep some track of the actions being done to review in case there're problems.
Edit: I can develop application using C which sleeps in the memory and performs file copy when gets TERM or INT signals. This must work in my environment.
There is no stout / stderr on headless, so echo does not work.
Use logger -t yourservice message1 as debug, so that you see output on logread.
Examples: https://github.com/search?q=repo%3Aopenwrt%2Fopenwrt+logger+language%3AShell&type=code&l=Shell
1 Like
In service start I have some echoes and see their output on the system log.
Fri Dec 6 10:52:14 2024 daemon.notice procd: /etc/rc.d/S90myservice: sometext
and would expect the similar behavior when service is being stopped. I do not say you are not correct with logger - this is perfectly correct to see messages in the system log, the point is when I start/stop service manually I can see its output onto the console which is handy.
Edit: another point for the proof. When system is being rebooted, I see in the log
Fri Dec 6 10:49:06 2024 daemon.notice procd: /etc/rc.d/K11myservice: sometext
which proves that under normal circumstanced the service is being stopped, but sysupgrade does not stop services, it just forcefully kills all the processes.
The binary application seems to work as expected, however I did not see its output because I do not have access to the console. But the end result is what had been expected.
The app is very simple - waiting for semaphore which is posted when app receives the INT or TERM signal.
1 Like
Another question - related to sysupgrade and plaform.sh. If modifying platform.sh file after system startup, I start getting strange failures during the upgrade - like it can not find /lib/functions.sh. However if I install platform.sh into the original image during compilation, upgrade functions properly.
Question - after sysupgrade starts, where does it obtain /lib/upgrade/* files - from current image, or from the ROM? If from ROM, then it partially explains the behavior, and the only right way is to install platform.sh file into the ROM. Not sure if installing it within the additional package will work, or it must be within the main image.