Where does the file /lib/upgrade/keep.d/base-files come from

Device gl-ar150 with openwrt-18.06.5-ar71xx-generic-gl-ar150-squashfs-sysupgrade.bin installed there is a file /lib/upgrade/keep.d/base-file
with the content:

/etc/config/
/etc/config/network
/etc/config/system
/etc/crontabs/
/etc/dropbear/
/etc/profile.d

I do not like the blank config/ and the etc/profile.d so I am trying to find out where it comes from in the source code, to learn why someone thought this was a good idea.

Does anybody have and idea where this file comes from?

I search the project/openwrt/openwrt.git but I did not find anything. Next I am going to search the installed package one by one but before I do that I try my luck here.

1 Like

Why do you dislike the blank ...?

It´s generated at build time...

Edit:
If you wanna exclude some specific file, you can add something like the following into /etc/sysupgrade.conf
! /path/to/file

5 Likes

See also https://openwrt.org/docs/guide-user/troubleshooting/backup_restore

1 Like

You do realize that the /etc/config folder won't be blank when you're backing up your config files, correct?

I don't like the blank because it makes no sense to add the whole directory and two files in that directory. Secondly config/ clearly contain files from other packages having one package including files of other packages is kind of messy.

It surprised me that all config files was saved even if they were not changed, that was why I wanted to know if it was on purpose or not. Logically all files or only change files could be right.

Thanks for the quick response it saved me some time as I was on the wrong track.

Yes.
Blank was referring to include all files in the directory.

This does not seem to work: below config excludes the entire /root directory (I tried the opposite order as well).

/root/
! /root/file1
1 Like

I agree that a "default" of not copying over files that are unchanged from the ROM makes sense in some situations.

On master (and perhaps 19.07), there are options to control some of that behavior (I have not tried any of them).

jeff@office:~$ sysupgrade 
Usage: /sbin/sysupgrade [<upgrade-option>...] <image file or URL>
       /sbin/sysupgrade [-q] [-i] [-c] [-u] [-o] [-k] <backup-command> <file>

upgrade-option:
	-f <config>  restore configuration from .tar.gz (file or url)
	-i           interactive mode
	-c           attempt to preserve all changed files in /etc/
	-o           attempt to preserve all changed files in /, except those
	             from packages but including changed confs.
	-u           skip from backup files that are equal to those in /rom
	-n           do not save configuration over reflash
	-p           do not attempt to restore the partition table after flash.
	-k           include in backup a list of current installed packages at
	             /etc/backup/installed_packages.txt
	-T | --test
	             Verify image and config .tar.gz but do not actually flash.
	-F | --force
	             Flash image even if image checks fail, this is dangerous!
	-q           less verbose
	-v           more verbose
	-h | --help  display this help

backup-command:
	-b | --create-backup <file>
	             create .tar.gz of files specified in sysupgrade.conf
	             then exit. Does not flash an image. If file is '-',
	             i.e. stdout, verbosity is set to 0 (i.e. quiet).
	-r | --restore-backup <file>
	             restore a .tar.gz created with sysupgrade -b
	             then exit. Does not flash an image. If file is '-',
	             the archive is read from stdin.
	-l | --list-backup
	             list the files that would be backed up when calling
	             sysupgrade -b. Does not create a backup file.
1 Like

Probably it was not that clever to add /etc/conf/ as a whole to the base-files package...

I do not understand why this was done instead of adding only the file which the base-files package adds...

Here is the commit that adds the full /etc/conf/ directory to the conffiles of the base-files package. Probably @jow knows why the complete directory.

But the other two files in the conf dir are added later, so I think the one who added the files system and network should have removed the entry of the directory /etc/conf/.

confusing indeed...

Likely it was done as an easy way to add the whole uci config, with all present files there.

Just remember that "base-files" is not a real package by itself, but should be seen as an integral part of the core system. It contains, defines and sets all kind of globally used stuff.

Also note that the commit did not actually "add" that, but it just moved the hard-coded definition from the sysupgrade script itself (line 58) to the file list.
The practice originates from 2007, from really ancient times, when the ability to preserve config was introduced for the first time.

2 Likes

Good point, but how do you think about that any package, also non real packages like base-files, should only set the files that should be preserved that are part of it self?

base-files = core system.
Core system needs to be able to define actions globally.
/etc/config has been defined 12 years ago as the central uci config directory to be preserved in sysupgrade.

1 Like

The generic defaults of /etc/config/, /etc/passwd, /etc/shadow etc. were added to accommodate for devices without installed opkg, on where we don't have the ability to use opkg list-changed-conffiles to discover changed files.

If someone feels like working on it, then I'd appreciate a patch making this behavior dependent on whether opkg is available at sysupgrade backup time or not.

4 Likes

Thanks it god information for later, right now I am just looking for a workable upgrade procedure and the giant list of conserved files need a investigation. Now I am looking at

opkg list-changed-conffiles

It returns files that I did not modify like "passwd" and "group" and does not include files I changed like "/etc/config/system" It is not what I expected so I need to know more. I am not sure where the source code is do you know? I am guessing is in opkg-lede.git/libopkg/opkg_cmd.c but the name of repository is a bit strange.

1 Like

You haven´t changed the password after first boot?

According to the Makefile of the opkg package, you are right.

I did change the password but I do not think that change the passwd file, isn't the change in the shadow file only. I can even delete the overlay versions of the group and passwd file and still they end up in list-changed-conffiles. Anyway it is not important for the system it just helps me understanding the logic behind the file comparison.

1 Like

You can compare them with their /rom versions to see what changed: som packages make additional changes there.
I also remember seeing that some of the files are hard coded in opkg and will always be reported as changed.

1 Like

I did and I am guess your are right, I just haven found it yet. /etc/config/system and /etc/config/network does not belong to any package according opkg search and they are created so detecting change pose a challenge.

1 Like

Those two config files are auto generated on the first boot.