HOWTO: backup settings to a linux server

This is my first post in this forum. Please be kind :slight_smile:

I am doing automated backups of my openwrt config to a linux server and I wanted to do some documentation on how it is done, so why not document it here, for others to read, increase and use my solution?

I am only using standard linux tools on the linux side and openwrt on the router. Currently a TP-Link Archer C7 v2 is my main router, but this should be transferable to almost any device.

OpenWRT:
I want to backup my rrd database for statistics, unnecessary but nice. And I export a list of all the installed packages, in case of a reinstall you can install all of them at once in a shell.

System -> Scheduled Tasks:

# auto backup list of installed packages to a file
52 * * * *     /usr/sbin/listuserpackages | sort > /etc/config/userpackages.list

System -> Backup -> Configuratioon -> Backup file list:

## This file contains files and directories that should
## be preserved during an upgrade.

# /etc/example.conf
# /etc/openvpn/
/usr/sbin/reconnect
/usr/sbin/listuserpackages
# backup rrd DB for statistics - may increase size A LOT!
/tmp/rrd
# add here whatever you want to keep

Linux server side
I wrote a script to use the OpenWRT backup tool to create a config file and transfer it to my linux machine. To automate this, I execute the script every hour via cron as a normal user, this creates a backup file containing creation day. This way I archive the last backup every day (one per day), the current backup overwrites the one from an hour ago. A full year of backups is <42MB for me so storage is no concern.

backup-router (must be executable)

#!/bin/sh
# creates a backup and fetches it to my backup server
# by Stich IX 2018

filename=backup-router-$(date "+%Y-%m-%d").tar.gz

/usr/bin/ssh -4 -i /home/stich/.ssh/owrt.key root@router /sbin/sysupgrade --create-backup /tmp/$filename
/usr/bin/scp -4 -i /home/stich/.ssh/owrt.key root@router:/tmp/$filename /mnt/backup/router/
/usr/bin/ssh -4 -i /home/stich/.ssh/owrt.key root@router rm /tmp/$filename

To make this work, you need an ssh keypair - here called owrt.key for the private key on linux and the public key in authorized_keys on OpenWRT.

crontab on linux:

55 * * * *      stich   /home/stich/bin/backup-router > /dev/null 2>&1

This is my solution for an automated backup of an OpenWRT installation to backup storage.
If you find something missing, typos, or improvements, please contact me!

1 Like

How big is the userpackages.list that you are writing on the routers internal flash memory 8760 times a year?

2 Likes

About 1k.
I know what you are thinking, maybe I should sort it and compare to the existing one as it changes not very often.
Or write it to /tmp

Could always write it as:

if [ "$(diff /usr/sbin/listuserpackages \
/etc/config/userpackages.list)" != "" ]; \
then /usr/sbin/listuserpackages | sort > \
/etc/config/userpackages.list; fi

I think this diffs the code for listuserpages with the output. Probably what you want is:

/usr/sbin/listuserpackages | sort > /tmp/userpkgs

if [ "$(diff /tmp/userpkgs /etc/config/userpkgs)" != "" ]; ...
1 Like

EDIT: Sorry re-read your message more attentively... My bad - I see what you mean. I just put a conditional on the original backup command the OP used, without really thinking about what exactly it compared. I'll test it it a moment on the router.

UPDATE (after the edit):

root@Bifrost:~# opkg list-installed | awk '{print $1}' > /etc/config/userpackages.list

# actually needed to install diffutils now

root@Bifrost:~# diff <(opkg list-installed | awk '{print $1}') /etc/config/userpackages.list
12d11
< diffutils

So the final version would be bellow (note ash does not support process substitution like this, hence this work in bash only):

if [ "$(diff <(opkg list-installed | awk '{print $1}') /etc/config/userpackages.list)" != "" ]; then opkg list-installed | awk '{print $1}' >/etc/config/userpackages.list; fi

# Tested and it worked:

root@Bifrost:~# if [ "$(diff <(opkg list-installed | awk '{print $1}') /etc/config/userpackages.list)" != "" ]; then opkg list-installed | awk '{print $1}' >/etc/config/userpackages.list; fi
root@Bifrost:~# diff <(opkg list-installed | awk '{print $1}') /etc/config/userpackages.list
root@Bifrost:~# 


Original post below. No longer relevant.

$ cat file_1
1
2
3

$ cat file_2
1
2
3

$ cat file_3
1
2
3
4

$ if [ "$(diff file_1 file_2)" = "" ]; then echo "Files are the same"; fi
Files are the same

if [ "$(diff file_1 file_3)" != "" ]; then echo "Files are different"; fi
Files are different

Hence the logic: if the output of the diff is not an empty string, then the files are different, proceed with backup. Easy to test yourself as I did.

1 Like

Thanks for your ideas, I found an even simpler aproach yesterday:
I write the userpackages.list to /tmp and backup it from there. This works quite well, too, and I only use the list after an update to a newer version to remember which packages are needed to get my USB modem up and running.

edit: As the list of packages does not change that much (for me) I will dump it once a day - that's enough.

edit 2: I guess I cannot correct the first post ... Anyway, I realized that listuserpackages is not an OpenWRT script, so it is missing here for someone willing to try this:

#!/bin/ash
# echo >&2 User-installed packages are the following:
sed -ne '/^Package:[[:blank:]]*/ {
        s///
        h
}
/user installed/ {
        g
        p
}' /usr/lib/opkg/status

I found this somewhere around here but I cannot remember where, so I cannot credit the author - it was not me.

1 Like

Oh that's pretty neat - I didn't know /usr/lib/opkg/status actually specified packages explicitly installed by the user.

That was a good exercise! Solution you provided is more elegant compared to what I came up with (below), imho.

grep -E "Package|Status" /usr/lib/opkg/status | \
sed -e ':a;N;$!ba;s/\nStatus: / /g' | \
grep "user installed" | \
awk '{print $2}'

My two cents: each of my routers run a "rsync" daemon, then I make a daily copy of some folders like /etc, /overlay, /mnt, etc., using "rsnapshot".

2 Likes

When I first installed a new version of OpenWRT I had to reverse engineer which packges my USB modem needs to run, so I searched for a solution to dump the list of installed packages and reinstall them after updating the base system.
And this is what I found.

I thought about rsync aswell, but this way I don't need anyting but ssh on the router.

@eduperez: Can you tell us more about your rsync solution?

Not much to explain, really. Just install and configure "rsync" on the router, so the whole file structure is accessible. Then install and configure "rsnapshot" to copy the folders that are interesting to you. That's pretty much all.

sketchy wrappers
#!/bin/bash

#set -x

D="$(date +%Y%m%d-%H%M)"
H="${HOSTNAME}"

AKEYS="/etc/dropbear/authorized_keys" ############# openssh-server will need something else here
rUSER="root";
ROUTERIP="192.168.1.1";
ROUTERIP="10.2.3.6"
ROUTERIP="10.2.3.1"
ROUTERIP="10.2.3.186"

dbgthresh=5
rtrCONFd="./rtrCONF"


osis2() {
if [ -d /etc/dropbear ]; then #if which arp; then
	AKEYS="/etc/dropbear/authorized_keys"
else
	AKEYS="/etc/ssh/authorised_keys"
fi
}

rdobool() {
	command="$1"
	cpath="`which $command`"
	if [ -z $cpath ]; then
		echo "Notfound $command" && return 1
	else
		echo "Found $command binary at $cpath" && return 0
	fi
}


checkFILE() { if [ ! -f "${1}" ]; then echo "$2> ${1} file not found" && exit 1; fi }

checkDIR() {
if [ ! -d "${1}" ]; then echo "$2> ${1} directory not found" && exit 1; fi
}


oscheck2() {
cat /proc/version | grep -q OpenWrt
if [ $? -ne 0 ]; then
	echo "No ip given and not wrt = default pull"
fi
}


booloscheck() {
cat /proc/version | grep -q OpenWrt; if [ $? -ne 0 ]; then return 1; else return 0; fi
}


checkupandsshport() {

ROUTERIP="${1}"
ping -c 2 -W 2 $ROUTERIP > /dev/null
if [ $? -ne 0 ]; then return 5; fi
nc -z $ROUTERIP 22
if [ $? -ne 0 ]; then return 6; fi
return 0

}


sshfsmountdual() {

#opkg update; opkg install openssh-sftpserver

echo "     REMOTEUSER: $rUSER"
echo "       REMOTEIP: $REMOTEIP"
echo "      REMOTEDIR: $sshfsREM"
echo "       LOCALDIR: $sshfsDIR"
echo "           USER: $USER"
echo "       ROUTERIP: $ROUTERIP"


case "$1" in
	mount)
		if [ ! -d "$sshfsDIR" ]; then
			echo "local sshfs-dir: $sshfsDIR [create]"
			mkdir -p $sshfsDIR
		else
			echo "local sshfs-dir: $sshfsDIR [exists]"
		fi
		if mount | grep -q "$sshfsREM"; then
			echo "sshfsdir is mounted"; sleep 1; mount | grep "$sshfsREM"
		else
			sshfs -o allow_other,IdentityFile=~/.ssh/id_rsa $sshfsREM $sshfsDIR
			mount | grep fuse.sshfs
		fi

	;;
	umount)
		if mount | grep -q "$sshfsREM"; then
			fusermount -u $sshfsDIR
			mount | grep "fuse.sshfs"
		else
			echo "sshfsdir is not mounted"
		fi
		if [ -d "$sshfsDIR" ]; then
			lsnum="`ls -1 $sshfsDIR/ | wc -l`"
			if [ "$lsnum" -lt 1 ]; then
				echo "Removing sshfsdir: $sshfsDIR"; sleep 1
				rm -rf $sshfsDIR
			else
				echo "Leaving sshfsdir: $sshfsDIR [ has files:$lsnum ]"; sleep 1
			fi
		fi
	;;
esac

}



rtrsiggen() {

ROUTERMAC="`cat /proc/net/arp | grep "$ROUTERIP " | tail -n 1 | sed -r -e \"s/[\t\ ]+/ /g\" | cut -d' ' -f4 | cut -d':' -f 1,2,3,4,5,6 | sed -e s/\:/\"\"/g`"
if [ -z $ROUTERMAC ]; then
	echo " [nomac]" && return 2
elif [ "$ROUTERMAC" == "000000000000" ]; then
	echo " [mac-nan]" && return 3
else
	rtrSIG="$ROUTERMAC-$ROUTERIP" #rtrSIG="$ROUTERIP-$ROUTERMAC"
fi

}


checksshaccess() {

	ssh -o BatchMode=yes $rUSER@$ROUTERIP 'exit'
	if [ $? -eq 0 ]; then return 0; else return 1; fi

}


rdopkglist() {

rtrOUTd="$rtrCONFd/$ROUTERMAC"
mkdir -p $rtrOUTd

    rUSER=`echo $1 | cut -d'@' -f1`
	ROUTERIP=`echo $1 | cut -d'@' -f2`

	if [ -z "$PKGLIST" ]; then
		ssh $rUSER@$ROUTERIP 'opkg list-installed | cut -d" " -f1'
        ssh $rUSER@$ROUTERIP 'opkg list-installed | cut -d" " -f1' >$rtrOUTd/packages-all.txt-$(date +%Y%m%d-%H%M)
        ssh $rUSER@$ROUTERIP 'opkg list-installed | cut -d" " -f1' >$rtrOUTd/packages-all.txt
    else
		echo "export to: $PKGLIST"
		ssh $rUSER@$ROUTERIP 'opkg list-installed | cut -d" " -f1' >$PKGLIST
	fi

}



rtrpkgrestore() {

rUSER=`echo $1 | cut -d'@' -f1`
ROUTERIP=`echo $1 | cut -d'@' -f2`
PKGLIST="${2}"

if [ -z "$PKGLIST" ]; then echo "PKGLIST: $PKGLIST no passed" && exit 1; fi
if [ ! -f "$PKGLIST" ]; then echo "PKGLIST: $PKGLIST does not exist" && exit 1; fi

if [ -f "/tmp/opkgrestore.log" ]; then rm -f /tmp/opkgrestore.log; fi
for pkg in $(cat $PKGLIST); do
	echo -n "####### $pkg"
	ssh $rUSER@$ROUTERIP 'opkg install '$pkg >/tmp/opkgrestore.log ; retcode=$?
	if [ $retcode -eq 0 ]; then
		echo " $retcode"
	else
		echo " $retcode"
	fi
done
echo "pkg add complete!"

if [ -f "/tmp/opkgrestore.log" ]; then
	echo "Errors occurred see /tmp/opkgrestore.log"
	cat /tmp/opkgrestore.log
fi
}



usage() {
echo ""
echo "$0 [root@routerip]"
echo ""
echo "			-c <command>"
echo "			"
echo "			-opkgrestore <pkgs-file>"
echo "			-opkgdump <optional-outputtofile or stdout>"
echo ""
echo "          -sysupget"
echo "			-sysupput BACKUP.tar.gz"
echo "			"
echo "			-M		(mount sshfs /)"
echo "			-UM		(unmount sshfs /)"
echo ""
echo " -exportservice adblock [-f|force]... tba -d outstartpath"

}



sysupgradebackup() {

rtrOUTd="$rtrCONFd/$ROUTERMAC"
mkdir -p $rtrOUTd

ssh $rUSER@$ROUTERIP sysupgrade -b -k /tmp/backup.tar.gz 2>/dev/null 1>/dev/null;
retval="$?"
if [ $retval -ne 0 ]; then
    echo "backup:$retval"
    sleep 1
fi
scp $rUSER@$ROUTERIP:/tmp/backup.tar.gz $rtrOUTd/backup.tar.gz 2>/dev/null 1>/dev/null
scp $rUSER@$ROUTERIP:/tmp/backup.tar.gz $rtrOUTd/backup-${rtrSIG}-$(date +%Y%m%d-%H%M).tar.gz 2>/dev/null 1>/dev/null
retval="$?"
if [ $retval -ne 0 ]; then
    echo "scp copy of backup.tar.gz failed"
    return 1
fi
echo "output: $rtrOUTd/backup-${rtrSIG}-$(date +%Y%m%d-%H%M).tar.gz"

}


sysupgraderestore() {

echo "restoring: ${1} < $RESTOREFILE"
if [ ! -f "$RESTOREFILE" ]; then echo "-sysupput FILE not PRESENT: $RESTOREFILE" && exit 1; fi
scp $RESTOREFILE $rUSER@$ROUTERIP:/tmp/backup.tar.gz 2>/dev/null 1>/dev/null
ssh $rUSER@$ROUTERIP sysupgrade -r /tmp/backup.tar.gz 2>/dev/null 1>/dev/null

}



if echo "${1}" | grep -q "@"; then
	rUSER=`echo $1 | cut -d'@' -f1`; ROUTERIP=`echo $1 | cut -d'@' -f2`; doREMOTE="y";
	shift 1
else
	doREMOTE="n"
fi



########################################################### for sshfs > todo
sshfsDIR="./sshfs" #realpathof #echo "`realpath $sshfsDIR`"
sshfsREM="${rUSER}@${REMOTEIP}:${REMOTEDIR}"
rUSER="root";
REMOTEIP="10.2.3.11";
REMOTEDIR="/"
sshfsREM="${rUSER}@${ROUTERIP}:${REMOTEDIR}"
########################################################### for sshfs > todo



if [ "${doREMOTE}" == "n" ]; then
	if booloscheck; then
		ecmd "payload-run"
		sendit="n"
	else
		ecmd "payload-setup"
		sendit="y"
	fi
fi

	while [ "$#" -gt 0 ]; do
	case "$1" in
    -h) usage;
	exit 0;;
    --help) usage;
	exit 0;;
    -z) BBUG="y";
	echo "full debug enabled"
	if [ "$2" == "file" ]; then
		BBUGF="./debugfile.txt"
		echo "Outputfile: $BBUGF"
		shift 2
	else
		shift 1
	fi
	;;
    -f|force) force="y"; shift 1; ;; #for exportservice
	-M)
		which sshfs &>/dev/null || echo "sshfs is not installed"
		which sshfs &>/dev/null || exit 1
		sshfsmounty "mount" "$sshfsREM" "$sshfsDIR"
		exit 0
	;;
	-UM)
		sshfsmounty "umount" "$sshfsREM" "$sshfsDIR"
		exit 0
	;;

    -c) doCOMMAND="y";
	shift 1;
	docmd="$@";
	break
	;;
    -opkgrestore)
	rtrpkgrestore "$rUSER@$ROUTERIP" "$2"
	exit 0
	;;
    -opkgdump)
	if [ ! -z "$2" ]; then PKGLIST="$2" && shift 2; else shift 1; fi
	doopkgdump="y"
	;;
    -sysupget)
	dosysupget="y"
	shift 1
	;;
    -sysupput)
	if [ -z "$2" ]; then echo "-sysupput NEEDFILE" && exit 1; fi
	if [ ! -f "$2" ]; then echo "-sysupput FILEINVALID: $2" && exit 1; fi
	RESTOREFILE="${2}"
	dosysupput="y"
	shift 2
	;;
    -*)
	echo "unknown option: $1 ... \
	need: -d dotno -s slpsec -n nonew" >&2;
	usage
	exit 1;;
    #*)
	#usagerestore
	#exit 1;;
  esac
done



if [ "$BBUG" == "y" ]; then
	if [ -z "$BBUGF" ]; then
		ecmd() { echo $1; sleep 1; }
	else
		ecmd() { echo $1 >> $BBUGF; }
	fi
else
	ecmd() { echo $1 > /dev/null; }
fi


if ! checkupandsshport "$ROUTERIP"; then echo "host $ROUTERIP down or no ssh" && exit 1; fi

if ! checksshaccess "$ROUTERIP" 2>/tmp/sshaccesserr; then
	sshmsg="key-auth fail"
	if cat /tmp/sshaccesserr | grep -q 'Offending'; then
		sshmsg="$sshmsg Host key has changed"
		ssh-keygen -f "${HOME}/.ssh/known_hosts" -R ${ROUTERIP} 2>/dev/null 1>/dev/null
	fi
	echo "host $ROUTERIP $sshmsg please enter password to add key"
	ssh $rUSER@$ROUTERIP "tee -a $AKEYS" < ${HOME}/.ssh/id_rsa.pub > /dev/null
	if [ $? -eq 0 ]; then
		echo "SSH KEY HAS BEEN PUSHED"
	fi
fi


checkforinfofile() {

	if [ "$dbgthresh" -gt 2 ]; then echo "Checking for infofile: $1"; sleep 1; fi

	ssh $rUSER@$ROUTERIP "test -f $1"; retVAL="$?"
	if [ $retVAL -ne 0 ]; then
		echo " [no infofile: $1]"; #return 5
		ssh $rUSER@$ROUTERIP "echo rtrSIG=\"$rtrSIG\" >> $1"
		ssh $rUSER@$ROUTERIP "echo installDATE=\"$D\" >> $1"
		ssh $rUSER@$ROUTERIP "echo importSIG=\"${ROUTERMAC}-$D\" >> $1"
		ssh $rUSER@$ROUTERIP "ubus call system board >> $1"
		#ssh $rUSER@$ROUTERIP "cat $1"
	else
		echo "[ok]" #if [ "$dbgthresh" -gt 2 ]; then echo "[ok]"; sleep 1; fi
	fi
}




rtrsiggen #sets ROUTERMAC and rtrSIG[ip+mac] > locally
rtrMODEL=$(ssh $rUSER@$ROUTERIP "cat /tmp/sysinfo/board_name" | sed 's/,/./g')

checkforinfofile "/system.info" 


if [ "$dbgthresh" -gt 2 ]; then
	echo "         rtrMODEL: $rtrMODEL"
	echo "           rtrSIG: $rtrSIG"
	sleep 2
fi


if [ "$doCOMMAND" == "y" ]; then ssh $rUSER@$ROUTERIP $docmd; exit 0; fi
if [ "$doopkgdump" == "y" ]; then echo "NEEDS just dump rom or dump all with locations"; sleep 3; rdopkglist "$rUSER@$ROUTERIP"; fi
if [ "$dosysupget" == "y" ]; then sysupgradebackup "$rUSER@$ROUTERIP" "$2"; fi
if [ "$dosysupput" == "y" ]; then sysupgraderestore "$rUSER@$ROUTERIP" "$2"; fi

exit 0