Automatic update script

@zorxd I've made script some days ago which can survive and run after sysupgrade. One part of the script lives on your linux server and downloads official flash image and o'package files, then uploads them to your device, flashes it. The second part lives on your device and reads the offline cache (ipk files), tries to reinstall your packages and reboot your device then to (re)turn it to online operation. If then packages non crucial to operation are missing, the script autoruns a second time on the device and tries to (re)install your preferred packages from the openwrt online repo. I can share it on request, just open a thread and ping me as this is not the right place for management ;-).

3 Likes

Sounds great, i assume you have tested them thoroughly?
Im in the works of planning my upgrade and was thinking of making my own image with imagebuilder, but it will be the first time i upgrade so im looking for all info concerning howto and best practice

@user674574 I've documented how to do a custom build here ( State of Archer C7 v2 in mid-2020 )

Yes, the scripts have completed testing successfully and work reliably. But: I only can use the "on device" script at the moment due to bug ( https://forum.openwrt.org/t/19-07-xx-strange-kernel-jffs2-bug ) where TP Link Archer OpenWrt Firmware cannot persist large files during flash because of an SPI read "problem" which trips "jffs2 error" and config reset unintentionally on my hardware. So I'm upgrading with a custom build of official's OpenWrt 19.07.4 source code plus packages I require to connect to the internet and let then the script do the rest on first boot after flash to reinstall other packages from the online repo.

To setup the "reinstall opkg after flash" script:

  • Edit /etc/rc.local
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

( /bin/sleep 30; /bin/sh /root/opkg_reinstall.sh ) &

exit 0

/etc/sysupgrade.conf

## This file contains files and directories that should
## be preserved during an upgrade.

# /etc/example.conf
# /etc/openvpn/

/root

/root/opkg_reinstall.sh

#!/bin/sh
trap "" SIGHUP
#
#
# Command line
## sh /root/opkg_reinstall.sh force
#
# Notes
## opkg remove ath10k-firmware-qca988x kmod-ath10k kmod-batman-adv wpad-mesh-openssl; opkg remove batctl-full;
## opkg install ath10k-firmware-qca988x-ct kmod-ath10k-ct
#
# Consts
PATH=/usr/bin:/usr/sbin:/sbin:/bin
#
# Note: If we log to "/root/" during "insmod", "opkg", "rmmod" commands affecting kernel modules, the kernel will panic and the device will reboot.
LOGFILE="/root/opkg_reinstall.log"
LOG_MAX_LINES="1000"
#
LOG_COLLECTOR_HOSTNAME="WifiAP-01"
INSTALL_BATMANADV="1"
INSTALL_OPENVPN="1"
INSTALL_RELAYD="0"
INSTALL_WIFI_NON_CT_DRIVERS="1"
PACKAGE_CACHE="/root/packages"
#
# Runtime vars
INSTALL_ONLINE="1"
REBOOT_REQUIRED="0"
#
#
# -----------------------------------------------------
# -------------- START OF FUNCTION BLOCK --------------
# -----------------------------------------------------
logAdd ()
{
	TMP_DATETIME="$(date '+%Y-%m-%d [%H-%M-%S]')"
	TMP_LOGSTREAM="$(tail -n ${LOG_MAX_LINES} ${LOGFILE} 2>/dev/null)"
	echo "${TMP_LOGSTREAM}" > "$LOGFILE"
	echo "${TMP_DATETIME} $*" | tee -a "${LOGFILE}"
	return
}


opkgInstall () {
	# Syntax:
	# 	opkgInstall "[PACKAGE_NAME]"
	#
	# Online mode.
	if [ "${INSTALL_ONLINE}" = "1" ]; then
		RESULT="$(eval "opkg install ${@}" 2>&1)"
		logAdd "[INFO] opkgInstall: - ${RESULT}"
		if ( echo "${RESULT}" | grep -q "Collected errors:"); then
			return 1
		fi
		return 0
	fi
	#
	# Offline mode
	IPKG_FULLFN=""
	for package_name in $@; do
		if ( ! ls -1 ${PACKAGE_CACHE} | grep -q "^${package_name}_" ); then
			logAdd "[ERROR] opkgInstall: Package missing in cache - [${package_name}]"
			continue
		fi
		IPKG_FULLFN="${IPKG_FULLFN} ${PACKAGE_CACHE}/$(ls -1 ${PACKAGE_CACHE} | grep "^${package_name}_")"
	done
	if [ -z "${IPKG_FULLFN}" ]; then
		return 0
	fi
	RESULT="$(eval "opkg --cache ${PACKAGE_CACHE} install ${IPKG_FULLFN}" 2>&1)"
	if ( echo "${*}" | grep -q "kmod-" ); then
		REBOOT_REQUIRED="1"
		# logAdd "[INFO] opkgInstall: Kernel module added. Will sleep a bit to avoid crash."
		sleep 10
	fi
	logAdd "[INFO] opkgInstall: - ${RESULT}"
	if ( echo "${RESULT}" | grep -q "Collected errors:"); then
		return 1
	fi
	return 0
}


opkgRemove () {
	# Syntax:
	# 	opkgRemove "[PACKAGE_NAME]"
	#
	for package_name in $@; do
		logAdd "[INFO] opkgRemove: Removing ${package_name}"
		RESULT="$(eval "opkg remove ${package_name}" 2>&1)"
		if ( echo "${package_name}" | grep -q "^kmod-" ); then
			REBOOT_REQUIRED="1"
			# logAdd "[INFO] opkgRemove: Kernel module removed. Will sleep a bit to avoid crash."
			sleep 10
		fi
		logAdd "[INFO] opkgRemove: - ${RESULT}"
		if ( echo "${RESULT}" | grep -q "Collected errors:"); then
			return 1
		fi
	done
	return 0
}


runInstall () {
	# Syntax:
	#	runInstall
	#
	# Global vars
	# 	[IN] INSTALL_ONLINE
	#
	# Variables
	PKG_TO_INSTALL=""
	PKG_TO_REMOVE=""
	#
	# If we are offline, check if we have a package cache available.
	if [ "${INSTALL_ONLINE}" = "0" ] && [ ! -d "${PACKAGE_CACHE}" ]; then
		logAdd "[ERROR] runInstall: We are offline but don't have a package cache available at ${PACKAGE_CACHE}"
		return 1
	fi
	#
	if [ "${INSTALL_ONLINE}" = "1" ]; then
		logAdd "[INFO] Downloading package information ..."
		RESULT="$(opkg update | grep "http://" 2>&1)"
		logAdd "[INFO] opkg update: - ${RESULT}"
	fi
	#
	# Add packages to the install and remove queue.
	PKG_TO_REMOVE="${PKG_TO_REMOVE} wpad wpad-basic"
	PKG_TO_INSTALL="${PKG_TO_INSTALL} libopenssl1.1 wpad-mesh-openssl"
	#
	# Replace ct with non-ct drivers
	if [ "${INSTALL_WIFI_NON_CT_DRIVERS}" = "1" ]; then
		# Remove order is important
		PKG_TO_REMOVE="${PKG_TO_REMOVE} kmod-ath10k-ct ath10k-firmware-qca988x-ct"
		PKG_TO_INSTALL="${PKG_TO_INSTALL} ath10k-firmware-qca988x kmod-ath10k"
	fi
	#
	# batman-adv
	if [ "${INSTALL_BATMANADV}" = "1" ]; then
		PKG_TO_INSTALL="${PKG_TO_INSTALL} batctl-full kmod-batman-adv kmod-crypto-crc32c kmod-lib-crc16 kmod-lib-crc32c kmod-crypto-hash librt"
	fi
	#
	# Only install these packages if we are online
	if [ "${INSTALL_ONLINE}" = "1" ]; then
		#
		# Base system
		PKG_TO_INSTALL="${PKG_TO_INSTALL} bash curl htop lua luafilesystem mailsend nano terminfo tcpdump wget"
		#
		# FTP service
		PKG_TO_INSTALL="${PKG_TO_INSTALL} vsftpd"
		#
		# Relayd
		if [ "${INSTALL_RELAYD}" = "1" ]; then
			PKG_TO_INSTALL="${PKG_TO_INSTALL} luci-proto-relay relayd"
		fi
		#
		# OpenVPN
		if [ "${INSTALL_OPENVPN}" = "1" ]; then
			PKG_TO_INSTALL="${PKG_TO_INSTALL} luci-app-openvpn openvpn-easy-rsa openvpn-openssl"
		fi
		#
		# Syslog-ng, logd
		if ( echo "${HOSTNAME}" | grep -q "^${LOG_COLLECTOR_HOSTNAME}" ); then
			PKG_TO_REMOVE="${PKG_TO_REMOVE} logd"
			PKG_TO_INSTALL="${PKG_TO_INSTALL} syslog-ng"
		else
			PKG_TO_REMOVE="${PKG_TO_REMOVE} syslog-ng"
			PKG_TO_INSTALL="${PKG_TO_INSTALL} logd"
		fi
		#
		# USB storage drivers
		PKG_TO_INSTALL="${PKG_TO_INSTALL} block-mount e2fsprogs kmod-fs-ext4 kmod-fs-msdos kmod-scsi-core kmod-usb-storage libncurses libpcre"
	fi
	#
	logAdd "[INFO] runInstall: Remove packages"
	opkgRemove "${PKG_TO_REMOVE}"
	if [ ! "$?" = "0" ]; then
		return $?
	fi
	#
	logAdd "[INFO] runInstall: Install packages"
	opkgInstall "${PKG_TO_INSTALL}"
	if [ ! "$?" = "0" ]; then
		return $?
	fi
	#
	return 0
}


waitForInternetConnection () {
	# Syntax:
	# 	waitForInternetConnection
	#
	# Assume we can wait for internet connection if hostapd started fine.
	# If it did not start, we maybe miss internet connectivity via mesh.
	#
	if [ -z "$(iw dev)" ]; then
		logAdd "[INFO] WiFi is not initialized"
		return 1
	fi
	#
	# Give WiFi mesh interface time to connect and get internet connectivity.
	logAdd "[INFO] Waiting for WiFi to initialize"
	UPTIME_IN_SECONDS="$(cat /proc/uptime | cut -d "." -f 1)"
	SECONDS_TO_SLEEP="$((120-${UPTIME_IN_SECONDS}))"
	sleep "${SECONDS_TO_SLEEP}"
	#
	return 0
}
# ---------------------------------------------------
# -------------- END OF FUNCTION BLOCK --------------
# ---------------------------------------------------
#
#
# Check command line.
if ( echo "${*}" | grep -q "force" ); then
	runInstall
	exit 0
fi
#
# Check if the script should run on boot.
if [ -f "/etc/_OPKG_INSTALL_COMPLETE" ]; then
	echo "[INFO] /etc/_OPKG_INSTALL_COMPLETE exists."
	exit 99
fi
#
waitForInternetConnection
# 
logAdd "[INFO] Checking internet connection ..."
if ( ! echo -e "GET / HTTP/1.1\r\nHost: downloads.openwrt.org\r\n" | nc downloads.openwrt.org 80 > /dev/null ); then
	logAdd "[INFO] No internet connection. Switching to offline mode."
	INSTALL_ONLINE="0"
fi
#
runInstall
if [ ! "$?" = "0" ]; then
	logAdd "[ERROR] One or more packages FAILED to install"
else
	logAdd "[INFO] All packages installed successfully"
	touch "/etc/_OPKG_INSTALL_COMPLETE"
fi
# 
logAdd "[INFO] Cleanup"
if [ ! -z "${PACKAGE_CACHE}" ]; then
	rm -rf "${PACKAGE_CACHE}"
fi
#
if [ "${REBOOT_REQUIRED}" = "1" ]; then
	logAdd "[INFO] Rebooting device"
	reboot -d 3
	exit 0
fi
#
logAdd "[INFO] Done."
exit 0

Things you should change and configure to your own needs in "opkg_reinstall.sh":

  • In the header
(...)
INSTALL_BATMANADV="1"
INSTALL_OPENVPN="1"
INSTALL_RELAYD="0"
INSTALL_WIFI_NON_CT_DRIVERS="1"
(...)
  • In function "opkgInstall": Insert your own package cocktail there which you want uninstalled and installed. If you don't need special conditional scenarios, you can delete the code until the "opkgRemove" block.
(...)
	PKG_TO_REMOVE="${PKG_TO_REMOVE} wpad wpad-basic"
	PKG_TO_INSTALL="${PKG_TO_INSTALL} libopenssl1.1 wpad-mesh-openssl"
(...)
(if INSTALL ONLINE ...)
(...)
# ...
opkgRemove ...

It works like this:

  • You upload the above mentioned files to your OpenWrt devices. (I've verified it working on TP Link Archer C7's, v2 and v5)
  • You manually create the file "/etc/_OPKG_INSTALL_COMPLETE" if you don't want the script to take action on next reboot (without a flash).
  • You log on to Web UI and perform a firmware flash with KEEP SETTINGS enabled.
  • After that, the router looses the "done marker" file: /etc/_OPKG_INSTALL_COMPLETE
  • The router will autorun the "opkg_reinstall.sh" script and look for an internet connection.
  • Two ways to go from there: You've either put "internet connection critical" IPK files to "/root/packages" (which MUST correspond to the version you're updating to) OR you have internet connection after flash. If the script finds something sufficient, it will continue to run and remove/install your packages using offline(or online) repository as you need it and - only after successful installation - reboot the router to get "VPN/mesh/etc." using the newly installed O'packages.

@user674574 @zorxd @Catfriend1

Yeah,mine does not need usb but is more complicated if you need packages to get even online. Therefore I have a second script to find and download the correct ipk files from the repo for each make and model of router I have.