Extroot using (z)ram drive with the backup&restore option over NFS share

Community Builds, Projects & Packages
23

Hi Guys, thank you for the references. I did a small lab testing of all the possibilities to have overlay to be located on the network share, namely iscsi, sshfs, nfs and cifs and would like to share the results with you.
What I did is very simple. I made a firmware with included one by one these network share access methods and checked the free blocks left after a first boot. This is the most important value in my case. Then I created an overlay on a network share, downloaded with opkg several packages (large and small) on it

mkdir /dnl && cd /dnl
opkg update
opkg install --download-only luci mc luci-ssl nano asterisk dawn mariadb-server php8

and installed those packages, measuring time of execution and cpu load.

The results are below.
What surprised me much is that sshfs left far way less free blocks for the user than nfs and cifs. I've thought that having dropbear in the firmware already installed, sshfs should be a tiny wrap-up for a dropbear. I was wrong. It either not a thin wrap-up, or not a wrap-up at all.


**open-iscsi:**

Free blocks after installation: 480

time -v opkg install --cache /dnl/ *.ipk
	User time (seconds): 79.62
	System time (seconds): 20.84
	Percent of CPU this job got: 86%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 1m 56.27s
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 34656
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 94
	Minor (reclaiming a frame) page faults: 176415
	Voluntary context switches: 38607
	Involuntary context switches: 33005
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

**sshfs:**

Free blocks after installation: 864

time -v opkg install --cache /dnl/ *.ipk
	User time (seconds): 77.32
	System time (seconds): 21.10
	Percent of CPU this job got: 65%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 2m 30.77s
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 48272
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 322
	Minor (reclaiming a frame) page faults: 172187
	Voluntary context switches: 37709
	Involuntary context switches: 32282
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

**nfs:**

Free blocks after installation: 1504

time -v opkg install --cache /dnl/ *.ipk
	User time (seconds): 82.79
	System time (seconds): 26.60
	Percent of CPU this job got: 72%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 2m 29.97s
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 38112
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 427
	Minor (reclaiming a frame) page faults: 184747
	Voluntary context switches: 40550
	Involuntary context switches: 49396
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

**cifs:**

Free blocks after installation: 1568

time -v opkg install --cache /dnl/ *.ipk
	User time (seconds): 82.46
	System time (seconds): 28.49
	Percent of CPU this job got: 67%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 2m 44.41s
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 29440
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 578
	Minor (reclaiming a frame) page faults: 182750
	Voluntary context switches: 40182
	Involuntary context switches: 48728
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

-- Dmitry

24

A simple script to create root overlay over nfs. The main idea is to have a disk image file on nfs share that will be attached as a loop device in openwrt and used as root overlay

OLD_ROOT="/old_root"                                                                                                                        
RAM_ROOT="/ram-root"                                                                                                                        
NEW_ROOT="/tmp/root"                                                                                                                        
NEW_OVERLAY="/tmp/overlay" 
ITM_OVERLAY="/tmp/itm_overlay"
IMAGE="roof-router.img"                                                                                                                 
NFS_SHARE="local-nfs:/overlay/nfsshare/"

mkdir -p $NEW_OVERLAY $ITM_OVERLAY $RAM_ROOT
mount -t nfs -o nolock $NFS_SHARE $ITM_OVERLAY
losetup /dev/loop0 $ITM_OVERLAY/$IMAGE
mount /dev/loop0 $NEW_OVERLAY
mkdir -p $NEW_ROOT ${NEW_OVERLAY}/upper ${NEW_OVERLAY}/work
mount -t overlay -o noatime,lowerdir=/,upperdir=${NEW_OVERLAY}/upper,workdir=${NEW_OVERLAY}/work $RAM_ROOT $NEW_ROOT                 
mkdir -p ${NEW_ROOT}${OLD_ROOT}; mount -o bind / ${NEW_ROOT}${OLD_ROOT}                                                             
mount -o noatime,nodiratime,move /proc ${NEW_ROOT}/proc                                                                             
pivot_root $NEW_ROOT ${NEW_ROOT}${OLD_ROOT}                                                                                         
for dir in /dev /sys /tmp; do mount -o noatime,nodiratime,move ${OLD_ROOT}${dir} ${dir}; done                                       
mount -o noatime,nodiratime,move $NEW_OVERLAY /overlay

Disk image file roof-router.img should exist in local-nfs:/overlay/nfsshare/ and be formatted as ext4. Preparing of this image can be done in any system. In most cases 100MB image is enough for everything you might need in openwrt. Each openwrt host mush have its own separate disk image file on nfs share.
To restore original configuration reboot is required

4 Likes
25

Hello how are you?
Wow how amazing, it was exactly a solution like that I was thinking.
I mounted with rclone a virtual disk to be able to install packages in a router without USB using the flag --dest <path> n opkg, like for example opkg -d apk install luci-app-openvpn , where "apk" is the name of the path /mnt/apk which I edited in the opkg.conf file. I tried using googledrive, but I was not happy. I saw that xman was happy to try his script with nextcloud. I tried to download your script, but the link no longer works. Can you share it again? I would like to try your solution.

Blockquote

26

This is the latest version of my script:
https://drive.google.com/file/d/11MTon42fBQWlZrFG26m-hyB3dfXwJaI3/view?usp=drivesdk

Introduction:
+++++++++++++

  • If you have a router without a USB port, it is very likely that with more intense usage of your router, you will need more disk space
    and/or
  • You may want to test new software or config options or upgrade package(s) without changing your original setup to make sure everyting works properly.

-These can be achieved by creating a ram drive (tmpfs) in your router's memory and using it as your new storage space. With this option, you can install
and test new packages beyond your flash driveā€™s capacity to see how they work. As soon as you reboot your router you will go back to your former state.

-The ability of saving your ram drive in a local/remote share and restoring it back to your router is also given to you as an option.

-Installing optional 'zram-swap' is highly adviced as this package provides you approx. %50 more storage space.

-For further explanation/investigation please check the self-explanatory config file.

Installation:
+++++++++++++

-Download the provided ā€˜ram-root.tar.gzā€™ file and extract it with this command: 'tar -C / -vzxf ram-root.tar.gz'

-After the extraction you should have a new directory named ā€˜/ram-rootā€™.

-You should additionally make some changes in the ā€˜configā€™ file according to your needs before running the script for the 1st time.

-Install optional 'zram-swap' package considering your available flash drive space.

-The script will check if all the required packages are installed and will do it automatically if not.

How to use:
+++++++++++

  • '/etc/ram-root/ram-root.sh init'
    This has to be run for the first time from your console to start the ram-root installation on your router and the server.
    The script will copy itself into '/etc/init.d' for further operations.

  • '/etc/init.d/ram-root start'
    This starts ram-root opereation from your backup if you already have one.

  • '/etc/init.d/ram-root stop'
    This is the command to stop the process and reboot the router to its original state.
    If backup option is selected, a backup will be done before rebooting.

  • '/etc/init.d/ram-root backup'
    If you want to keep your new ram-root settings, it is possible to make a backup in your local/remote share.
    The server could be anywhere reachable by your router.
    If you select the backup option in the ā€˜configā€™ file, the script will make the backup for the first installation automatically.
    You can also backup any time you want by initiating the command from your console.

  • '/etc/init.d/ram-root reset'
    If you need to reset your ram drive backup and make a fresh start, enter this command. The command will bypass your backup file.

  • '/etc/init.d/ram-root status'
    Shows ram-root status if it is running or not.

  • '/etc/init.d/ram-root upgrade'
    Run 'opkg upgrade' command to upgrade the system.

Notes:
++++++

You can enable the autorun option by entering '/etc/init.d/ram-root enable' to boot your router through the 'ram-root',
if you make sure that everything works quite nicely.

The previous state of the router is mounted under '/old-root'. You can simply switch to that and make changes to your base system.

Included scripts in 'tools' directory:
++++++++++++++++++++++++++++++++++++++

'opkgclean.sh' : Cleans unsuccesful installation attempt debris due to flash storage shortage

'opkgdeps.sh' : Lists all dependant packages for the given package

'opkgupgrade.sh' : Upgrades whole system packages interactively

'overlaypkgs.sh' : Saves user-installed packages before upgrade and restore them later.

'ssh-copy-id' : Copies your ssh credentials to the remote device for easier ssh connections.

27

Hi. Great work. Do you think the script could go to GitHub for better community development action?

28

Hi there.

I have no experience on GitHub. Do you mind giving me some information?

Thank you for your concern.

29

I posted my solution at 22.03.3 tiny official(!) imagebuilder configuration for 4/32MB devices (ath79, TL-WA860RE v1 etc.), using ramdisk extroot and develop it on GitHub.

30

I have created an account on GitHub for this script:

2 Likes
31

If you want to install packages that should not be included in your backup, you can define them in the config file. Those packages will be installed each time automatically when you start ram-root operation. Otherwise, all the packages that you install and/or other changes that you make will be saved by making a backup.

Example: Lets assume your router's flash capacity is very limited. By defining 'PACKAGES' in the config file you can install them on-the-fly. If you make configuration changes later, making a backup includes only those changes. That means you are saving your flash drive capacity. The penalty, in this case, is that those packages have to be installed next time if you reboot the router.