Extroot using (z)ram drive with the backup&restore option over NFS share

Community Builds, Projects & Packages
1

Introduction

  • If you have a router without a USB port, it is very likely that with more intense usage of your router, you will need more disk space.

  • You may want to test new software or config options without changing your original setup.

These can be achieved by creating a ram drive and using it as your new storage space. With this option, you can install and test new packages beyond your flash drive’s capacity to see how they work without changing your original setup. As soon as you reboot your router you will go back to your former state. The ability of saving your ram drive in a NFS share and restoring it back to your router is also given to you as an option.

Additionally, you can use your NFS share as your external drive without using any (z)ram drive.

How it works

The script simply creates a drive in the memory. This drive could be a ‘tmpfs’ or ‘zram’ . The zram drive basically compresses the data with the average 2:1 compression ratio. That means you will consume less memory for the drive.

Then, it creates an ‘overlayfs’ and mount it as the new root. The new overlayfs based on the combined /rom (squashfs) and /overlay (jffs2) directories as the lowerdir and the ram drive as the upperdir. The lowerdir is read only, so your previous data is protected.

After the successful ram-root process, the router gets ready for your enjoyment.

If the swap file option over nfs share is selected, the script will make it ready for you, too.

Prerequisites

  • The latest Openwrt/Lede firmware with the kernel 4+

  • kmod-fs-nfs and nfsutils packages for the backup option

  • swap-utils package for the swap option

  • kmod-fs-ext4, e2fsprogs, kmod-zram packages for the zram drive option

Download link: https://drive.google.com/file/d/1MYzSseqjWeaAt3QgxAzpt6DsPMPXAePR/view?usp=sharing

Installation

Download the provided ‘ram-root.tar.gz’ file and extract it in the / directory: 'tar -C / -vzxf ram-root.tar.gz'

After the extraction you should have a new directory in /etc named ‘ram-root’. You may need to enter ‘chmod +x /etc/ram-root/ram-root.sh’ command to make it executable.

Install the prerequisites considering your available flash drive space and 'config' options. The script will check all of them whether they are installed and will do it automatically if not.

You should additionally make some changes in the ‘config’ file according to your needs before running the script.

How to use

/etc/ram-root/ram-root.sh or
/etc/init.d/ram-root start
Run the first command from your console to start the ram-root process for the first time. The script will copy itself to /etc/init.d. So, you can use the second command after the first run.

/etc/init.d/ram-root stop
This is the command to stop the process and reboot the router to its previous state. If backup option is selected, a backup will be done before rebooting.

/etc/init.d/ram-root backup
If you want to keep your new ram-root settings, it is hopefully possible to make a backup in a ‘nfs’ share. The nfs server could be anywhere reachable by your router. If you select the backup option in the ‘config’ file, the script will make the backup for the first installation automatically. You can also backup any time you want by initiating the command from your console.

/etc/init.d/ram-root reset
If you need to reset your ram drive backup, enter this command. The command will bypass your backup file and do a fresh start.

Notes
You can enable the autorun option by entering '/etc/init.d/ram-root enable' to boot your router through the ram drive, if you make sure that everything works quite nicely.

The ram drive size should be entered accordingly. Start with a small amount and install your extra packages by checking the available size each time before the installation. If you are running out of space, simply increase the ram drive size in the ‘config’ file and restart your router. Keeping the ram drive size just right, no more than required for your packages, helps the router to use the ram efficiently.

The previous state of the router is mounted under /old-root. You can simply switch to that directory and make changes.

UPDATE :

The changes with this updated version are:

1- There is no kernel restriction now. It runs with older kernels, too.
2- A new 'config' option CHROOT is added. You can find the info in the 'config' file.
3- ZIP archive option is added.
4- New colored console prompt.

3 Likes
2

(Moved to the Community Builds, Projects & Packages section.)

4

A newer version: https://drive.google.com/file/d/12GZKhkevArsceSedqefP0wl_kKWWaVLM/view?usp=sharing

5

thanks for sharing! this is very very cool...

6

You are very welcome.

This new version uses 'sshfs' instead of 'nfs' for server connection. Faster, safer and easier way of server interaction. The server configuration is automated by the script. No user intervention required.

Tip: You can upgrade the system with 'opkg upgrade pkgname' and backup the current setup with '/etc/init.d/ram-root backup'

Please let me know if you have suggestions.

1 Like
7

yes... i've started messing around with it a little... mainly

[ -z "$SERVER" ] && #@do_init/etc./etc/ to support local dir etc. @'SHARE' ||&& alternate mount types or just simply bypass the backup logix if not available

and i've added a PANIC_ACTION="reboot||none" to .cfg @ error for testing or maybe case *) type $PANIC_ACTION && $PANIC_ACTION etc. etc. so it will run a user defined function etc.
and added 'mount | grep -q ' /overlay' || error #@early to validate there is one available
nc -z IP PORT # is very handy on non-openwrt systems @chkconnection but I havent found a good native openwrt equivalent... you could possibly depend on full netcat or openssh-client-utils can also perform similar validation I think #i.e.  netcat -w 3 -p 22 10.2.3.157 2>&1 | grep -q refused
8

it informs the user and reboots in 30 secs if there is any error. You may break counting seconds and search for the cause. For 'init' operation it just stops operation wait for your inspection.

1 Like
9

This is another script 'opkgupgrade.sh' for easy 'opkg upgrade'. You can use it with 'ram-root' accordingly. I usually run this script to upgrade the system instead of flashing a new firmware. It works flawlessly.

#!/bin/ash

#takes one argument/parameter: < -i > for interactive upgrade
#example: opkgupgrade.sh [-i]

# set -x

#ANSI — Color Escape Codes

#Option	Description
#-e	Enable interpretation of backslash escapes
#\e[	Begin the color modifications
#COLORm	Color Code + ‘m’ at the end
#\e[0m	End the color modifications

#ANSI Code	Description
#0		Normal Characters
#1		Bold Characters
#4		Underlined Characters
#5		Blinking Characters
#7		Reverse video Characters

#echo -e "\e[1mBold Text\e[0m"
#echo -e "\e[1;33;4;44mYellow Underlined Text on Blue Background\e[0m"

#Color		Foreground Code	Background
#Black		30		40	
#Red		31		41	
#Green		32		42	
#Brown		33		43	
#Blue		34		44	
#Purple		35		45	
#Cyan		36		46	
#Light Gray	37		47
#Dark Gray	1;30		1;40	
#Light Red	1;31		1;41	
#Light Green	1;32		1;42	
#Yellow		1;33		1;43	
#Light Blue	1;34		1;44	
#Light Purple	1;35		1;45	
#Light Cyan	1;36		1;46	
#White		1;37		1;47

#echo -e "\e[31mRed Text\e[0m"
#echo -e "\e[1;34mLight Blue Text\e[0m"


ask_bool() {
    local default="$(( ${2:-0} ))"
    [ "$interactive" != "true" ] && return $default
    [ $default -eq 0 ] && echo -ne ${1}" \e[1mY${COL_RESET}/n : " || echo -ne ${1}" y/\e[1mN${COL_RESET} : "
    read -t 3 opt
    case "$opt" in
	y|Y) return 0;;
	n|N) return 1;;
	*) [ $default -eq 0 ] && echo -e ${COL_BLUE}"Y"${COL_RESET} || echo -e ${COL_BLUE}"N"${COL_RESET}; return $default;;
    esac
}

err() {
  cat <<EOF
Usage: $(basename ${0}) [-i]

Upgrades 'opkg' packages

    -i Interactive upgrade mode
  
EOF
  exit 1
}


COL_BLUE="\e[34m"
COL_RED="\e[31m"
COL_RESET="\e[0m"

[ $# -gt 1 ] && err
[[ $# -eq 1 && "$1" != "-i" ]] && err
[ "$1" == "-i" ] && interactive=true || interactive=false

dir=$(cat /etc/opkg.conf | grep lists_dir | awk '{print $3}')
[ -d $dir ] && [ $(ls -1 $dir/* | wc -l) -gt 0 ] && echo -e "Bypassing ${COL_BLUE}opkg update${COL_RESET}" || opkg update

FLAG=false
for name in $(opkg list-upgradable | awk '{print $1}'); do
  FLAG=true
  ask_bool "Upgrade ${COL_BLUE}${name}${COL_RESET} ?" && opkg upgrade $name
done

case $FLAG in
  true)  exit 0;;
  false) echo -e "\e[1;5;31mNo packages to upgrade"${COL_RESET}; exit 1;;
esac
1 Like
10

A little bit of changes:

Introduction:

  • If you have a router without a USB port, it is very likely that with more intense usage of your router, you will need more disk space
    and/or
  • You may want to test new software or config options or upgrade package(s) without changing your original setup to make sure everyting works properly.

These can be achieved by creating a ram drive or mounting a remote share and using it as your new storage space. With this option, you can install and test new packages beyond your flash drive’s capacity to see how they work without changing your original setup. As soon as you reboot your router you will go back to your former state. The ability of saving your ram drive in a local/remote share and restoring it back to your router is also given to you as an option.

Additionally, you can use your remote share as your external drive without using any (z)ram drive.

For further investigation please check the self-explanatory config file.

How it works:

if 'OVERLAY_DRIVE' is:

  • 'SSHFS' : this mounts a remote share. Then copies '/' directory into that mount and makes it your new root.

  • 'RAM' / 'ZRAM' : this creates a drive in the memory. Created drive could be a ‘tmpfs’ or ‘zram’ . The zram drive basically compresses the data with the average 2:1 compression ratio. That means you will consume less memory for the drive at the expense of more CPU usage. Then, it creates an ‘overlayfs’ and mounts it as the new root. The new overlayfs based on the combined /rom (squashfs) and /overlay (jffs2) directories as the 'lowerdir' and the ram drive as the 'upperdir'. The 'lowerdir' is read-only, so your previous data is protected.

After the successful ram-root process, the router gets ready for your enjoyment.
If the swap option is selected, the script will make it ready for you, too.

Prerequisites:

'netcat, sshfs' packages for the backup or remote share option

'kmod-fs-ext4, e2fsprogs, kmod-zram' packages for the zram drive option

'swap-utils' package for the swap option

Installation:

Download the provided ‘ram-root.tar.gz’ file and extract it in the / directory: 'tar -C / -vzxf ram-root.tar.gz'

After the extraction you should have a new directory in '/etc' named ‘ram-root’. You may need to enter ‘chmod +x /etc/ram-root/ram-root.sh’ command to make it executable.

Install the prerequisites considering your available flash drive space and 'config' options. The script will check all of them whether they are installed and will do it automatically if not.

You should additionally make some changes in the ‘config’ file according to your needs before running the script.

How to use:

'/etc/ram-root/ram-root.sh init'
This has to be run for the first time from your console to start the ram-root process. The script will copy itself into '/etc/init.d' for further operations.

'/etc/init.d/ram-root start'
This starts ram-root opereation from your backup if you already have one or from scratch.

'/etc/init.d/ram-root stop'
This is the command to stop the process and reboot the router to its previous state. If backup option is selected, a backup will be done before rebooting.

'/etc/init.d/ram-root backup'
If you want to keep your new ram-root settings, it is hopefully possible to make a backup in your local/remote share. The server could be anywhere reachable by your router. If you select the backup option in the ‘config’ file, the script will make the backup for the first installation automatically. You can also backup any time you want by initiating the command from your console.

'/etc/init.d/ram-root reset'
If you need to reset your ram drive backup, enter this command. The command will bypass your backup file and do a fresh start.

'/etc/init.d/ram-root upgrade'
Run 'opkg upgrade' command to upgrade the ram-root system.

Notes:

You can enable the autorun option by entering '/etc/init.d/ram-root enable' to boot your router through the ram drive, if you make sure that everything works quite nicely.

The ram drive size should be entered accordingly. Start with a small amount and install your extra packages by checking the available size each time before the installation. If you are running out of space, simply increase the ram drive size in the ‘config’ file and restart your router. Keeping the ram drive size just right, no more than required for your packages, helps the router to use the ram efficiently.

The previous state of the router is mounted under '/old-root'. You can simply switch to that directory and make changes to your base system.

Download link: https://drive.google.com/file/d/1dE11eWERVkw4jzKvHLQQkYdwpoHCc4Y1/view?usp=sharing

1 Like
11

New download link with a few useful scripts:

https://drive.google.com/file/d/1W3RdX-m6NiBgApSbDWkoExJHhF9zTWqq/view?usp=sharing

1 Like
12

Hi, google says

Sorry, the file you have requested does not exist.
Make sure that you have the correct URL and the file exists.

Could you, please, share it once again
Thank you

1 Like
14

Thank you!
-- Dmitry

15

hi.

I shared a new link.

Let me know if anything wrong.

Thanks.

1 Like
16

New link: https://drive.google.com/file/d/1GnlPjtuuPnlsDFaPCjCTXcMHESLLOjVN/view?usp=drivesdk

2 Likes
17

Ok, thank you. As I can see, this version works only with sshfs. May be you still have a version, that works with nfs? As far as security is not a concern in my house, nfs looks more easy to manage for me.

--Dmitry

18

Thanks for sharing. I have done experiments with nfs and sshfs with Nextcloud and was surprised how fast sshfs can be. For devices that are low on both disk and memory, I was wondering if iSCSI could be an option - I have used it with Virtualbox to mount an old Win7 image - very good performance.

Based on the following link it should be possible - I did a quick Google search and don't see any forum discussions on the subject - would be interested to hear your feeling - some Arm bootloaders allow for iSCSI booting, but it should be possible to flush a minimal Kernel containing an iSCSI initiator.

19

Hi!

Unfortunately I do not have that 'nfs' version anymore. It has been almost 2 years since I switched to 'sshfs' only connection. This way it is considerably simplified for making connections to the server. Actually, you do not have to do any setup on the server. First time running the script with the 'init' option sets up everything automatically.

If you want to try new things without saving on a server you can select BACKUP option 'N' and set 'SERVER' address as your own ip address in the config file. This way, you can install different applications and try them without changing your original setup.

Just to be curious, why do you need a NFS connection specifically?

Have a great day!

20

Hi, the reason is simple. nfs client without encryption loads the system less then sshfs, at least in my tp-link wr1043nd. My idea is to put all the required packets ( mwan3, dawn, gre, usb-utils, openvpn, wireguard ) in firmware and all other packets like luci, tcpdump, netperf, nano, etc put on external drive.

I started to learn your script and noticed, that you make overlay to ram disk only, at least I didn't find how to make overlay to external drive. Is that correct? If yes, why?

21

you can use any drive instead of ram drive. This is what you need to focus on:

do_logger "Creating: ram disk"

do_exec mount -t tmpfs -o rw,nosuid,nodev,noatime tmpfs $NEW_OVERLAY

If you change it for mounting your NFS share that should be enough. But please keep thisin your mind that it is much slower and unstable than a ram drive. For limited memory devices you can use zram-swap.

Faruk Tezcan

22

Thank you, Faruk. I'll try. If I come up with something interesting, I'll post it here

I'll check this option too. Thanks for reference

--Dmitry