Mounting swap on zram with discard

rsalvaterra · November 15, 2019, 2:51pm

Hi, guys! Zram supports discard with really useful semantics, immediately returning the pages backing freed blocks to the kernel. How about changing the /etc/init.d/zram script and adding the -d option to the swapon command? (I've actually been doing this on my builds for a long time.) Thanks!

NoTengoBattery · November 15, 2019, 3:31pm

Oh. That's a good point! Thanks for the advice. I've investigating and zram seems to return the pages as needed but the discard option will definitely help to perform the action explicit.

I'm using a custom version of the script because it did not match my quality criteria, so in turn of your advice, maybe you can find something useful in here:

github.com

NoTengoBattery/openwrt/blob/fc44a-v0.20/files/bin/zram

#!/bin/sh /etc/rc.common

MEMORY_PERCENT=20.835
KIB=1024
EXTRA_COMMANDS="status"

compressor="$(uci -q get system.@system[0].zram_comp_algo)"
case $compressor in
  'zstd')    EXPECTED_RATIO=4.80 ;;
  'deflate') EXPECTED_RATIO=4.70 ;;
  'lz4hc')   EXPECTED_RATIO=3.60 ;;
  'lzo')     EXPECTED_RATIO=3.40 ;;
  'lz4')     EXPECTED_RATIO=3.35 ;;
  *)         EXPECTED_RATIO=3.00 ;;
esac

# Log messages
log_msg() {
  local MESSAGE="$1"
  logger -s -t zram -p user.info "$1"

This file has been truncated. show original

Ignore the comments about 1/4 of the ram. I forgot to update it.

rsalvaterra · November 15, 2019, 4:04pm

Juicy! You size your zram swap device based on the selected compression algorithm's expected compression ratio, that would also be quite nice to have. I do the calculation by hand and just specify the calculated size in /etc/config/system.
What kernel version are you running, in order to have zstd (which I'm really looking forward to)? And what's your use case for zswap?
All in all, wonderful stuff! Thanks!

NoTengoBattery · November 15, 2019, 4:29pm

Well, I'm using a custom kernel for my "Comunity Build". Kernel 4.19 includes zstd but need to be enabled. zstd is (theorically) upto 5 times faster than deflate. Still slower than lz4 and lz4hc.
In order for the script to excels you'll need to use your custom kernel with zstd and lz4hc, and to manually change the algorith im /etc/config/system because LuCI does not recognize it. Also, leave empty the size and let it guess a "best case" value based on the expected compression ratio.
This is the relevant kernel seed:

github.com

NoTengoBattery/openwrt/blob/8c616e9508fddaae99b15a45ca8922bc1fb860dc/kernel.seed#L12


# --- --- --- Start of the kernel configuration seed --- --- --- #
CONFIG_DEFAULT_HOSTNAME="EA6350v3"
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y


# -- Boot
CONFIG_ATAGS=y
CONFIG_CMDLINE="rcutree.kthread_prio=5"
CONFIG_CMDLINE_EXTEND=y
# -- Crypto
CONFIG_CRYPTO_LZ4HC=y
CONFIG_CRYPTO_ZSTD=y
# -- CPU/Thermal Governor
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=n
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y
CONFIG_THERMAL_DEFAULT_GOV_POWER_ALLOCATOR=y
CONFIG_THERMAL_DEFAULT_GOV_STEP_WISE=n
# -- IO Scheduler
CONFIG_DEFAULT_CFQ=y
CONFIG_DEFAULT_DEADLINE=n

The use case for zswap is that it's able to provide decent savings and when using lz4 at almost no delay/CPU cost. While evicting older pages to zram which requieres more time but does a better job at compressing. The relevant explanation is found here:

github.com

NoTengoBattery/openwrt/blob/8c616e9508fddaae99b15a45ca8922bc1fb860dc/notes-final.txt#L152


For using a feature which needs kernel modules (such as NTFS-3G), you should use
`opkg` to manually install the kernel modules that are needed. Once the kernel
dependencies are installed, you can install the package from the OpenWrt
snapshots repository.


Sometimes `opkg` will refuse to install the kernel modules. It does not matter
because the packages in the compressed *.tar.bz2* came from the running kernel.
You can force opkg to install the packages without risk, given that you use the
packages corresponding to the running version.


### Memory subsystem


The device has only 256 MB of RAM. That's plenty but sometimes it might get a
little short (i.e. when indexing a huge library with miniDLNA). For this, the
memory subsystem has been tuned to support almost any NAS/Adblock load:


- First, the system will try to use the swap cache, that's a pool of the 5%
  (by default) for compressed memory that the system is not actively using.
  - It's fast and it's compression rate is good (about 3.00), so you can store
    upto 38 MB of data using only 5 % of the RAM.
- When the pressure gets higher, the system will begin to swap. But before it

So, in theory your kernel can get pages in and out in the fastest way possible from the zswap pool as needed, and it will evict pages that are not used in log time periods to zram, allowing the kernel to place more "rarely used" pages in zswap.

And the relevant script that enables zswap is here:

github.com

NoTengoBattery/openwrt/blob/8c616e9508fddaae99b15a45ca8922bc1fb860dc/files/bin/prebootz#L74


  case $(board_name) in
  linksys,ea6350v3)
    # -- Take advantage of zram. This is a tradeoff between reading slow and
    #    expensive SQUASHFS blocks or swapping to a slow device. We prefer
    #    to swap.
    VM=/proc/sys/vm
    _echo_and_print "75"     "$VM/swappiness"
    # -- Take advantage of UKSM
    UKSM=/sys/kernel/mm/uksm
    _echo_and_print "1"      "$UKSM/run"
    # -- Take advantage of zswap
    ZSWAP=/sys/module/zswap/parameters
    _echo_and_print "lz4"    "$ZSWAP/compressor"
    _echo_and_print "z3fold" "$ZSWAP/zpool"
    _echo_and_print "5"      "$ZSWAP/max_pool_percent"
    _echo_and_print "Y"      "$ZSWAP/enabled"
    ;;
  esac
}


boot() {

You should note that they are scripts in /bin, that's because I updated all of them to be procd services, as shown here:

github.com

NoTengoBattery/openwrt/blob/8c616e9508fddaae99b15a45ca8922bc1fb860dc/files/etc/init.d/zram#L4


#!/bin/sh /etc/rc.common


START=15
USE_PROCD=1
NAME="zram"
PROG="/bin/zram"
EXTRA_COMMANDS="status"


start_service() {
  procd_open_instance "zram"
  procd_set_param command "$PROG"
  procd_append_param command start
  procd_set_param stdout 0
  procd_set_param stderr 0

Why I would do such a crazy thing? Well... I begun my days as developer in XDA Developer (always in my heart) about 7 or 8 years ago. So we cared a lot about memory costraints and optimizations for what Android and ARM was, in general, in that moment. So I have some crazy techniques under my sleeve.