OpenWrt Forum Archive

Topic: F2FS (Flash Friendly Filesystem) support

The content of this topic has been archived on 27 Nov 2017. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Why not including F2F2 package in trunk? It exists since mainline kernel 3.8 and trunk is currently > 3.8, IIRC.
How much friendly is?: Phoronix benchmarks

Lets start:
How do I create a package for f2fs-tools?
Is "package/kernel/linux/modules/fs.mk" a good place for the patch?

cd ~/build/openwrt/trunk
svn diff
Index: package/kernel/linux/modules/fs.mk
===================================================================
--- package/kernel/linux/modules/fs.mk    (revisión: 39223)
+++ package/kernel/linux/modules/fs.mk    (copia de trabajo)
@@ -136,6 +136,25 @@
 $(eval $(call KernelPackage,fuse))
 
 
+define KernelPackage/fs-f2fs
+  SUBMENU:=$(FS_MENU)
+  TITLE:=F2FS filesystem support
+  KCONFIG:= \
+    CONFIG_F2FS_FS \
+    CONFIG_F2FS_STAT_FS=y \
+    CONFIG_F2FS_FS_XATTR=y \
+    CONFIG_F2FS_FS_POSIX_ACL=n
+  FILES:= $(LINUX_DIR)/fs/f2fs/f2fs.ko
+  AUTOLOAD:=$(call AutoLoad,30,f2fs)
+endef
+
+define KernelPackage/fs-f2fs/description
+ Kernel module for F2FS filesystem support
+endef
+
+$(eval $(call KernelPackage,fs-f2fs))
+
+
 define KernelPackage/fs-hfs
   SUBMENU:=$(FS_MENU)
   TITLE:=HFS filesystem support
svn: El directorio «package/mac80211/.svn» con la información de administración de la CdT no está

Offtopic: I manually "rm -rf package/mac80211/*", because it conflicts with package/kernel/mac80211, so every svn command breaks there. How do I recover the svn health?

make menuconfig
  -> Kernel modules                                                   │  
    -> Filesystems
      -> kmod-fs-f2fs (M)

I would also like a submenu for the extra options, but I make the right decision for you wink
    CONFIG_F2FS_STAT_FS=y Because that fancy name looks required tongue
    CONFIG_F2FS_FS_XATTR=y Because from kernel 3.14 small files are saved inline the 4 kiB f2fs inode in a xattr*
    CONFIG_F2FS_FS_POSIX_ACL=n Because I don't use ACL tongue
* Like inline_data in ext4 from kernel 3.8
The package depends on "BLOCK", but I don't know how to enforce the dependence nor which package is "BLOCK".

make world
ls -la bin/*/packages/kmod-fs-f2fs*
-rw-r--r-- 59524 feb  4 18:21 bin/ar71xx/packages/kmod-fs-f2fs_3.10.26-1_ar71xx.ipk

Everything looks good, test follows... Soon.

Because routers with FTL devices for storage are rare, if not non-existing. Almost all routers have raw flash as storage, so F2FS is useless here.

Almost all devices have USB ports support (especially Atheros ones) and there we plug in our flash based devices, and for this purpose F2FS is built! We need F2FS in OpenWrt no to replace JFFS on internal flash but to extend it for EXTERNAL flash devices (SD cards, USB drives, etc...)

Zajec wrote:

Because routers with FTL devices for storage are rare, if not non-existing. Almost all routers have raw flash as storage, so F2FS is useless here.

You missed the point, nobody is talking about replacing JFFS with F2FS for internal storage.

(Last edited by valentt on 16 Feb 2014, 22:43)

valentt wrote:

Almost all devices have USB ports support (especially Atheros ones) and there we plug in our flash based devices, and for this purpose F2FS is built!

OK. Follow https://dev.openwrt.org/wiki/SubmittingPatches then. Patches from forum are not going to be picked wink

Nilfred wrote:

Offtopic: I manually "rm -rf package/mac80211/*", because it conflicts with package/kernel/mac80211, so every svn command breaks there. How do I recover the svn health?

You should have used "svn del" in the first place to keep svn happy. Easiest might be first to "svn revert package/mac80211" to recover the missing files and make svn happy, and then use "svn del package/mac80211" to remove them with an svn approved way.

Other alternative might be "svn resolve" with some additional options. (as "svn status" probably currently shows an error/conflict related to that missing package directory)

(Last edited by hnyman on 17 Feb 2014, 12:44)

The proposed patch is going to break some platforms. There are still a few laggards with 3.3 or 3.6, so you can't introduce functionality requiring 3.8+ without having a kernel version dependency.

I think it should be:
DEPENDS:=@!LINUX_3_3 @!LINUX_3_6

See e.g. http://nbd.name/gitweb.cgi?p=openwrt.gi … =HEAD#l469

(Last edited by hnyman on 17 Feb 2014, 09:31)

Tests doesn't conduct well: It doesn't mount.

My svn health has been restored:

svn status
svn resolved package/mac80211
svn status
rm -rf package/mac80211
svn status

There was a "Conflict": The directory used to have my mess with regdb.txt so it didn't automatically delete with svn up.

Tested so far:
- Install f2fs-tools in Ubuntu http://mirrors.kernel.org/ubuntu/pool/u … _amd64.deb

sudo mkfs.f2fs -l f2fs -s 1 -z 1 /dev/sdc2 wrote:

F2FS-tools: Ver: 1.1.0 (29-Nov-2012)
Info: segments per section = 1
Info: sections per zone = 1
Info: sector size = 512
Info: total sectors = 931840 (in 512bytes)
Info: zone aligned segment0 blkaddr: 512
Info: This device doesn''t support TRIM
Info: format successful

Then there is f2fs in LuCI mount dropdown list! smile
defaults, enable, etc. Save & Apply: nothing sad

ssh wrote:

mount -t f2f2 -o rw,sync,noatime,nodiratime /dev/sda2 /mnt/dt512

nothing sad

ssh wrote:

logread

nothing sad

Go back to Ubuntu

sudo mke2fs -t ext4 -I 512 -L ExtRoot -m 0 -O ^has_journal,^resize_inode,^uninit_bg,extent,meta_bg,flex_bg /dev/sdc2

mount -t ext4 -o rw,sync,noatime,nodiratime /dev/sda2 /mnt/dt512

ext4 do mount happily sad

Image setup for trunk r39586 wrote:

for router in WR1043 WR842 MR3420 MR3220
do
echo "CONFIG_TARGET_ar71xx=y
# CONFIG_TARGET_ar71xx_generic_Default is not set
CONFIG_TARGET_ar71xx_generic_TL$router=y
CONFIG_PACKAGE_block-mount=y
CONFIG_PACKAGE_luci=y
CONFIG_PACKAGE_luci-app-qos=y
CONFIG_PACKAGE_kmod-fs-ext4=y
CONFIG_PACKAGE_kmod-fs-f2fs=y
CONFIG_PACKAGE_kmod-ledtrig-heartbeat=y
CONFIG_PACKAGE_kmod-input-gpio-keys-polled=y
CONFIG_PACKAGE_kmod-usb-storage=y
CONFIG_ATH_USER_REGD=y
CONFIG_PACKAGE_prosody=m
CONFIG_PACKAGE_umurmur-polarssl=m
CONFIG_PACKAGE_umurmur-openssl=m
CONFIG_PACKAGE_exfat=m
CONFIG_PACKAGE_exfat-utils=m
CONFIG_BUSYBOX_CUSTOM=y
CONFIG_BUSYBOX_CONFIG_INOTIFYD=y" > .config
make defconfig
make -j4 world
done

There was a typo, now I got something else:

mount -t f2fs /dev/sda2 /mnt/dt512 wrote:

mount: mounting /dev/sda2 on /mnt/dt512 failed: Invalid argument

logread | tail wrote:

Tue Feb 18 22:49:26 2014 kern.err kernel: [  117.500000] F2FS-fs (sda2): Failed to get valid F2FS checkpoint

Back to Ubuntu, I create a folder to have some nonempty data.

sudo mount -t f2fs -o rw,sync,noatime,nodiratime /dev/sdc2 /media/mnt
sudo mkdir /media/mnt/test
sudo umount /media/mnt

Still a no go.
I'm running out of ideas.

If I have f2fs-tools in OpenWrt life would be easier.
I already have exfat-tools. f2fs-tools shouldn't be hard.

(Last edited by Nilfred on 19 Feb 2014, 04:23)

New findings:
Backports found: https://github.com/nowcomputing/f2fs-backports
This guy cross-compile f2fs-tools: http://bkhome.org/blog2/?viewDetailed=00122
The config from Ubuntu is different than I specified in OpenWrt:

grep F2FS /boot/config-3.11.0-18-generic wrote:

CONFIG_F2FS_FS=m
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
CONFIG_F2FS_FS_SECURITY=y

mount options wrote:

background_gc_off    Turn off the cleaning operation, aka garbage collection, in background triggered when I/O subsystem is idle.
disable_roll_forward    Disable the roll-forward recovery routine during SPOR.
discard    Issue discard/TRIM commands when a segment is cleaned.
no_heap    Disable heap-style segment allocation in which finds free segments for data from the beginning of main area, while for node from the end of main area.
nouser_xattr    Disable Extened User Attributes. Note: xattr is enabled by default if CONFIG_F2FS_FS_XATTR is selected.
noacl    Disable POSIX Access Control List. Note: acl is enabled by default if CONFIG_F2FS_FS_POSIX_ACL is selected.
inline_xattr    Enable the inline xattrs feature.
inline_data    Enable the inline data feature: New created small(<~3.4k) files can be written into inode block.

svn diff wrote:

Index: package/kernel/linux/modules/fs.mk
===================================================================
--- package/kernel/linux/modules/fs.mk    (revisión: 39729)
+++ package/kernel/linux/modules/fs.mk    (copia de trabajo)
@@ -136,6 +136,26 @@
$(eval $(call KernelPackage,fuse))


+define KernelPackage/fs-f2fs
+  SUBMENU:=$(FS_MENU)
+  TITLE:=F2FS filesystem support
+  DEPENDS:= @!LINUX_3_3 @!LINUX_3_6
+  KCONFIG:= \
+    CONFIG_F2FS_FS \
+    CONFIG_F2FS_STAT_FS=n \
+    CONFIG_F2FS_FS_XATTR=n \
+    CONFIG_F2FS_FS_POSIX_ACL=n
+  FILES:= $(LINUX_DIR)/fs/f2fs/f2fs.ko
+  AUTOLOAD:=$(call AutoLoad,30,f2fs)
+endef
+
+define KernelPackage/fs-f2fs/description
+ Kernel module for F2FS filesystem support
+endef
+
+$(eval $(call KernelPackage,fs-f2fs))
+
+
define KernelPackage/fs-hfs
   SUBMENU:=$(FS_MENU)
   TITLE:=HFS filesystem support
Index: package/utils/e2fsprogs/patches/002-no_malloc_h.patch
===================================================================
--- package/utils/e2fsprogs/patches/002-no_malloc_h.patch    (revisión: 39729)
+++ package/utils/e2fsprogs/patches/002-no_malloc_h.patch    (copia de trabajo)
@@ -1,10 +1,12 @@
--- a/util/symlinks.c
+++ b/util/symlinks.c
-@@ -8,7 +8,6 @@
+@@ -8,9 +8,6 @@
  #endif
  #include <stdio.h>
  #include <stdlib.h>
+-#ifdef HAVE_MALLOC_H
-#include <malloc.h>
+-#endif
  #include <string.h>
  #include <fcntl.h>
  #include <sys/param.h>
Index: package/utils/e2fsprogs/Makefile
===================================================================
--- package/utils/e2fsprogs/Makefile    (revisión: 39729)
+++ package/utils/e2fsprogs/Makefile    (copia de trabajo)
@@ -8,8 +8,8 @@
include $(TOPDIR)/rules.mk

PKG_NAME:=e2fsprogs
-PKG_VERSION:=1.42.4
-PKG_MD5SUM:=b6e296f210d642361b7394437ff0f318
+PKG_VERSION:=1.42.9
+PKG_MD5SUM:=3f8e41e63b432ba114b33f58674563f7
PKG_RELEASE:=1

PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz

As you can see: I'm diverting to ext4 inline_data, because I can't get f2fs work for me. Hope someone take it from where I left sad

Were you able to get f2fs-tools compiled and working on OpenWrt? How? Also check out this great article regarding F2FS performance boost when using F2FS instead EXT3 or EXT4 - https://plus.google.com/+WentookOSHW/posts/D3Ny4f5uu9k

Found two really nice articles that explain why F2FS is better for consumer flash devices:
* http://unix4.com/f/f2fs-a-new-file-syst … 11115.html
* https://lwn.net/Articles/518988/

And here is a benchmark comparing F2FS vs EXT4 and btrfs:
* http://www.phoronix.com/scan.php?page=a … &num=3

(Last edited by valentt on 6 May 2014, 20:31)

valentt wrote:

Were you able to get f2fs-tools compiled and working on OpenWrt?

Nope.
I formated the pendrive in a Ubuntu box, then put the formated pendrive in a OpenWrt box.

I have made made own Git-branch for f2fs under openwrt, it is little and big endian mips tested (lantiq, ar71, brcm47xx)
See https://github.com/j0ju/openwrt-14.07 branch f2fs. I backported for big-endian HW some patches upto kernel 3.16.

I have also probably found a f2fs-tools bug, upstream has been notified. The f2fs-tools as a package will follow later into a repository.

I like to use f2fs as external rootfs, until now it is working fine.

(Last edited by abraXxl on 7 Oct 2014, 17:25)

I see that there is support for F2FS in 15.05 already, but I can't make it automount F2FS-formatted USB flash drive.
I have the following section in the /etc/config/fstab:

config mount
    option device '/dev/sda1'
    option target '/mnt/usbdrive'
    option enabled '1'
    option fstype 'f2fs'

but when I insert USD drive nothing happens. I have all necessary packages installed and I can mount it manually using

mount -t f2fs /dev/sda1 /mnt/usbdrive

and it is working fine after that. Automounting works for the same flash drive if I reformat it to ext4 and change option fstype in /etc/config/fstab so difference is only in the FS type.

Where can I look to understand why this happens?

I tested f2fs + openwrt trunk  on arm and ar71xx.
Arm works well.
Ar71xx has troubles with f2fs tools that look like they're caused by big-endianness.
Also on ar71xx i experienced filesystem corruptions of f2fs formatted in ubuntu kernel 3.13.
Could not make fs on the device because f2fs-tools broken.

(Last edited by bolvan on 24 Mar 2016, 17:15)

bolvan wrote:

Ar71xx has troubles with f2fs tools that look like they're caused by big-endianness.
Also on ar71xx i experienced filesystem corruptions of f2fs formatted in ubuntu kernel 3.13.
Could not make fs on the device because f2fs-tools broken.

I'm currently testing f2fs + OpenWrt Chaos Calmer on ar71xx (with the help of https://forum.openwrt.org/viewtopic.php … 06#p316506 ).

I formatted the USB stick on my PC so no idea about your f2fs-tools problem (maybe you want to explain it some more?) but a quick format onto a file did work:

root@OpenWrt:~# mkfs.f2fs -l test ./testfs

        F2FS-tools: mkfs.f2fs Ver: 1.4.0 (2014-09-18)

Info: Label = test
Info: sector size = 512
Info: total sectors = 633995 (in 512bytes)
Info: zone aligned segment0 blkaddr: 512
Info: Discarding device
Info: format successful

Also no corruptions so far (but Chaos Calmer uses kernel 3.18.20 with backports while my PC uses kernel 4.3.0, not 3.13... You might want to update your Ubuntu).

v10lator wrote:

I formatted the USB stick on my PC so no idea about your f2fs-tools problem (maybe you want to explain it some more?) but a quick format onto a file did work:

 -----------------------------------------------------
 DESIGNATED DRIVER (Bleeding Edge, r48151)
 -----------------------------------------------------
root@bn:~# ll /dev/sd*
brw------- 1 root root 8, 0 Jan  1  1970 /dev/sda
brw------- 1 root root 8, 1 Jan  1  1970 /dev/sda1
brw------- 1 root root 8, 2 Jan  1  1970 /dev/sda2
brw------- 1 root root 8, 3 Jan  1  1970 /dev/sda3
brw------- 1 root root 8, 4 Jan  1  1970 /dev/sda4
root@bn:~# mkfs.f2fs /dev/sda4

        F2FS-tools: mkfs.f2fs Ver: 1.4.0 (2014-09-18)

Info: sector size = 512
Info: total sectors = 501377302265856 (in 512bytes)
Info: zone aligned segment0 blkaddr: 256
Info: Discarding device
Info: This device doesn't support TRIM
        Error: Failed to Initialise the SIT AREA!!!
        Error: Could not format the device!!!
root@bn:~# uname -a
Linux bn 4.1.13 #15 PREEMPT Sat Jan 16 22:48:42 MSK 2016 mips GNU/Linux
root@bn:/home/root# ll
total 107532
drwxr-x--- 2 root root      4096 Mar 28 14:42 .
drwxr-xr-x 9 root root      4096 Mar 28 14:42 ..
-rw-r--r-- 1 root root 110100480 Mar 28 14:43 f
root@bn:/home/root# mkfs.f2fs f

        F2FS-tools: mkfs.f2fs Ver: 1.4.0 (2014-09-18)

Info: sector size = 512
Info: total sectors = 215040 (in 512bytes)
Info: zone aligned segment0 blkaddr: 512
        Error: Failed to prepare a super block!!!
        Error: Could not format the device!!!

Its not the first time I discover probably compiler/libc caused problems.
I found broken or semi-broken xl2tpd-control on arm, xz, zip, unrar on mips
If I recompile them on target platform they work.
And I dont use prebuild images, I always compile myself.
Never had such problems until DD and movement towards musl. I believe CC works because its still on uclibc.
I can recompile f2fs tools on target but compiling kmod will require copying there whole kernel tree or just kernel headers which I'll have to prepare manually.

(Last edited by bolvan on 28 Mar 2016, 12:51)

Just tried latest openwrt and lede releases with f2fs-tools and mkfs.f2fs still fails under openwrt/lede... this is the message I get:

root@base-test-1:~# mkfs.f2fs /dev/sda1 -l hyhelp-f2fs

    F2FS-tools: mkfs.f2fs Ver: 1.8.0 (2017-02-03)

Info: Debug level = 0
Info: Label = 
Info: Trim is enabled
Info: [/dev/sda1] Disk Model: Ultra HS-SD/MMC1.76
Info: total device sectors = 30244864 (in 512 bytes)
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 0 (0 MB)
Info: zone aligned segment0 blkaddr: 512
    Error: Device size is not sufficient for F2FS volume
    Error: Failed to prepare a super block!!!
    Error: Could not format the device!!!
root@base-test-1:~# mkfs.f2fs
    Error: Device not specified

and on my Fedora based laptop I formated same card without issue:

sudo mkfs.f2fs /dev/sdd1 -l hyhelp-f2fs

    F2FS-tools: mkfs.f2fs Ver: 1.7.0 (2016-07-28)

Info: Debug level = 0
Info: Label = hyhelp-f2fs
Info: Trim is enabled
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 30244864 (14768 MB)
Info: zone aligned segment0 blkaddr: 512
Info: format version with
  "Linux version 4.10.17-100.fc24.x86_64 (mockbuild@bkernel02.phx2.fedoraproject.org) (gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC) ) #1 SMP Mon May 22 16:32:22 UTC 2017"
Info: Discarding device
Info: This device doesn't support BLKSECDISCARD
Info: This device doesn't support BLKDISCARD
Info: Overprovision ratio = 1.650%
Info: Overprovision segments = 247 (GC reserved = 129)
Info: format successful

Also putting back f2fs formated sdcard into openwrt/lede worked out ok:

# block mount
block: /dev/mtdblock4 is already mounted on /rom
block: /dev/mtdblock5 is already mounted on /overlay
Info: Force to fix corruption
Info: [/dev/sda1] Disk Model: Ultra HS-SD/MMC1.76
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 30244864 (14768 MB)
Info: MKFS version
  "Linux version 4.10.17-100.fc24.x86_64 (mockbuild@bkernel02.phx2.fedoraproject.org) (gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC) ) #1 SMP Mon May 22 16:32:22 UTC 2017"
Info: FSCK version
  from "Linux version 4.4.61 (root@) (gcc version 5.4.0 (LEDE GCC 5.4.0 r0-cec7c05) ) #0 Mon May 29 12:48:32 2017"
    to "Linux version 4.4.61 (root@) (gcc version 5.4.0 (LEDE GCC 5.4.0 r0-cec7c05) ) #0 Mon May 29 12:48:32 2017"
Info: superblock features = 0 : 
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 30244864 (14768 MB)
Info: CKPT version = 4
[ASSERT] (build_nat_area_bitmap:1855) fsck->entries
block: check_filesystem: /usr/sbin/fsck.f2fs returned 255

The discussion might have continued from here.