Hi,
since OpenWRT's squashfs mode is tailored towards running from slow and short-living mtd flash, I was wondering if using the ext4 installation on x86 make any real difference regarding flash wear out. Or to put it in other words: Is there any advantage in running OpenWRT (as router/firewall) from an msata ssd over using a sdcard?
Many thanks in advance.
If you decide to experiment, there is a "flash-friendly filesystem" called "f2fs". I have been using it for my exroot.
The standard image for the standard user doesnt really write anything beside config changes. Certain plugins will affect that, for example saving logs or using a local adblock.
You can disable journaling on ext4.
In general f2fs help a lot in matter of wear and write performance if the flash storage doesnt have respectively a controller and/or own cache like your sdcard.
So go for f2fs with the image builder, only because of performance advantages.
At the end, it shouldn't matter until you write a lot of data on that sdcard.
The squashfs version of 19.07.2 by default uses a f2fs overlay. This overlay is formed at boot time by an unusual process that splits the /dev/sda2 partition into the original squashfs (alias /dev/root) and a f2fs which starts at the end of the squashfs (alias /dev/loop0).
This makes it not trivial (though certainly possible) to expand the f2fs and have a larger overlay.
If you are storing a lot of data it would make sense to create and mount an additional data partition, then that data can be kept separate while upgrading the OS.
Thank you for these suggestions! And yes I know about f2fs and the ImageBuilder, but I was looking for something that is also easily upgradable afterwards without the need to rebuild anything.
I did some experiments with the combined-squashfs image before going for the ext4-rootfs but it seemed like it didn't have almost any space for installing even basic stuff (such as drivers for my wifi and usb-lan adapters).
You can see the issue here:
root@OpenWrt:/# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 3.0M 3.0M 0 100% /rom
tmpfs 56.2M 872.0K 55.3M 2% /tmp
/dev/loop0 77.0K 7.0K 65.0K 10% /overlay
tmpfs 56.2M 124.0K 56.0M 0% /tmp/root
overlayfs:/tmp/root 56.2M 124.0K 56.0M 0% /
/dev/vda1 15.7M 3.8M 11.6M 25% /boot
/dev/vda1 15.7M 3.8M 11.6M 25% /boot
tmpfs 512.0K 0 512.0K 0% /dev
root@OpenWrt:/# opkg install wireguard
Installing wireguard (0.0.20190702-1) to root...
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/x86_64/base/wireguard_0.0.20190702-1_x86_64.ipk
Installing ip-tiny (5.0.0-2.1) to root...
Collected errors:
* verify_pkg_installable: Only have 65kb available on filesystem /overlay, pkg ip-tiny needs 138
* opkg_install_cmd: Cannot install package wireguard.
Is this behavior intended? I didn't observe this on other targets (e.g. ath79).
Strange that your loop0 is so small, it is about 240 MB on mine.
@mk24 I found the fault - I was testing this image beforehand in a VM and I used the image directly. The following approach fixes this:
wget https://downloads.openwrt.org/releases/19.07.2/targets/x86/64/openwrt-19.07.2-x86-64-combined-squashfs.img.gz
gunzip openwrt-19.07.2-x86-64-combined-squashfs.img.gz
dd if=/dev/zero of=disk.img bs=1M count=300
dd if=openwrt-19.07.2-x86-64-combined-squashfs.img of=disk.img conv=notrunc
So it looks to me like running the squashfs image works just fine. And this is basically much better than ext4 from the wear-leveling perspective. I think I am going for this solution then. 
Note (when retrying this months later): If you use KVM and raw DD images, KVM remembers the size. So if you do it wrong the first time, you have to remove and add the dd image again, so that KVM realizes the right disk size and communicates this to the running OpenWRT.
You'd best use the proper tools like these:
virsh blockresize
qemu-img resize
I've used an SD card for my x86 box for 2 years, no issue. It was a 64 GB Sandisk card, and I opted not to do a resize of the image after the fact. I know that for SSDs, the flash would automatically ignore any bad blocks and write to good ones, so I never bothered to resize as the drive would've already done this. Additionally, never needed the space. I used the EXT4 combined image.
SD cards are so cheap that it doesn't matter. Throw them out every 2 years.
Personally, I consider wasting resources (SD cards) and lifetime to work around issues that don't occur when using a suitable filesystem not a favorable approach.
it all depends on what packages you install. By default the OpenWrt system does not write anything to storage so SD card or USB flash drive is fine.
If you install packages that need to write things constantly in the storage (update a list/database of information, save logs to disk, download files from the internet) then a SD card will wear out.
Wear leveling works on SD cards, on USB drives and on SSDs, f2fs filesystem or ext4 filesystem don't change write endurance.
Even if you use f2fs the SD card will wear out and fail after you have written around the same amount of data to it.
The main difference is performance, f2fs should be faster.
But this matters only if you write things to the storage and on reboot of the device.