Interesting read. Thanks. Yeah 512MB is RAM is low, I can't even encrypt the drives.
I think you can do all of this with BTRFS. It certainly 100% has checksumming, snapshots, and remote replication (through btrfs-send).
Encryption I think isnāt done at a filesystem level, but if you want to btrfs-send to a bitstream and encrypt the bitstream with a key and push that to a remote location so you can pull it back later and decrypt it and rebuild it (like for remote backups) that is doable with btrfs-send | gpg
1 Like
I remember we used encrypted drives like 15 years ago and it wasnāt that memory hungry. I bet AES code is almost identical except now itās hardware accelerated and 1000 times faster.
Probably I will challege myself setting up encrypted drive on Alix board with 256MB of RAM, IDE drive connection and HW AES128 accelerator.
There is no encryption in Btrfs. It is done on lower level. If youāre not using snapshots then forget about Btrfs. Use something simpler, XFS is a good balance between simple fixed inode ext4 and Btrfs. Stay lean, donāt install things unless you totally need those.
2 Likes
Also technically you can just encrypt the offsite disk aswell. Does ZFS use LVM or it has it's own encryption system though?
ZFS handles everything (device and volume management, encryption etc) itself, which has another big advantage over other solutions in that it supports TRIM natively. With LUKS and other drivers below the filesystem level this can otherwise prove difficult, and without TRIM one would need to overprovision SSDs a lot in order to avoid performance degradations.
But you are right, btrfs is indeed more powerful than I initially expected, and a lot can be made to work, itās just that ZFS is a mature and well established one-stop solution for all my requirements (which include encryption in-transit and at-rest, also for all remote copies, as a mandatory requirement).
We are talking about low-end CPU paired with 512MB of RAMā¦
Actually this thread is about āNAS devices that you can install Openwrt toā as the OP put it. A lot of those have Intel N100 x64 CPUs nowadays, and 16GB of RAM are possible, if not common. But OpenWrt falls short of some features one would want in a NAS, so I guess the conclusion is clear.
Qnap TS-433 (and 133/233) is RK3568 based with 4G/4G ram/storage.
0001:10:00.0 PCI bridge [0604]: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor [1d87:3566] (rev 01)
0001:11:00.0 SATA controller [0106]: JMicron Technology Corp. JMB58x AHCI SATA controller [197b:0585]
0002:20:00.0 PCI bridge [0604]: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor [1d87:3566] (rev 01)
0002:21:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
This is just an assumption, N100 is popular but you can't put an equal sign between NAS & this CPU
late replay but may you could offload crypto?
just let der NAS do the file hosting and use
-mdam
-exit4
-borgbackup (https://en.wikipedia.org/wiki/Borg_(backup_software) over ssh)
with clients like VORTA (crypt + compression)
or have you tried:
-mdadm raid 1
-ext4
-e4crypt
and even i can be stable it wonĀ“t be fast.
Thank you! I personally settled on ZFS and wonāt use systems running OpenWrt for storage purposes.
If you need ZFS on ARM-based NAS there is always Armbian, Iāve used that and it was ok on 2GB SBC with 12TB of storage, from the standpoint that I could use it as a sync destination without errors. 2GB RAM on a SBC is quite small, many are 8GB or 16GB now. However 2GB RAM on a typical OpenWRT target is gigantic and rarely seen outside of repurposed Enterprise devices which are more like PCs in custom chassis than home routers.
I have a OpenWRT NAS, a Zyxel NSA 325with basic ext4 and syncthing to take backups. It is slow but keeps running, I try not to ask much of it but it fills a specific niche. For real-time access the drive speeds are terrible but as a mostly-idle backup it is passable if you are very patient.
1 Like
iĀ“ m seeing 30MB/s via smb and thats mostly enough for home useā¦.nfs should be around the same speedā¦. with MD raid1 with ext4 on top. ftp should also be in this speed category. while sftp is unusable speedwise (<1MB/s).
maybe i try btrfs on top of mdraid, cause of the really useful featureset, but it might negatively influence speed.
i would love to have a reduced layer of complexity and use btrfs raid 1 directly but i found out the openwrt btrfs raid support is broken for years.
Same here, MD RAID1 is fine. I donāt bother with btrfs myself, itās been around for many years but the EC code still appears untrusted and I suspect we will see mature EC with tiering from BcacheFS before btrfs gets EC support reliable (drama aside, I have no interest in the BcacheFS LKML arguments). I know mdadm and itās trivial to support for me, easy to generate alert emails and to view resync progress. Neither of which I have got an easy setup for with btrfs.
for me, btrfs user for 10+ years, it was always very reliabe (beside the raid5/6 writehole problems, but this was 10 years ago and evolved much since then). i use it on every x86 installation as root fs with all the features. mainly on debian (spiral linux spin) with grub snapshots. and i love it. āsavedā (eased) my live several times. i also would always trust in a btrfs raid1 setup (used a root on my bare metal server setups).
with al that experience and trust i was very disappointed when finding btrfs is such a mess @ openwrt (single drives are ok afaik) 
btw i also see the giant potential of bcachefs. while personalities are/was/will be very discussable, there are rules that where established from and for a big community. and you need to trade with it. hans reiser was also very special, back then. but i believe if time goes by bcachefs will be back in business/kernel some day (soon). its to good and to important to have.
1 Like
How do you send notifications on a RAID1 drive failure? With mdadm we can use /etc/mdadm.conf and ZFS has the ZED utilities for this. Btrfs appears to have https://github.com/ximion/btrfsd but itās a 3rd party utility instead of a default component, and many alternative utilities exist as well.
For OpenWRT this is important as OpenWRT NAS installs are likely to be running as low-power backup solutions. There doesnāt seem to be a clear recommendation from the community to monitor BtrFS when it is used as a remote fileserver.
eg.
homeassisant via openwrt addon or snmp for ANY fs. i dont do btrfs speciffic stuff. you could also do some scripting and send email alerts.
this wont go together with the openwrt conf @ /etc/config/mdadm.
it will surely work with the āoptionsā uci setting. but i didnt find a doku which options will work and which dont.
i would be happy to update (create) @ the wiki when i get some info.
we could create a new thread where we could collect info?
My home all-in-one server is Banana Pi BPI-R3. It is main router connected directly to ISP, and it is main NAS. I have installed M.2->SATA convertor board with 6 SATA connectors, and Banana is used for:
- main Wi-Fi access point (I also have extender) with all bells and whistles
- Transmission
- Samba (to use what Transmission has downloaded)
- Syncthing
- recorder for CCTV (based on ffmpeg)
- Chrony (quiet reliable Stratum-2 NTP server)
- Wireguard (interconnection with my country house)
- i2pd, tor, xray (we live in difficult times)
I have no need for RAID or ZFS because everything what I really don't want to lose is syncthing'ed to another reliable storage. But I use btrfs to make pool of three old external USB-SATA drives (250+400+500 MB) to use them as one big temporary trash storage. Just for fun, actually.
The only bad thing that happened with this configuration is significantly lowered SnR on 2.4GHz band because of SATA and USB3.0. But is is used only for couple of IoT devices, and it's enough.