1 (edited by aleksey_t 2011-02-12 21:43:48)

Topic: remotefs

[s]Update: Forgot to mention, binary packages provided for mipsel are for 2.4 kernel. Not sure if they will work with 2.6 kernel.[/s]

Homepage: http://remotefs.sourceforge.net/


I'd like to present you remotefs i've implemented for use with my NAS.

The reasons i've decided to make it are a) samba isn't working for me at all b) NFS is not working well for me c) sshfs is great, but not very fast.

In comparison with sshfs, reading performance on my hardware has improved roughly three times and writing performance has almost doubled. So if you have trouble with samba and NFS (as I do), then you may be interested in remotefs.

I’ve quickly put together this solution using FUSE. I'm using Asus WL-500gP with OpenWrt, so i've built server package for Kamikaze (mipsel). Client package is available for Debian-based i386 systems. Build scripts are available in SVN, so you could make packages for your platform(s) if you wish so.

The project is hosted at SourceForge: https://sourceforge.net/projects/remotefs , where you can get all sources and build scripts in SVN. Binary packages are available in downloads section: https://sourceforge.net/project/showfil … _id=229928.

First, a warning: it is still in beta and should never be exposed to untrusted networks.
Second, before you start using it (even for testing), take a look at the release notes for details on known issues. (https://sourceforge.net/project/shownot … _id=229928)


Starting the server:
/etc/init.d/rfsd start (do not forget to write down exports in /etc/rfs-exports and set passwords with rfspasswd utility (if you need them)).

Exports file:
The file is NFS-like, except there are no options for export entry and no subnet mask for ip-addresses.

Client setup:

Mounting export:
rfs begemoth/

It is NFS/sshfs like. If you are not familiar with it, here is its format: “rfs host:export_name mount_point". Host is ip-address or host name, export name is the string from exports file, mount_point  has the same meaning as in the any other mount operation.

You may also specify username/password or FUSE option(s). See rfs -h for details.

If you interested in the project or feel there's a feature missing, or have suggestions or any remarks, I'd be glad to hear from you by e-mail at aleksey_t@users.sourceforge.net.

2 (edited by aleksey_t 2008-06-10 22:45:09)

Re: remotefs

I've done some work to improve read/write performance and managed to get more than 2.5Mb/s both ways on my hardware (WL-500gP). This performance improvement is good for large files only. An application which writes small amounts of data to many files will not experience a performance increase in comparison with the previous version of remotefs.

Cache which is used for this improvement will consume up to 1,5Mb RAM (on client's side and up to 512Kb on server's) from time to time, but I guess it is worth it. This feature is optional as it is not enabled by default (yet). Use rfs options -ord_cache=1, -owr_cache=1, or -ordwr_cache=1 to enable reading cache, writing cache, or both.

Example: rfs begemoth/ -ordwr_cache=1

You can also specify which port and which address the server will listen to. Use -a [address] and -p [port_number] options. Defaults are specified in /etc/init.d/rfsd script.

Salt for password encryption has been changed to an empty string. All previously set passwords will need to be reset. Sorry for the inconvenience.

See changelog for details on changes made and release notes (http://sourceforge.net/project/shownote … _id=605847) for the list of remaining issues.

Still in beta, do NOT use outside of trusted networks.

Download link for 0.8 release: http://sourceforge.net/project/showfile … _id=605847

Re: remotefs

I've rewritten my write cache implementation and now it may potentially consume 758Kb less memory on client's side and 256Kb less on server's while transferring large files. You may experience a little (a *very* little) write performance degradation. Most likely performance will be the same as in version 0.8.

Since version 0.9, both read and write caches are enabled by default. You may use options -ord_cache=0, -owr_cache=0, and -ordwr_cache=0 to disable read cache, write cache or both.

New feature: read-only exports. Read-only exports can be defined like this: /tmp, alex, (ro)

Server is now able to lower privileges to specified user using -u [username] option. Make note that rfsd will get filesystem permissions of specified user and it still needs to be initially run by root to properly do chroot(). After mounting of export (chroot()), process's user id will be changed to the one specified with -u option. Default user (root) is specified in the /etc/init.d/rfsd script.

I'm currently looking for people who could review code and point out security issues. Any help would be appreciated.

Next release will be most likely the final one – I've implemented everything I expected from remotefs, but there won’t be a 1.0 release until the code has been reviewed for security issues as mentioned above. Since this review will probably not happen in the nearest future, if you interested in remotefs, I suggest you to use this release as final, but without exposing rfsd to untrusted networks.

Please take a look at changelog for more details on chages made: http://sourceforge.net/project/shownote … _id=229928

Download link for 0.9 release: http://sourceforge.net/project/showfile … _id=608613

Re: remotefs


With new developer who joined the project, we were working on performance optimization and gained a significant improvement. I've managed to get 3,5Mb/s on my hardware while reading a (large) file from NAS and 4.0Mb/s while writing a file to NAS. Version 0.10 has also (really) fast directory listing.

This release also has some nice features you may find useful:
- auto-reconnect
- reload support if you changed server settings (exports and/or passwords) and don't want to restart all remotefs connections
- IPv6 support (in SVN only)
- access rights lowering for each export with options "user="  and "group="  in exports file.

New major feature is support of UGO-model: if you'll specify "ugo"  option for export, rfsd will act a lot like NFS server – it will log the user into the remote system setting its pid, gid and setting up groups for username. You need to create users with the names same as you use for logging in to rfsd. Server's and client's users (/etc/passwd) and groups (/etc/group) need to be synchronized. remotefs could work without synced systems, but in this case you would potentially get some side effects, like mistakenly reported file owner and group. Please refer to man pages for a more detailed description of these side effects.

We have also updated documentation with man pages, you could find it here: rfsd: http://remotefs.sourceforge.net/rfspasswd.8.html , rfspasswd: http://remotefs.sourceforge.net/rfsd.8.html , rfs: http://remotefs.sourceforge.net/rfs.1.html .

Download link: https://sourceforge.net/project/showfil … _id=634369
Changelog: https://sourceforge.net/project/shownot … _id=229928

Still in beta, do not expose to untrusted network(s). Please report any bugs send your feedback to aleksey_t@users.sourceforge.net.

Re: remotefs


We have released a new version of remotefs with POSIX ACL and SSL support. Both are not included in packages, but available in SVN. Memory usage on server is also reduced (512Kb less for each connection).

Changelog and release notes: https://sourceforge.net/project/shownot … _id=661183
Download link: https://sourceforge.net/project/showfil … _id=661183
SVN tag: https://remotefs.svn.sourceforge.net/sv … /tags/0.11

Re: remotefs

looks interesting...
can it be compiled to mips as well ? (atheros /fonera 2)
Couldn't find the compiled package here yet :http://ipkg.nslu2-linux.org/feeds/openwrt/fon-ng/mips/
I'm curious if it's as fast as you mentioned in this thread

Re: remotefs

Take a look at these packages: http://sourceforge.net/project/showfile … _id=661183 . Make note that MIPS package is marked as experimental because we don't have actual MIPS hardware so we haven't checked if it *really* works (however, i don't have complaints about experimental packages yet).

I also recommend to disable prefetch while using 0.11 version (`rfs -o rd_cache=0`).

Re: remotefs


New release of remotefs is available. This release main feature is NSS module for automatic id-mapping (if UGO is used).

Changelog: http://remotefs.svn.sourceforge.net/vie … vision=986
Download link: http://sourceforge.net/projects/remotef … tefs/0.12/

9 (edited by XChesser 2011-01-24 12:05:16)

Re: remotefs


I faced the problem trying to use remotefs: I've installed server to OpenWrt 10.03.1 RC4 from the package from OpenWrt repo for my router (MIPS/ar71xx). It is build 854. To my laptop I've installed 0.14-1 deb-package from SF.
Export on router-side:


where is client IP.
Server is launched with -a option.
On client I mount the share with

rfs rfs

and it seems to be OK - I don't see any errors. But when I try to get into mountpoint I get "Transport endpoint is not connected".
I also tried to install client to the router (it is also 854 build) and it works well.
0.14-1 server MIPS package from SF can't be installed to my router:

root@OpenWrt:~# opkg install /tmp/rfsd.ipk 
Unknown package 'rfsd'.
Collected errors:
 * pkg_hash_fetch_best_installation_candidate: Packages for rfsd found, but incompatible with the architectures configured
 * opkg_install_cmd: Cannot install package rfsd.

Is my problem related to server/client versions incompatibility or something wrong with my configs?

10 (edited by aleksey_t 2011-01-24 15:23:42)

Re: remotefs

It's client-server incompatibility indeed, your configuration looks ok. Concerning architecture issues - packages on SF are for kamikaze and not for backfire, unfortunately. I'm working on it.

If 845 is a SVN revision, then your server package corresponds to remotefs 0.12 (most likely) or 0.11.

Or if you wish to try a quick hack (didn't tried, but should work), then i can suggest to edit file "control.tar.gz/control" inside of rfsd_0.14-1_???.ipk (it's just archive file) and replace mips with your architecture (`opkg print-architecture`, i have brcm-2.4 on my 10.03.1-rc4) or all. The only thing changed for remotefs packaging is architecture name, really.

Re: remotefs


Thanks a lot! I changed architecture to ar71xx, after that ipk (0.14-1) was successfully installed. Now I can list, read and rewrite files in my share. But I can't write new files: "Input/output error". What can it be?

P.S. My test copies with dd gave 12.0MiB/s read and 10.4MiB/s rewrite on TP-Link TL-WR1043ND (dd's MB=1 000 000 Bytes were converted to true megabytes).

Re: remotefs

That's odd. "Input/output error" usually indicates wrong message received by client/server (sort of incompatibility again).

If you're using the same version of client and server (0.14 i assume), then i'd say it's a bug in remotefs. Could you give me your `uname -a` from both client and server?

Re: remotefs


root@OpenWrt:/mnt# rfsd -v
root@OpenWrt:/mnt# uname -a
Linux OpenWrt #1 Fri Nov 19 20:27:50 PST 2010 mips GNU/Linux
root@OpenWrt:/mnt# cat /proc/version
Linux version (openwrt@ampere) (gcc version 4.3.3 (GCC) ) #1 Fri Nov 19 20:27:50 PST 2010

Client 1 (list, read and rewrite works; new files write doesn't work - I/O error) [Ubuntu 10.04 x64]:

keyone@keyone-laptop:~$ rfs -v
keyone@keyone-laptop:~$ uname -a
Linux keyone-laptop 2.6.35-22-generic #34~lucid1-Ubuntu SMP Mon Oct 11 14:34:26 UTC 2010 x86_64 GNU/Linux
keyone@keyone-laptop:~$ cat /proc/version
Linux version 2.6.35-22-generic (buildd@allspice) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #34~lucid1-Ubuntu SMP Mon Oct 11 14:34:26 UTC 2010
keyone@keyone-laptop:~$ fusermount -V
fusermount version: 2.8.1

Client 2 (I/O error with any operation including ls) [Ubuntu 10.10 x86]:

xchesser@Notebook-XChesser:~$ rfs -v
xchesser@Notebook-XChesser:~$ uname -a
Linux Notebook-XChesser 2.6.35-25-generic #43-Ubuntu SMP Thu Jan 6 22:25:16 UTC 2011 i686 GNU/Linux
xchesser@Notebook-XChesser:~$ cat /proc/version
Linux version 2.6.35-25-generic (buildd@roseapple) (gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5) ) #43-Ubuntu SMP Thu Jan 6 22:25:16 UTC 2011
xchesser@Notebook-XChesser:~$ fusermount -V
fusermount version: 2.8.4

I also have another computer with Sabayon Linux 5.3 x64 (2.6.37) but I couldn't install rfs to it. When I make the digest with ebuild from SF I get:

thebestkey@sabayon ~ $ sudo ebuild /usr/local/portage/sys-fs/rfs/rfs-0.14-r1.ebuild digest
Error(s) in metadata for 'sys-fs/rfs-0.14-r1':
  IUSE contains defaults, but EAPI doesn't allow them

Makefile in sources is not ordinary. It offers to build package/ebuild. And 'make ebuild' doesn't work for me either: "No rule to make target".

14 (edited by aleksey_t 2011-01-25 18:31:45)

Re: remotefs

This sounds like the problem is in server, i'll check MIPS build.

XChesser wrote:

Makefile in sources is not ordinary. It offers to build package/ebuild. And 'make ebuild' doesn't work for me either: "No rule to make target".

make ebuilds

Sorry for the confusion, i'll fix that.

And if you're doing `make ebilds` in trunk, then it will link to 0.15 tarball, which is not on SF yet. You need to either create ebuilds in 0.14 tag directory or to create 0.15 tarball with `make tbz`. (0.14 and 0.15 should be fully compatible, i believe)

Re: remotefs

I'm very sorry, I have forgotten to add client2 IP to the export. sad Now its behavior is the same as with client1: can list, read, rewrite, delete, but not create new files.

aleksey_t wrote:

make ebuilds

Really! I was too lazy to look through Makefile. But new ebuild is the same as previous.

And if you're doing `make ebilds` in trunk, then it will link to 0.15 tarball, which is not on SF yet. You need to either create ebuilds in 0.14 tag directory or to create 0.15 tarball with `make tbz`. (0.14 and 0.15 should be fully compatible, i believe)

It is 0.14 release, not SVN build. New ebuild is identical.

16 (edited by aleksey_t 2011-02-12 22:54:42)

Re: remotefs

I've fixed the bug, but it's not suitable for just bugfix release unfortunately. We'll be doing new release soon (but not very soon) and this bugfix will be there.

If you wish to try trunk version, then i've built packages for some of your machines. Every package is from SVN 1251. Just note to twimc: these packages are not from 0.15 release, just intermediate build of SVN trunk.

[links removed]

[links removed]

First server package is build like on SF, last one is build like in OpenWrt. Either should work, but here is both packages just in case.

XChesser wrote:

Really! I was too lazy to look through Makefile. But new ebuild is the same as previous.

I'm not sure if i understand you correctly, i thought you're want to modify ebuilds. Anyway, if ebuilds doesn't work, they will be fixed before release.

(I'm not familiar with Sabayon Linux. Ebuilds are tested against Gentoo, so if they will work on recent Gentoo, then i hope they will work on Sabayon too).

Re: remotefs

aleksey_t, thank you, now it works!

Sabayon Linux is Gentoo-based distro that has the same portage system as Gentoo. And all the commands for ebuilds management are the same: emerge, ebuild, layman, etc. Moreover it has no its own portage tree at all and uses Gentoo's one. So I think Sabayon is just the same as Gentoo in the sense of ebuilds installation.
To install ebuild I must generate its digest first, right? But operation fails with "IUSE contains defaults, but EAPI doesn't allow them".

Re: remotefs

I'm glad it's fixed. Thank you for the feedback.

Concerning ebuilds, well, they worked for the time of 0.14 release. Google says something has changed in portage. Ebuilds will be adjusted before 0.15 release to cover this issue (and/or remotefs changes in makefiles).

Re: remotefs

remotefs 0.15 is out: http://sourceforge.net/projects/remotef … tefs/0.15/

Mostly bugfix release.

Re: remotefs

I would write a howto in the wiki:

Concentrate on the configuration (and administration) part, the rest will (maybe) be done by somebody else.
We do have   http://wiki.openwrt.org/meta/templates

Re: remotefs

Orca wrote:

I would write a howto in the wiki:

We have wiki here: http://sourceforge.net/apps/mediawiki/r … =Main_Page

Re: remotefs

The idea is, that this package is found more easily, but well, suit yourself

Re: remotefs

I appreciate your suggestion, but i can't afford myself to support several wikis. And i don't want to just place link to our wiki in OpenWRT's wiki.

Re: remotefs


I'm glad to announce that after long pause in development remotefs 1.0 is finally out.

Some packages are available at SourceForge: http://sourceforge.net/projects/remotef … otefs/1.0/

You also can find all packages in OpenWRT's truck snapshots: http://downloads.openwrt.org/snapshots/trunk/


0.16 1.0rc1 1.0rc2 1.0

[+] client-server compatibilty handshake, 0.16 is incompatible
    with previous versions

[*] fixed issue with truncate for large files
[*] fixed newline handling at the end of exports
[*] fixed file locking (only full file locks are supported)


[+] hostnames resolution in exports


[+] connection and i/o timeout to avoid remotefs hangs

[*] enabled big writes for FUSE>=2.8
[*] simpler (faster) write cache optimized for sequental writes