[OpenIndiana-discuss] XtreemFS Fuse Client on OpenSolaris / OpenIndiana

[Paolo Marcheschi]

Hi
I forward to the lists this announcement, I think may be of some interest.

Paolo

-------- Messaggio originale --------
Oggetto: 	XtreemFS Fuse Client on OpenSolaris / OpenIndiana
Data: 	Mon, 05 Dec 2011 16:32:30 +0100
Mittente: 	Michael Berlin <michael.berlin.xtreemfs at googlemail.com>
A: 	xtreemfs at googlegroups.com
CC: 	Paolo Marcheschi <paolo.marcheschi at gmail.com>



Hi all,

Thanks to the efforts of Paolo Marcheschi I succeeded to patch the
XtreemFS Fuse client to support OpenSolaris / OpenIndiana. The changes
are present in the SVN trunk revision 2830. At this point we won't
provide binaries for Solaris, so you have to compile it on your time.

While basic testing of the XtreemFS client, I found several limitations
in the Fuse support of OpenIndiana (tested with version oi_151a):

1. Max 4kb read() and write() buffer size

Similar to old Linux Fuse versions, it seems the maximum supported read
and write buffer size is 4k. Without higher buffer sizes, you'll see a
poor sequential read/write performance on Solaris and/or a high CPU usage.

2. No readahead support

The Linux Fuse provides implicit read-ahead. Therefore we haven't
supported it as part of the XtreemFS client itself. Without read-ahead,
sequential read()s won't be able to reach the maximum bandwidth over
links with high latency.

3. Not possible to invalidate the aggressive file cache

Solaris or Solaris' Fuse (I don't know whose fault it is) do cache the
content of read/written files. In consequence, the XtreemFS client does
not see subsequent read()s if XtreemFS is still mounted and the file was
already written or read. I don't know how long the file content is
cached or how to invalidate it. Any suggestions are welcome.

Due to this cache, it is not possible to see changes of files made by
other XtreemFS client as long as the data is cached. This does also
break the functionality of our tool "xtfsutil". If you try to execute it
on Solaris, you'll see the following error:

$ ./xtfsutil /mnt/
xtfsutil read back the same text which was written into the pseudo
control file: {"operation":"getattr","path":"/"}

This means a file content cache prevents xtfsutil from working correctly.
This is a known issue on Solaris.


In summary, the XtreemFS client on Solaris works so far. However, it's
evident that the Solaris Fuse does lag behind its Linux and MacOSX
implementations. As long as this state remains, you probably won't have
a lot of fun using XtreemFS on Solaris. I could not find news on the
Solaris Fuse developement but it does not seem to be active any more?
Are there newer Solaris Fuse version than the one in OpenIndiana?

At last: Here's how to compile the XtreemFS client on Solaris:

0. Get the boost libraries and compile them on your own.
@Paolo, can you please write the steps you took to accomplish this?

1. Check out the XtreemFS sources
$ svn checkout http://xtreemfs.googlecode.com/svn/trunk/ XtreemFS-trunk

2. Compile it. Specify the absolute path of the boost root directory in
the environment variable BOOST_ROOT.
$ cd XtreemFS-trunk
$ BOOST_ROOT=/root/boost_1_47_0 gmake client

3. The resulting binaries are in the directory "bin". Try to mount our
public demo server:
$ ./bin/mount.xtreemfs demo.xtreemfs.org/demo /mnt
Unmount it again:
$ fusermount -u /mnt

Best regards,
Michael