[OpenIndiana-discuss] fault-tolerant distributed file systems

[Geoff Nordli]

On Mon, Oct 31, 2011 at 3:05 PM, Jeppe Toustrup <openindiana at tenzer.dk> wrote:
> On Mon, Oct 31, 2011 at 22:04, Geoff Nordli <geoffn at gnaa.net> wrote:
>> I am looking for a fault-tolerant distributed file system to store
>> medium sized files (150MB-4GB) which can scale across 100s of servers
>> and keep N replicas of each file.
>
> You say you need a file system, does that have to be a POSIX
> compatible file system (ie. mountable) or do you just need to store
> some files?
> If you just need to store some files, you could have a look at Hadoop
> or more specifically HDFS (Hadoop Distributed File System), which
> consists of java daemons which can synchronize files around for you
> and manage all the redundancy. Libraries exists for all kinds of
> languages to hook into the system, so there should be something which
> could fit into your needs.
>

It needs to be a mountable file system.

thanks,

Geoff