[OpenIndiana-discuss] Recommendations for fast storage

[Wim van den Berge]

Hello,

 

We have been running OpenIndiana (and its various predecessors) as storage
servers in production for the last couple of years. Over that time the
majority of our storage infrastructure has been moved to Open Indiana to the
point where we currently serve (iSCSI, NFS and CIFS) about 1.2PB from 10+
servers in three datacenters . All of these systems are pretty much the
same, large pool of disks, SSD for root, ZIL and L2ARC, 64-128GB RAM,
multiple 10Gb uplinks. All of these work like a charm. 

 

However the next system is  going to be a little different. It needs to be
the absolute fastest iSCSI target we can create/afford. We'll need about
10-12TB of capacity and the working set will be 5-6TB and IO over time is
90% reads and 10% writes using 32K blocks but this is a data analysis
scenario so all the writes are upfront. Contrary to previous installs, money
is a secondary (but not unimportant) issue for this one. I'd like to stick
with a SuperMicro platform and we've been thinking of trying the new Intel
S3700 800GB SSD's which seem to run about $2K. Ideally I'd like to keep
system cost below $60K.

 

This is new ground for us. Before this one, the game has always been
primarily about capacity/data integrity and anything we designed based on
ZFS/Open Solaris has always more than delivered in the performance arena.
This time we're looking to fill up the dedicated 10Gbe connections to each
of the four to eight processing nodes as much as possible. The processing
nodes have been designed that they will consume whatever storage bandwidth
they can get.

 

Any ideas/thoughts/recommendations/caveats would be much appreciated.

 

Thanks

 

W