[OpenIndiana-discuss] Recommendations for fast storage

[Richard Elling]

On Apr 14, 2013, at 8:15 AM, Wim van den Berge <wim at vandenberge.us> wrote:
> Hello,
> 
> We have been running OpenIndiana (and its various predecessors) as storage
> servers in production for the last couple of years. Over that time the
> majority of our storage infrastructure has been moved to Open Indiana to the
> point where we currently serve (iSCSI, NFS and CIFS) about 1.2PB from 10+
> servers in three datacenters . All of these systems are pretty much the
> same, large pool of disks, SSD for root, ZIL and L2ARC, 64-128GB RAM,
> multiple 10Gb uplinks. All of these work like a charm. 
> 
> However the next system is  going to be a little different. It needs to be
> the absolute fastest iSCSI target we can create/afford. We'll need about
> 10-12TB of capacity and the working set will be 5-6TB and IO over time is
> 90% reads and 10% writes using 32K blocks but this is a data analysis
> scenario so all the writes are upfront. Contrary to previous installs, money
> is a secondary (but not unimportant) issue for this one. I'd like to stick
> with a SuperMicro platform and we've been thinking of trying the new Intel
> S3700 800GB SSD's which seem to run about $2K. Ideally I'd like to keep
> system cost below $60K.

Does "fast" mean "low-latency"? If so, the general rules are:
	+ mirror
	+ go direct, no expanders
	+ iSCSI tends to not use ZIL very much, but  you can verify on your workload.

There are a number of vendors who have been selling SSD-only ZFS systems
for a few years. You might ask around for experiences and specs.
 -- richard

> This is new ground for us. Before this one, the game has always been
> primarily about capacity/data integrity and anything we designed based on
> ZFS/Open Solaris has always more than delivered in the performance arena.
> This time we're looking to fill up the dedicated 10Gbe connections to each
> of the four to eight processing nodes as much as possible. The processing
> nodes have been designed that they will consume whatever storage bandwidth
> they can get.
> 
> 
> 
> Any ideas/thoughts/recommendations/caveats would be much appreciated.
> 
> 
> 
> Thanks
> 
> 
> 
> W
> 
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

--

Richard.Elling at RichardElling.com
+1-760-896-4422