[OpenIndiana-discuss] Raid type selection for large # of ssds

[Jan Owoc]

Hi Grant,

On Fri, Oct 5, 2012 at 10:39 AM, Grant Albitz <GAlbitz at albitz.biz> wrote:
> I feel bad asking this question because I generally know what raid type to pick.
>
> I am about to configure 24 256gig ssd drives in a ZFS/Comstar deployment. This will serve as the datastore for a vmware deployment. Does anyone know what raid level would be best. I know the workload will determine alot, but obviously there is varying workload across a vmware environment. Since we are talking about ssds I dont see a particular reason to not create 1 big zfs pool, with the exception that I know people generally try to keep the drive count from getting out of control. Raid 10 seems like a waste of space with little benefit in performance in this case. i am leaning towards raid z2 but wanted to get everyones input.
>
> The datastore will host a fileserver, and exchange server for about 50 users. The environment is all 10g and they have solid states in all desktops so essentially that is the reason for such a large SSD deployment for a small # of users.
>
> There seems to be varying opinions, especially when you factor in trying to keep writes low for ssds.

There is no single right answer given the information you've provided.
Richard Elling made a spreadsheet to help you find a compromise
between bandwidth, IOPS and mean-time-to-data-loss:
https://blogs.oracle.com/relling/entry/sample_raidoptimizer_output

In addition there are some general rules of thumb for RAIDZ/2/3, like
that it may be good to have 2^N+P drives in each stripe. Your
alternatives are:
2x9-drive RAIDZ
2x10-drive RAIDZ2
2x11-drive RAIDZ3
4x5-drive RAIDZ
4x6-drive RAIDZ2 = 24
3x7-drive RAIDZ3

So, just going off the rule of thumb (and assuming you want some form
of RAIDZ), you'd make a pool that has 4 sets of 6-drive RAIDZ2s. The
pool will have the capacity of 16 drives, and the IOPS/bandwidth of a
4-way stripe. Assuming you have the CPU power to keep up with RAIDZ
parity calculations, I don't think you can come up with a RAID10
configuration that would offer comparable iops/bandwidth to the
4-way-RAIDZ2.

If you have some time before the machine needs to be "in-production",
you can try benchmarking a few of the configurations that you think
might offer an acceptable MTTDL.

Jan