[OpenIndiana-discuss] SSDs for ZFS Pool?

[Sebastian Gabler]

Hi,

I was asked to set up a pool for storing oracle datafiles. The databases 
are for test purpose, so that there is no ultra-high resiliency 
required. There is however the requirement to have really good read 
I/Os, and there is a comparably low budget available.  I thought of 
using NFS4, or ASM-on-NFS4 for the databases. I could probably have 
1,2-2,4 TB of usable storage, which would allow to run 6-12 databases 
with 1-2 instances connected to them (RAC, optional).

One idea is to use a couple of MLC SSDs w/ SATA interface, in raidz.

The existing environment runs OI 151, has 24 GB RAM, no ZIL, a couple of 
LSI 1068 based HBAs, and a 16x 3,5" external drive bay with extender 
connected to it, that has 8 slots free.
The server already serves two raidz/raidz2 pools w/ app 10 TB each, 
SATA-based, serving NFS4 as boot drives for a couple of VMware based 
VMs, iSCSI to a Win2008R2 target, and CIFS file services. Performance is 
"good enough". File services are provided through a 2x1GBE LACP.

Questions:

- I understand that TRIM will not work in any stripe set, be it parity 
based or not, correct?
- will heavy writes (such as transferring data from dumps, which will 
happen relatively often) lead to contention?
- Should I upgrade the HBAs to something newer like LSI2008, for 
compatibility reasons (no, I probably don't need the extra bandwidth SAS 
6g offers!)
- Are there any SSDs that you could recommend?
- with some desktop HDs, I experienced that they will be kicked out of 
the array for no obvious reason - does that happen with consumer SSDs as 
well, i.e. while the controller re-balances free cells for writing?
- should I better use 10 K SAS drives w/ flash-based ZIL?

Regards,

Tonmaus