[OpenIndiana-discuss] Recommendations for fast storage

[Timothy Coalson]

On Tue, Apr 16, 2013 at 3:48 PM, Jay Heyl <jay at frelled.us> wrote:

> My question about the rationale behind the suggestion of mirrored SSD
> arrays was really meant to be more in relation to the question from the OP.
> I don't see how mirrored arrays of SSDs would be effective in his
> situation.
>

There is another detail here to keep in mind: ZFS checks checksums on every
read from storage, and with raid-zn used with block sizes that give it more
capacity than mirroring (that is, data blocks are large enough that they
get split across multiple data sectors and therefore devices, instead of
degenerate single data sector plus parity sector(s) - OP mentioned 32K
blocks, so they should get split), this means each random filesystem read
that isn't cached hits a large number of devices in a raid-zn vdev, but
only one device in a mirror vdev (unless ZFS splits these reads across
mirrors, but even then it is still fewer devices hit).  If you are limited
by IOPS of the devices, then this could make raid-zn slower.

Disclaimer: this is theory, I haven't tested this in practice, nor have I
done any math to see if it should matter to SSDs.  However, since it is a
configuration question rather than a hardware question, it may be possible
to acquire (some of) the hardware first and test both setups before
deciding.

Tim