[OpenIndiana-discuss] Recommendations for fast storage

[Sašo Kiselkov]

On 04/16/2013 11:25 PM, Timothy Coalson wrote:
> On Tue, Apr 16, 2013 at 3:48 PM, Jay Heyl <jay at frelled.us> wrote:
> 
>> My question about the rationale behind the suggestion of mirrored SSD
>> arrays was really meant to be more in relation to the question from the OP.
>> I don't see how mirrored arrays of SSDs would be effective in his
>> situation.
>>
> 
> There is another detail here to keep in mind: ZFS checks checksums on every
> read from storage, and with raid-zn used with block sizes that give it more
> capacity than mirroring (that is, data blocks are large enough that they
> get split across multiple data sectors and therefore devices, instead of
> degenerate single data sector plus parity sector(s) - OP mentioned 32K
> blocks, so they should get split), this means each random filesystem read
> that isn't cached hits a large number of devices in a raid-zn vdev, but
> only one device in a mirror vdev (unless ZFS splits these reads across
> mirrors, but even then it is still fewer devices hit).  If you are limited
> by IOPS of the devices, then this could make raid-zn slower.

If you are IOPS constrained, then yes, raid-zn will be slower, simply
because any read needs to hit all data drives in the stripe. This is
even worse on writes if the raidz has bad geometry (number of data
drives isn't a power of 2).

Cheers,
--
Saso