[OpenIndiana-discuss] Recommendations for fast storage

[Edward Ned Harvey (openindiana)]

> From: Jay Heyl [mailto:jay at frelled.us]
> 
> > So I'm just assuming you're going to build a pool out of SSD's, mirrored,
> > perhaps even 3-way mirrors.  No cache/log devices.  All the ram you can fit
> > into the system.
> 
> What would be the logic behind mirrored SSD arrays? With spinning platters
> the mirrors improve performance by allowing the fastest of the mirrors to
> respond to a particular command to be the one that defines throughput.

When you read from a mirror, ZFS doesn't read the same data from both sides of the mirror simultaneously and let them race, wasting bus & memory bandwidth to attempt gaining smaller latency.  If you have a single thread doing serial reads, I also have no cause to believe that zfs reads stripes from multiple sides of the mirror to accelerate - rather, it relies on the striping across multiple mirrors or vdev's.

But if you have multiple threads requesting independent random read operations that are on the same mirror, I have measured the results that you get very nearly n-times a single disk random read performance by using a n-way mirror and at least n or 2n independent random read threads.


> There is no
> latency due to head movement or waiting for the proper spot on the disc to
> rotate under the heads. 

Nothing, including ZFS, has such an in-depth knowledge of the inner drive geometry as to know how long is necessary for the rotational latency to come around.  Also, rotational latency is almost nothing compared to head seek.  For this reason, short-stroking makes a big difference, when you have a data usage pattern that can easily be confined to a small number of adjacent tracks.  I believe, if you use a HDD for log device, it's aware of itself and does short-stroking, but I don't actually know.  Also, this is really a completely separate subject.