[OpenIndiana-discuss] Recommendations for fast storage

[Jim Klimov]

On 2013-04-16 23:56, Jay Heyl wrote:
> result in more devices being hit for both read and write. Or am I wrong
> about reads being issued in parallel to all the mirrors in the array?

Yes, in normal case (not scrubbing which makes a point of reading
everything) this assumption is wrong. Writes do hit all devices
(mirror halves or raid disks), but reads should be in parallel.
For mechanical HDDs this allows to double average read speeds
(or triple for 3-way mirrors, etc.) because different spindles
begin using their heads in shorter strokes around different areas,
if there are enough concurrent randomly placed reads.

Due to ZFS data structure, you know your target block's expected
checksum before you read its data (from any mirror half, or from
data disks in a raidzn); then ZFS calculates the checksum of the
data it has read and combined, and only if there is a mismatch, it
has to read from other disks for redundancy (and fix the detected
broken part).

HTH,
//Jim