[OpenIndiana-discuss] Recommendations for fast storage

[Timothy Coalson]

On Tue, Apr 16, 2013 at 6:01 PM, Jim Klimov <jimklimov at cos.ru> wrote:

> On 2013-04-16 23:56, Jay Heyl wrote:
>
>> result in more devices being hit for both read and write. Or am I wrong
>> about reads being issued in parallel to all the mirrors in the array?
>>
>
> Yes, in normal case (not scrubbing which makes a point of reading
> everything) this assumption is wrong. Writes do hit all devices
> (mirror halves or raid disks), but reads should be in parallel.
> For mechanical HDDs this allows to double average read speeds
> (or triple for 3-way mirrors, etc.) because different spindles
> begin using their heads in shorter strokes around different areas,
> if there are enough concurrent randomly placed reads.
>

There is another part to his question, specifically whether a single random
read that falls within one block of the file hits more than one top level
vdev - to put it another way, whether a single block of a file is striped
across top level vdevs.  I believe every block is allocated from one and
only one vdev (blocks with ditto copies allocate multiple blocks, ideally
from different vdevs, but this is not the same thing), such that every read
that hits only one file block goes to only one top level vdev unless
something goes wrong badly enough to need a ditto copy.

Tim