[OpenIndiana-discuss] Recommendations for fast storage

[Jim Klimov]

On 2013-04-17 01:12, Timothy Coalson wrote:
> On Tue, Apr 16, 2013 at 6:01 PM, Jim Klimov <jimklimov at cos.ru> wrote:
>
>> On 2013-04-16 23:56, Jay Heyl wrote:
>>
>>> result in more devices being hit for both read and write. Or am I wrong
>>> about reads being issued in parallel to all the mirrors in the array?
>>>
>>
>> Yes, in normal case (not scrubbing which makes a point of reading
>> everything) this assumption is wrong. Writes do hit all devices
>> (mirror halves or raid disks), but reads should be in parallel.
>> For mechanical HDDs this allows to double average read speeds
>> (or triple for 3-way mirrors, etc.) because different spindles
>> begin using their heads in shorter strokes around different areas,
>> if there are enough concurrent randomly placed reads.
>>
>
> There is another part to his question, specifically whether a single random
> read that falls within one block of the file hits more than one top level
> vdev - to put it another way, whether a single block of a file is striped
> across top level vdevs.  I believe every block is allocated from one and
> only one vdev (blocks with ditto copies allocate multiple blocks, ideally
> from different vdevs, but this is not the same thing), such that every read
> that hits only one file block goes to only one top level vdev unless
> something goes wrong badly enough to need a ditto copy.

I believe so too... I think, striping over top-level vdevs is subject
to many tuning and algorithmical influences, and is not as simple as
even-odd IOs. Also IIRC there is some size of data (several MBytes)
that is preferably sent as sequential IO to one TLVDEV, then it is
striped over to another, in order to better utilize the strengths of
faster sequential IO vs. lags of seeking.

But I may be way off-track here with this "belief" ;)

Jim