[OpenIndiana-discuss] WD reds any good for ZFS RAIDZ pools?

[Russ Price]

On 06/21/2013 06:48 PM, Ian Collins wrote:
> Christopher Chan wrote:
>> On Saturday, June 08, 2013 05:23 AM, Paul B. Henson wrote:
>>> For these large drives, raidz2 is recommended, as the likelihood of a
>>> second drive failure during a rebuild starts to get pretty high.
>>>
>> What's wrong with striped mirrors? raidz2 does not like random i/o. I
>> happen to run raidz2 and regret it now.
>
> Nothing much, it's just that 3TB drives can take a long time (days) to resilver
> on a moderately busy pool and raidz leaves you exposed to a second drive failure
> during that process.
>
> There's still a risk of two dives in one mirror failing, but that would be less
> than the raidz case.
>
> If someone is willing to suffer the performance of raidz, raidz2 is definitely a
> better choice!
>

Regardless of whether you run striped mirrors, RAIDZ, or RAIDZ2, beware of 
controller incompatibilities with 3 TB drives, especially if you're using an SAS 
HBA. I got bitten by that last week when trying to use 3 TB WD Reds on an Intel 
SASUC8I (reflashed with IT firmware that can properly negotiate link speed with 
6 GB/s drives).

In the process of trying to get to the reason why they were showing up as 2 TB 
drives and getting ashift=9 instead of ashift=12, I very nearly trashed my old 
storage pool (4x1.5 TB drives in striped mirrors). I'm just now recovering from 
that fiasco. Hint: DON'T DON'T DON'T futz with /kernel/drv/sd.conf unless you 
have up-to-date and OFFLINE backups! I don't care what the ZFS Advanced Format 
Disks wiki page says - you're looking at a world of hurt if you modify sd.conf.

Anyway, the 3 TB drives worked a lot better when connected to the motherboard 
SATA ports instead of the SASUC8I; I get ashift=12 and full capacity, no need to 
touch sd.conf at all.

Now I'm going to need to ditch the SASUC8I in favor of a 9211-8i. :/