[OpenIndiana-discuss] pool messup

[Mark]

On 12/04/2011 8:38 a.m., Roy Sigurd Karlsbakk wrote:
> Persistent mapping is disabled, and has been for some time. lsiutil shows no persistent mappings. The drives are a varity of makes. The system has been the same since I reinstalled it a month or two back.
>
> roy

I would suggest using lsiutil to check the port link speed of the whole 
SAS topology.
You can see the link status and negotiated speed for each port on the 
expander. A poor SAS cable connection to a JBOD can affect connectivity 
and reliability, as can incorrect negotiation of speed.

Moving a controller to a different slot will change the device id's.

I have experienced disk issues when under high load.
In my experience, very few disks work well with expanders.
I use only one model after having issues show up after months of trouble 
free running.

For Supermicro expanders, avoid all 6Gbit sata disks and all non RE WD.
A few Seagates are ok e.g. ST32000542AS , but ST32000641NS (6gbit) are not.
Hitachi disks seem to be a winner all round, but now that WD is buying 
them out, it will likely that they will soon disappear.

Bad or suspect blocks can also show up as a problem with timeouts.

The zfs timeout issue is usually the result of the long error retry that 
nearly all desktop (except Hitachi) disks use by default.
My take on this is the manufacturers method of forcing you to use 
Enterprise disks, since this behaviour breaks most raid systems.
The end user used to have control of the timeout, but it has now been 
hard coded to very long.


>
> ----- Original Message -----
>> Hi Roy,
>>
>> The drive order issue is likely to be persistent mapping.
>> It can be disabled using lsiutil.
>> I found it was enabled by default on early cards, and disabled on
>> later
>> ones, but changing the firmware doesn't change the origional setting.
>>
>> What model are the disks ?
>>
>> Did the origional system also use expanders ?
>>
>> Mark.
>>
>> On 12/04/2011 7:33 a.m., Roy Sigurd Karlsbakk wrote:
>>>> This is an entry level proliant, very standard gear. I cannot
>>>> believe that you would be the first one using them with ZFS (those
>>>> sell really well) Is the Jbod configured? I mean, are all the the
>>>> drives correctly defined in whatever config utility provided at
>>>> boot
>>>> time?
>>>
>>> Yes
>>>
>>>> Are the drives in both chassis failing?
>>>
>>> Yes
>>>
>>>> Supermicro is affordable and pretty basic but I fear using them in
>>>> any
>>>> other way than what they were designed for. Is there something on
>>>> those chassis that you were supposed to configure? Those devices
>>>> are
>>>> designed and tested with very traditional raid an jbod setups in
>>>> mind... think if there is something you didnt do but you would need
>>>> to
>>>> do in case of setting up a traditional jbod storage...
>>>
>>> We use supermicros on some rather largish servers, and they're very
>>> stable. Thee issue here is probably a driver or controller issue,
>>> see
>>> http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg32079.html
>>>
>>>> Back when I had my problem I was testing all the drives 1 by 1 and
>>>> all
>>>> the cables, and all the different combinations of things... it was
>>>> all
>>>> useless. when something so fishy appears beware of the controllers.
>>>> To
>>>> make a memtest is a good idea, you could let it running over night,
>>>> if
>>>> memory corrupts probably there is some hardware problem in the
>>>> motherboard.
>>>>
>>>> Thats all that comes to my mind... is the hardrive order swapping
>>>> between reboots?
>>>>
>>>> But my bet is supermicro, or the way you configured the jbod
>>>> utility.
>>>
>>> The controller is running a firmware without RAID support, so
>>> everything's JBOD. The storage chassises don't have any
>>> configuration, they just have a SAS expander, that's all.
>>>
>>> Vennlige hilsener / Best regards
>>>
>>> roy
>>> --
>>> Roy Sigurd Karlsbakk
>>> (+47) 97542685
>>> roy at karlsbakk.net
>>> http://blogg.karlsbakk.net/
>>> --
>>> I all pedagogikk er det essensielt at pensum presenteres
>>> intelligibelt. Det er et elementært imperativ for alle pedagoger å
>>> unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de
>>> fleste tilfeller eksisterer adekvate og relevante synonymer på
>>> norsk.
>>>
>>> _______________________________________________
>>> OpenIndiana-discuss mailing list
>>> OpenIndiana-discuss at openindiana.org
>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>
>>
>> _______________________________________________
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss at openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>