[OpenIndiana-discuss] Garrett D'Amore on using SATA drives in a SASsystem

[Richard Elling]

On Jun 26, 2012, at 10:36 AM, Dan Swartzendruber wrote:
> On 6/26/2012 1:15 PM, Richard Elling wrote:
>> On Jun 26, 2012, at 6:29 AM, Dan Swartzendruber wrote:
>> 
>>   
>>> Keep in mind this is almost 2 yrs old, though.  I seem to recall a thread
>>> here or there that has pinned the SATA toxicity issues to an mpt driver bug
>>> or somesuch?
>>>     
>> Not really. Search for other OSes and their tales of woe. In some cases, a
>> bad SATA drive can make the machine fail at POST, well before an OS is
>> loaded.
>> 
>> Best results for SATA is direct-connect: no expanders, no extenders.
>> Next best is SATA with a good-quality SATA/SAS interposer.
>> Known to be a poor mix: both SATA and SAS devices sharing an expander.
>> 
>> NB, we have seen changes in HBA, expander, and disk firmware all contribute
>> to happiness or sadness when SATA devices are used in SAS fabrics (STP).
>> At one time, we tried to keep a list of combinations known for happiness, but
>> maintaining that list proved to be impractical, due to the constant churn and
>> unavailability of disk firmware patches from various vendors. Also, do not assume
>> that you will be able to get a firmware upgrade, even if it exists :-(
>>  -- richard
>> 
>>   
> To clarify: on hardocp a few months back, there was a discussion on this whole issue and there was a cryptic reference to Oracle having supposedly root-caused a software/driver bug that was causing a specific problem.  I wish I could remember anything more specific than that…

I can recall no less than 5 times this was claimed to be fixed. Yet, problems persist.
IMHO the root cause is that the specifications are just loose enough that perfect
interoperability under all failure modes between SAS, STP, and SATA will take
many years, perhaps decades, to "fix"

For example, with one disk vendor, after 12 months of trouble, we were finally 
able to get enough traces and data for them to isolate a race condition they had
been fighting for years. Great work team! Then they EOLed the product without
producing a GA firmware upgrade. I know these devices do not work well under
STP and will never recommend them to be used with expanders. Fortunately, 
they are EOL, so as time flies by we should see fewer and fewer of them.
 -- richard

-- 

ZFS storage and performance consulting at http://www.RichardElling.com