[OpenIndiana-discuss] Recommendations for fast storage

[Sašo Kiselkov]

On 04/16/2013 10:57 PM, Bob Friesenhahn wrote:
> On Tue, 16 Apr 2013, Jay Heyl wrote:
>>
>> It's actually not all that difficult to saturate a 6Gb/s pathway with ZFS
>> when there are multiple storage devices on the other end of that path. No
>> single HDD today is going to come close to needing that full 6Gb/s,
>> but put
>> four or five of them hanging off that same path and that ultra-super
>> highway starts looking pretty congested. Put SSDs on the other end and
>> the
>> 6Gb/s pathway is going to quickly become your bottleneck.
> 
> SATA and SAS are dedicated point-to-point interfaces so there is no
> additive bottleneck with more drives as long as the devices are directly
> connected.

Not true. Modern flash storage is quite capable of saturating a 6 Gbps
SATA link. SAS has an advantage here, being dual-port natively with
active-active load balancing deployed as standard practice. Also please
note that SATA is half-duplex, whereas SAS is full-duplex.

The problem with SATA vs SAS for flash storage is that there are, as
yet, no flash devices of the "NL-SAS" kind. By this I mean drives that
are only about 10-20% more expensive than their SATA counterparts,
offering native SAS connectivity, but not top-notch "enterprise"
features and/or performance. This situation existed in HDDs not long
ago: you had 7k2 SATA and 10k/15k SAS, but no 7k2 SAS. That's why we had
to do all that nonsense with SAS to SATA interposers (I have an old Sun
J4200 with 1TB SATA drives that had an interposer on each of the 12
drives). Since then, NL-SAS largely made this route obsolete, so now I
just buy 7k2 NL-SAS drives and skip the whole interposer thing.

Now if any of the big storage drive got their shit together and started
offering flash storage with native SAS at slightly above SATA prices,
I'd be delighted. Trouble is, the manufacturers seem to be trying to
position SAS SSDs as "even more expensive/performing than SAS HDDs"
types of products. When I can buy a 512GB SATA SSD for the price of a
600GB SAS drive, that seems a strange proposition indeed...

--
Saso