[OpenIndiana-discuss] AMCC/3Ware 9550SX-8LP

[Christopher Chan]

On Friday, January 25, 2013 06:18 AM, Edward Ned Harvey (openindiana) wrote:
> When you're doing async writes (or sync writes with ZIL disabled, which are in effect, async writes) then the HBA write-back actually hurts your performance a little bit.  Like 5-10% or so.
>
> The conclusion to draw is:
>
> (a)  The absolute fastest thing you can do is use a dumb controller, and sync=disabled.  This maximizes performance for both sync & async writes.  But only if it's safe for you to run in that configuration.  Otherwise ...
>
> (b)  The second fastest thing you can do is use a dumb controller, and a dedicated log device.  This again maximizes performance for both sync & async, but now it's safe for whatever your usage patterns.
>
> (c)  The third fastest thing is to have a smart HBA with NV write-back.  This hurts async write performance slightly, but the sync write performance is far better than the dumb & naked bare-bones solution.
>
> (d)  The dumb & naked bare-bones solution is to use a dumb controller, with ZIL enabled (which is default), and no special log device.  Your async writes are optimized, but your sync writes are a dog, which tend to also drag your async writes into the gutter with them.  Believe it or not, dedicating a HDD slog to a HDD pool actually improves performance signfiicantly just because you're able to isolate the sync writes to a device while the async can still hit the pool without interference.  But this is kind of a dumb configuration, so we don't talk about it much except for entertainment purposes.  ;-)
>
> So basically, no matter what you do, you're not going to significantly impact the async write performance.  The sync performance is the only thing you have to think about optimizing.
>
> The performance delta from (d) to (c) is approximately proportional to (c) vs (b), which is approximately proportional to (b) vs (a).  A reasonable ballpark is to say approx 1.75x to 2x jump on each of those deltas.
>
> Long story short:  This was all a response to the intended-to-be-tangential comment "Who uses hardware raid anymore now that we have zfs?"  And the answer is, "Anybody who has an HBA, doesn't have an SSD, and can't or won't disable the ZIL."  This is a surprisingly high number of people, because people previously bought (and continue to buy) servers that have HBA's, which later get provisioned to run zfs.  And then lots of times, they don't know or can't spring for the slog solution.
>

:-D I'm here to entertain since I have not been able to spring for a ssd 
for use as a slog. :-D

Thanks Edward.