[OpenIndiana-discuss] Opinions on LSI RAID vs. ZFS

[Saso Kiselkov]

On 10/8/13 2:28 PM, Stefan Müller-Wilken wrote:
> Dear all,
> 
> I know, this is only in parts an OpenIndiana related questions but maybe someone might share their experience / opinions...
> 
> I have a customer running OI157_a8 on a Dell DL380-G7. Works like a charm and everyone's happy. There's only one downside to the configuration and that's the SmarArray P410i RAID controller. Crappy to use with ZFS as it can't be run in JBOD mode. We had to configure it with 8 RAID0 logical volumes which makes disk replacement a nightmare. Anyway, it's not a certified Oracle configuration so you sometimes have to make compromises...
> 
> Now a few weeks ago we received an Oracle T3-1 SPARC server in similar configuration: 8 port SAS backplane, MegaRaid 9261-8i, fully equipped with 8 disks. To my dismay I had to realize that as with the DL380 you *cannot* switch of the LSI's RAID feature and while HP's SmartArray acucli is clumsy, LSI's MegaCLI is just a joke.
> 
> Our Oracle tech rep gave advice to configure the 8 disks in single drive RAID0 and run them in RAIDZ but it seems that this is lose-lose: ZFS does not live up to its potential in volume and fault management (e.g. not being able to reliably switch to spares) and you still have to use MegaCLI to handle failed drives. On the other hand, using a pure LSI RAID setup, I lose all ZFS beauty. We were finally told that going for other controllers was no option with SUN SGX-SAS6-INT-Z not being certified for the T3-1.
> 
> Question now: what would you recommend? 8 LSI RAID0 LVs under ZFS, 8 drives under LSI RAID-5, or switch to an Oracle certified controller with JBOD mode (which one??)? Does it make sense to go for Soft RAID anyway, with that configuration?

I do have a bunch of systems where I'm running ZFS on top of LSI HW
RAID-1 for rpool. Works well enough for me with copies=2 and frequent
scrubs (HW RAID takes care of device failures, copies=2 takes care of
silent data corruption). Suboptimal, but since these are only boot pools
and are otherwise untouched during normal operation, it was a tradeoff I
was willing to make.

As for HW RAID vs. JBOD on high-volume production, I'd definitely
recommend going the single-drive RAID-0 route. Yes, disk replacement can
be a PITA, but the performance and reliability tradeoffs are simply too
good to give up.

Cheers,
-- 
Saso