[OpenIndiana-discuss] [smartos-discuss] hardware specs for illumos storage/virtualization server

[Ian Collins]

On 11/17/12 17:36, Paul B. Henson wrote:
> For the motherboard, I'm looking at the Supermicro X9DRD-7LN4F-JBOD,
> which is a dual LGA 2011 socket board with 16 DIMM slots, 2 x SATA3, 4 x
> SATA2, and 8 x SAS (LSI 2308 controller onboard) along with 4 intel i350
> based gig nics. My understanding is that illumos is perfectly happy with
> the LSI 2308 in IT mode. The -JBOD version of this motherboard comes
> from the factory with IT firmware. It doesn't seem readily available
> though, if I went with the regular version the LSI controller comes with
> RAID firmware, it's possible to reflash with IT but from what I've read
> it's a bit of a pain (you need to do it from the EFI shell). It also
> looks like illumos works with the intel i350 gig nics, and I assume
> there should be no issue with the onboard Intel AHCI SATA controller?

That board is similar to the one I use with Solaris 11, the X9DRH-7TF.  
I haven't tried booting a recent Illumos build on it, but I can report 
that, unlike those on Dell PERCs,  the 2308 hasn't been hobbled to 
prevent JBOD mode.  So as long as the recently added driver sees the 
card, you can use the megaCLI utility to enable it.

The on board SATA works fine and you shouldn't have any problems with 
the i350.

> I haven't really spec'd specific RAM, although I'm partial to crucial,
> it takes 1333MHz registered ECC DDR3. I think I want at least 32GB for
> the storage server side, and I'm not sure yet how much more I'll add in
> on top of that for virtualization.

I'm using the same RAM on my system.

> 8 of the 16 3.5" bays will be covered by the onboard LSI controller, I
> need to get an additional PCIe controller with 2 x SFF-8087 connectors
> to cover the rest. Seems there are a fair number of options, although
> I'm not sure if there's a clear winner among them. Any favorites?

I use an LSI 2911 which uses the well supported 2008 chip.

> Hard drives are the parts I'm least confident in 8-/. I'd like to go 2TB
> or 3TB, that's cost prohibitive for near-line SAS, and pretty darn pricy
> for "enterprise" SATA. I don't really want to go with desktop class
> drives though.
>
> Is there any opinion yet on the new WD Red "NAS" drives? They're only
> $170 for a 3TB drive, which is pretty cheap. On the plus side, they're
> engineered for 7x24 operation, have a three year warranty, and are
> supposed to be low power/low heat

I'm happy with WD (non-RAID) blacks for a home system.  As long as your 
case has good cooling on the drives, they'll be fine.

> I also don't really have a good handle on what SSD's to go with. As I
> mentioned, I'm thinking of getting two for rpool/l2arc, and hook them up
> to the onboard SATA3 controller. If I can find ones that are
> appropriate, I'd carve out a third partition on them for a mirrored
> slog; otherwise I'd get a separate third one and stick it in a 3.5" bay
> to be dedicated slog. I don't think I'd bother to mirror the slog if it
> is on a separate SSD, I believe there are no longer any critical failure
> modes from slog failure, worst-case being it fails when the pool is
> off-line and you need to manually import it. Any suggestions on good
> rpool/l2arc/slog SSD's, or rpool/l2arc SSD's with a different model slog
> SSD would be greatly appreciated.

I used a couple of Crucial M4 drives for the ARC and Intel 311s for the ZIL.

-- 
Ian.