[OpenIndiana-discuss] hardware specs for illumos storage/virtualization server

[Paul B. Henson]

On Sat, Nov 17, 2012 at 01:06:46PM +0100, Robin Axelsson wrote:

> What you are going for seems like quite an ambitious undertaking! 

Yeah, it's probably way overkill for my needs, but I'll have a lot of
fun setting it up and having it, and it should last for a long time.

> Regarding near-line SAS, I don't really understand what that is all 
> about. When looking at prices of hard drives, say a 2TB true-SAS drive 
> is maybe $10-20 more expensive than an equivalent S-ATA drive. So I 
> basically don't agree with you that SAS, is "ridiculously expensive" 
> compared to SATA, at least not on the hard drive side. With that in mind 
> it makes even less sense to go with near-line SAS.

I meant near-line SAS was way more expensive than desktop class SATA,
not enterprise SATA. There isn't too much price difference between
near-line SAS and enterprise SATA. For the most part I think near-line
SAS is just the SATA drive with a SAS interface, which gives you better
queueing, as well as being dual-ported. SATA also doesn't always play
well behind a SAS expander, so if you're using one of those you're
better off with SAS drives. I'd love to go near-line SAS or enterprise
SATA, but my budget is way high already and when you multiply the delta
by 13 drives it really adds up.

> As for the WD red, I would be careful with EcoGreen drives and the other 
> shoestring budget drives out there but that's my two cents.

I definitely wouldn't go WD Green. The red is intended for this type of
deployment, although probably on a smaller scale, and doesn't ever park
its heads with the default config.

> least make sure you have a good warranty policy for them which I don't 
> believe you will have.

3 years, not too bad.

> considering the other hardware specs of your system you seem to have 
> high requirements, so such drives will be prone to give you a headache 
> from time to time.

Yeah :(. For the other components, going premium on one piece here or
two pieces there isn't too bad, but with so many drives, going premium
just adds up too quick.

> If you need more HBAs, have you looked at the IBM ServeRAID M1015 ? If 
> you are happy with 3 Gb/s then you can go for an Intel SASUC8I.

I haven't, but thanks for the suggestions. The IBM card looks promising,
not too expensive, can be cross-flashed to IT firmware, 8 channels,
cool. Given I'm only going to have one drive per channel, 3Gb/s would be
fine, if I end up going that way.

> By "slog" I take it that you mean ZIL. If I understand correctly, if a 
> ZIL gets corrupted then the entire pool it belongs to will fall apart.

slog is the terminology for when you have a dedicated separate device
for the ZIL rather than having it live on the drives in the pool.

I don't believe that's the case. The ZIL is write-only except for when
the pool is imported, and if a write fails the slog is just marked bad
and the zil falls back to being embedded in the pool drives just like a
pool without a slog. In the past, a pool with a missing slog couldn't be
imported, and you couldn't remove a slog, so it was pretty important to
mirror them. But with current zfs code those issues have been resolved.
The only issue with failed slog is if there are unprocessed transactions
in the zil after a crash, and the slog fails before the pool is
imported, you lose them. But even in that case your pool isn't toast, it
just might be missing some changes clients think were finalized.

Thanks for the input...