[OpenIndiana-discuss] ZFS pool layout - request for opinion

[John McEntee]

My best servers have 24 hot swap bays in a 4 U case (DIY build) using 3 x 8 port LSI cards. 3 drives 3 way mirror across the 3 controllers ,2 x 256GB ssd one spinning rust, for rpool, early days of SSD and was worried about reliability.  Only 20Gb or so of the ssds is for the rpool the rest is parition as a ZIL and cache for the main zpool. The other 21 x 4TB disk are either 3 way mirrors (across the controllers), this time to rebuild from a failed 4TB disk is a long time, and I have a little extra user error protection at failure time. Or for a backup server 3 x 6 x 4TB RAIDZ2 ( 2 on each controller in a set) (currently 3 slots empty). In either build I can lose a single disk controller, or at least 2 disks and still function.

Only the backup servers have dedup enabled with 96GB of RAM. (ratio of 3.6)


-----Original Message-----
From: Jonathan Adams [mailto:t12nslookup at gmail.com] 
Sent: 26 February 2014 10:57
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] ZFS pool layout - request for opinion

2x 256GB + 2x(2x4TB) , 256GB in the root pool, 8Tb in the datapool.

However, when we got to 6 servers with the same configuration it was deemed to be wasteful on rpools (not to mention electricity and management costs), especially since they weren't being used for live storage, only for backup, so we created a new storage array, 6x4x4TB hot-plug disks, 2 SSD internal raid, dual LSI 16 port controllers ... but in that case most of the pools are 2x4TB mirrored, (12 pools of 4TB) but since they're archiving physical servers at 12 sites, and most of them have ~2TB mirrored at each we should be fine.

Ooh that was a long sentence.

Trouble with that configuration is that it's an all eggs in one basket issue, so we had to have a second one off-site that clones the original, looks a lot nicer than 6 towers though.

Jon


On 26 February 2014 10:47, Stefan Müller-Wilken < stefan.mueller-wilken at acando.de> wrote:

> Disks are cheaper than data - that is indeed true. So you're putting 
> data into a four-time mirrored configuration? Or RAIDZ-1 or RAIDZ-2?
>
> The question is more of maximum capacity. If you're dedicating two 
> disks to the root pool, you're left with max. 4 4TB disks for the rest 
> - effectively wasting >6TB of potential space. Booting RAIDZ pools is 
> not possible, right? If it was, you could put all six disks in one 
> large RAIDZ and cut rpool and data out of that, no?
>
> Cheers
>  Stefan
> ________________________________________
> Von: Jonathan Adams [t12nslookup at gmail.com]
> Gesendet: Mittwoch, 26. Februar 2014 10:30
> An: Discussion list for OpenIndiana
> Betreff: Re: [OpenIndiana-discuss] ZFS pool layout - request for 
> opinion
>
> Basically the same thing in our 6 disk systems.
>
> We generally have 2 256Gb disks in the mirrored root pool, and 4 4Tb 
> disks mirrored in the "datapool" ... but disks are cheap overall, 
> compared to your data.
>
> Jon
>
>
>
> On 26 February 2014 08:06, Stefan Müller-Wilken < 
> stefan.mueller-wilken at acando.de> wrote:
>
> > Dear all,
> >
> > one thing I stumble over again and again is ZFS pool layout for 6 
> > disk servers (e.g. 1 HU ProLiants) . What I do so far is reserving 
> > disk 1 & 2 for the rpool mirror and put disks 3 - 6 into a RAIDZ-1 
> > data pool. I
> don't
> > really like this layout as it means wasting a lot of space on the 
> > rpool while having only minimal redundancy for data. What do others 
> > do to
> combine
> > space efficiency with bootability and redundancy?
> >
> > Cheers
> >  Stefan
> >
> > ________________________________
> > Acando GmbH, Millerntorplatz 1, 20359 Hamburg, Germany | Geschäftsführer:
> > Guido Ahle | Amtsgericht Hamburg, HRB 76048 | 
> > Ust.Ident-Nr.:DE208833022 
> > _______________________________________________
> > OpenIndiana-discuss mailing list
> > OpenIndiana-discuss at openindiana.org
> > http://openindiana.org/mailman/listinfo/openindiana-discuss
> >
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
> ________________________________
> Acando GmbH, Millerntorplatz 1, 20359 Hamburg, Germany | Geschäftsführer:
> Guido Ahle | Amtsgericht Hamburg, HRB 76048 | 
> Ust.Ident-Nr.:DE208833022
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss at openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

_______________________________________________________________________

The contents of this e-mail and any attachment(s) are strictly confidential and are solely for the person(s) at the e-mail address(es) above. If you are not an addressee, you may not disclose, distribute, copy or use this e-mail, and we request that you send an e-mail to admin at stirling-dynamics.com and delete this e-mail.  Stirling Dynamics Ltd. accepts no legal liability for the contents of this e-mail including any errors, interception or interference, as internet communications are not secure.  Any views or opinions presented are solely those of the author and do not necessarily represent those of Stirling Dynamics Ltd. Registered In England No. 2092114 Registered Office: 26 Regent Street, Clifton, Bristol. BS8 4HG
VAT no. GB 464 6551 29
_______________________________________________________________________

This e-mail has been scanned for all viruses MessageLabs.