[OpenIndiana-discuss] [illumos-Developer] ZFS spare disk usage issue
Roy Sigurd Karlsbakk
roy at karlsbakk.net
Fri Mar 4 18:38:06 UTC 2011
We're using 12GB for the smaller ones and up to 64GB for the larger ones with more traffic. L2ARC is usually two 256GB Micron (former Crucial) C300s, but larger on others. And no, we're NOT using USB storage for this. IMHO USB storage is strictly for termporary/removable storage. The only exception I can think of, is some SuperMicro machines that come with an internal DVD attached to USB. Apart from that, forget USB mass storage. Use SAS or SATA or something else that works.
roy
----- Original Message -----
> http://bugs.illumos.org/ works for me from here.
>
> We're always delicate with our ZFS systems ... we've been trialling
> ZFS raid over USB for quite a while, since early ZFS days, and we were
> never sure if it was the USB or the ZFS ...
>
> If you remove all your ZFS drives, reboot, export, "devfsadm -C" and
> then carefully import drives into your array, not too many at once you
> can usually get the system back, but that could just have been down to
> not enough memory, or use of external USB devices.
>
> how much memory do you have in your computers?
>
> On 4 March 2011 17:03, Roy Sigurd Karlsbakk <roy at karlsbakk.net> wrote:
> > So should I post a bug, or is there one there already?
> >
> > Btw, I can't reach http://bugs.illumos.org/ - it times out
> >
> > roy
> >
> > ----- Original Message -----
> >> We've talked about this, and I will be putting together a fix for
> >> this
> >> incorrect state handling. :-)
> >>
> >> - Garrett
> >>
> >> On Fri, 2011-03-04 at 11:50 -0500, Eric Schrock wrote:
> >> > This looks like a pretty simple bug. The issue is that the state
> >> > of
> >> > the SPARE vdev is being reported as REMOVED instead of DEGRADED.
> >> > If
> >> > it were the latter (as it should be), then everything would work
> >> > just
> >> > fine. Please file a bug at bugs.illumos.org.
> >> >
> >> >
> >> > On a side note, this continues to expose the overly simplistic
> >> > vdev
> >> > state model used by ZFS (one which I can take a bulk of the
> >> > responsibility for). Back before the days of ditto blocks and
> >> > SPA3.0,
> >> > it was sufficient to model state as a fairly binary proposition.
> >> > But
> >> > this now has ramifications that don't necessarily make sense. For
> >> > example, one may be able open a pool even if a toplevel vdev is
> >> > faulted. And even when a spare has finished resilvering, it's
> >> > left
> >> > in
> >> > the DEGRADED state, which has implications for allocation
> >> > policies
> >> > (though I remember discussions around changing this). But the
> >> > pool
> >> > state is derived directly from the toplevel vdev state, so if you
> >> > switch spares to be ONLINE, then 'zpool status' would think your
> >> > pool
> >> > is perfectly healthy. In this case it's true from a data
> >> > protection
> >> > standpoint, but not necessarily from a "all is well in the world"
> >> > standpoint, as you are down one spare, and that spare may not
> >> > have
> >> > the
> >> > same RAS properties as other devices in your RAID-Z stripe (it
> >> > may
> >> > put
> >> > 3 disks on the same controller in one stripe, for example).
> >> >
> >> >
> >> > - Eric
> >> >
> >> > On Fri, Mar 4, 2011 at 7:06 AM, Roy Sigurd Karlsbakk
> >> > <roy at karlsbakk.net> wrote:
> >> > Hi all
> >> >
> >> > I just did a small test on RAIDz2 to check whether my
> >> > suspicion was right about ZFS not treating spares as
> >> > replicas/copies of drives, and I think I've found it
> >> > true.
> >> > The
> >> > short story: If two spares replaces two drives in raidz2,
> >> > losing a third drive, even with the spares active, makes
> >> > the
> >> > pool unavailable. See full report on
> >> >
> >> > ODT:
> >> > http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.odt
> >> > PDF:
> >> > http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.pdf
> >> >
> >> > Vennlige hilsener / Best regards
> >> >
> >> > roy
> >> > --
> >> > Roy Sigurd Karlsbakk
> >> > (+47) 97542685
> >> > roy at karlsbakk.net
> >> > http://blogg.karlsbakk.net/
> >> > --
> >> > I all pedagogikk er det essensielt at pensum presenteres
> >> > intelligibelt. Det er et elementært imperativ for alle
> >> > pedagoger å unngå eksessiv anvendelse av idiomer med
> >> > fremmed
> >> > opprinnelse. I de fleste tilfeller eksisterer adekvate og
> >> > relevante synonymer på norsk.
> >> >
> >> > _______________________________________________
> >> > Developer mailing list
> >> > Developer at lists.illumos.org
> >> > http://lists.illumos.org/m/listinfo/developer
> >> >
> >> >
> >> >
> >> > --
> >> > Eric Schrock
> >> > Delphix
> >> >
> >> >
> >> > 275 Middlefield Road, Suite 50
> >> > Menlo Park, CA 94025
> >> > http://www.delphix.com
> >> >
> >> >
> >> > _______________________________________________
> >> > Developer mailing list
> >> > Developer at lists.illumos.org
> >> > http://lists.illumos.org/m/listinfo/developer
> >
> > --
> > Vennlige hilsener / Best regards
> >
> > roy
> > --
> > Roy Sigurd Karlsbakk
> > (+47) 97542685
> > roy at karlsbakk.net
> > http://blogg.karlsbakk.net/
> > --
> > I all pedagogikk er det essensielt at pensum presenteres
> > intelligibelt. Det er et elementært imperativ for alle pedagoger å
> > unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de
> > fleste tilfeller eksisterer adekvate og relevante synonymer på
> > norsk.
> >
> > _______________________________________________
> > OpenIndiana-discuss mailing list
> > OpenIndiana-discuss at openindiana.org
> > http://openindiana.org/mailman/listinfo/openindiana-discuss
> >
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
--
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy at karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
More information about the OpenIndiana-discuss
mailing list