[OpenIndiana-discuss] dedup status

[Jonathan Adams]

On 1 October 2010 11:22, Albert Lee <trisk at opensolaris.org> wrote:

> On Fri, Oct 1, 2010 at 4:54 AM, Jonathan Adams <t12nslookup at gmail.com>
> wrote:
> > Thanks for this, please keep us informed.
> >
> > We have multiple systems with ZFS raid (3 disks per pool) that are
> > snapshot'd and those snapshots are rsynced over the internet and replayed
> > onto a duplicate server in a different location ... currently these
> servers
> > are on Solaris 10.
> >
>
> If it's an Solaris 10 update which has the zfs send stream format
> stabilised, you should be able to send snapshots incrementally,
> instead of via rsync.
>

we zfs send to files and rsync the snapshots, we found that if our network
died during a combined send/receive the server would hang and sometimes the
ZFS didn't recover nicely.

the rsync is used to automate the SFTP sessions with wrapper scripts at both
ends of the server chain.


> > I would be really interested to find out how well this sort of situation
> > works over time; on the current system, we have had some weird ZFS
> > corruption which we have had to patch for.
> >
>
> Any details on this?
>
> They were patched in Solaris 10, but we have snapshots that can't be
deleted and can't be accessed ... we cloned an earlier snapshot and made
that one the active branch so that we could receive into it again, but we
kinda have a dead lump hanging off the side that is taking up some space.

It's not too bad, it's in MB not GB ...

Also most of our ZFS storage is external USB, and there are all sorts of
limitations with USB's that you need to set kernel parameters to fix ... and
on a couple of systems (I think they're dells, but we have a few Sun x86
boxes) the ZFS system dies/reboots if you plug in a USB mouse.

We have also had to introduce more servers to our ZFS server pool because we
reached some kind of limit to the number of ZFS disks that could be mounted.
When we removed a hot spare the system stabilised ... somewhere around 9 USB
disks is about our limit, give or take a couple and a lot of luck.

Jon