[OpenIndiana-discuss] auto-snapshot replicate problems

[Jake Young]

I had issues using the auto-snapshots for "syncing".  I actually was just
sending regular snapshots from my OI laptop to my OI server.

I wrote a little perl script that might do what you need.

The basic concepts I use are to
1) Use a unique snapshot prefix for my "backup" snapshots
2) Only keep one "backup" snapshot on my laptop at any time
3) If the snapshot is missing anywhere, resend the whole filesystem

My script is a perl script, but below is a walk through of the zfs commands
I used.  The script is run from the sending machine.

I gave up on using the auto-snapshots for this once I realized that I
couldn't reliably predict when snapshots got cleaned up and especially when
auto-snapshots failed to be created.  I still use auto-snapshots for its
original purpose, to take frequent local snapshots.  The local snapshots
don't interfere with my backup snapshots and vice versa.

Basic logic is below:

# Loop through all of my zfs filesystems
zfs list -o name | while read zfspart

# Create a new snapshot on my laptop to send over to my server
zfs snapshot ${zfspart}@backup-${timestamp}
#store the name of the new snapshot in $zfssnap

# Get the latest snapshot on my server for this filesystem
zfssnapold=`ssh $host /usr/sbin/zfs list -Ho name -t snapshot | grep
"backup/laptop/$zfspart\@backup-" | sort | tail -1 `

#Check that the latest snapshot matches the earliest snapshot on my laptop

#if they match then send an incremental backup and then delete the old
local snapshot
zfs send -i $zfssnapold $zfssnap | ssh -C $host pfexec /usr/sbin/zfs recv
-F backup/laptop/$zfspart
zfs destroy $zfssnapold

#if they don't match and there is no local snapshot (but there is one on
the server); delete the remote snapshot and send the whole filesystem
ssh $host pfexec /usr/sbin/zfs destroy backup/laptop/$zfssnapold
zfs send $zfssnap | ssh -C $host pfexec /usr/sbin/zfs recv -F
backup/laptop/$zfspart

#if there is no snapshot on the server, then just send the whole filesystem
zfs send $zfssnap | ssh -C $host /usr/sbin/zfs recv -F
backup/laptop/$zfspart

Best of luck John!

Jake

On Mon, Mar 26, 2012 at 10:37 AM, John McEntee <
jmcentee at stirling-dynamics.com> wrote:

> Just reply just in-case someone else comes across this post.
>
> Altering the time-slider daemon would be tricky, but writing my own with a
> crontab would still take some time to replicate all the features I would
> like (clearing up old snapshots etc.)
>
> As the auto-replicate script is use ( any other script would be the same)
> needs to know the last destination snapshot and the current sending
> snapshot. I have implemented a simple zfs hold and zfs release to lock the
> last snapshot sent (tag of destination hostname) and release the previous
> one.
>
> John
> -----Original Message-----
> From: Milan Jurik [mailto:milan.jurik at xylab.cz]
> Sent: 14 March 2012 11:01
> To: Discussion list for OpenIndiana
> Subject: Re: [OpenIndiana-discuss] auto-snapshot replicate problems
>
> Hi,
>
> I think time-slider daemon is in JDS consolidation. But the question is
> - why do you want to hack it and maintain own hack? Why not to create zfs
> snapshots by own crontab entry?
>
> Best regards,
>
> Milan
>
> On 14.03.2012 12:38, John McEntee wrote:
> > I have a problem trying to keep a second fileserver in sync with the
> > primary one.
> >
> >
> >
> > I have a script (auto-replicate) that I have slightly altered that
> > basically
> > checks the latest snapshot of the both  master server and the dr
> > server. It
> > then does a send -I if there is a difference. This used to work fine
> > in the
> > old versions of the time-slider service.
> >
> >
> >
> > The problem I have is the later version of the time-slider service in
> > OpenIndiana cleans up old snapshots of zero size, which means the
> > latest
> > snapshot on the dr server no longer exists on the master server so
> > the send
> > -I fails.
> >
> >
> >
> > My fix would be to hack the time-slider service to not clean up the
> > zero
> > size snapshots. Please can anyone point me in the right direction? I
> > cannot
> > even find the code at the moment. I would also be interested in any
> > other
> > solutions
> >
> >
> >
> > Thanks
> >
> >
> >
> > John
> >
> >
> >
> >
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
>
> _______________________________________________________________________
>
> The contents of this e-mail and any attachment(s) are strictly
> confidential and are solely for the person(s) at the e-mail address(es)
> above. If you are not an addressee, you may not disclose, distribute, copy
> or use this e-mail, and we request that you send an e-mail to
> admin at stirling-dynamics.com and delete this e-mail.  Stirling Dynamics
> Ltd. accepts no legal liability for the contents of this e-mail including
> any errors, interception or interference, as internet communications are
> not secure.  Any views or opinions presented are solely those of the author
> and do not necessarily represent those of Stirling Dynamics Ltd. Registered
> In England No. 2092114 Registered Office: 26 Regent Street, Clifton,
> Bristol. BS8 4HG
> VAT no. GB 464 6551 29
> _______________________________________________________________________
>
> This e-mail has been scanned for all viruses MessageLabs.
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>