[OpenIndiana-discuss] Current ZFS Backup projects (OpenIndiana-discuss Digest, Vol 26, Issue 34)

[Ong Yu-Phing]

Jim, I assume you are referring to this: 
http://wiki.openindiana.org/oi/rsync+daemon+service+on+OpenIndiana, thanks!

My concern is that typically rsync will take quite a while to traverse a 
large set of files before sending only changed files; a classic example 
is backing up say 1TB of maildir emails, it may take 4+ hours, and you 
now have to deal with a situation where your midnight backup is really a 
"somewhere between midnight and 4am" backup.  And of course, if you want 
to take snapshots/backups of comstar volumes, rsync isn't quite the 
right fit.

On the other hand, a zfs snapshot gives an almost-at-the-time backup 
(give or take a few seconds), versus the aforementioned rsync.  The zfs 
snapshot can then be sent off-site, independent of the backup activity.

However, having used zetaback, warmer, as well as other zfs scripts, 
what I've noticed so far is that sometimes, the snapshot send/receive 
"hangs" (for want of a better word), then the other automated, scheduled 
jobs then queue and "hang" behind the first (since they tend to be 
incremental).

If anybody has encountered and found a solution or even a way to 
troubleshoot why that particular job "hangs", I'd appreciate any feedback!

Phing

On 12/09/2012 01:47, Jim Klimov wrote:
> 2012-09-11 18:56, Mark Creamer wrote:
>> A recent thread caused me to look for open source projects that leverage
>> ZFS to backup systems. I found a couple, such as OmniTI's
>> Zetaback<http://labs.omniti.com/labs/zetaback>,
>> but that one appears to be dead - at least the links don't work and 
>> the Git
>> page shows no recent activity. Nexenta's commercial product for Windows,
>> "Delorean", also appears to have been killed (unfortunately without 
>> first
>> being released to the community as far as I can tell). Wondering if 
>> anyone
>> knows of any other projects that use scripts or some other method to 
>> manage
>> system backups with zfs. I'm hoping to build on the ideas of someone 
>> more
>> knowledgeable to automate my snapshot and recovery efforts.
>
>
> Regarding backup of other OSes to ZFS servers and leveraging ZFS
> from there on, I can suggest rsync (or cwrsync in case of Windows
> clients). This is easily automatable on clients (push to rsync
> server) or on the server (pull from clients), provided that you
> set up the security/logins appropriately.
>
> The ZFS-enabled server can take care of snapshotting successful
> rsync run results, deduplication if appropriately spec'ed, etc.
>
> I posted an enhanced SMF script and config file snippets on OI
> wiki to run the rsync server and take ZFS auto-snapshots after
> successful completion of incoming rsync sessions (push mode);
> it is even more trivial in pull mode initiated by the server -
> it can "mkdir .../.zfs/snapshot/$SNAPNAME" given the proper
> access via "zfs allow").
>
> HTH,
> //Jim Klimov
>
>
>