[OpenIndiana-discuss] [zfs] ZFS High-Availability and Sync Replication

[Sašo Kiselkov]

On 11/17/2012 03:03 AM, Richard Elling wrote:
> On Nov 15, 2012, at 5:39 AM, Sašo Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
>> I've been lately looking around the net for high-availability and sync
>> replication solutions for ZFS and came up pretty dry - seems like all
>> the jazz is going around on Linux with corosync/pacemaker and DRBD. I
>> found a couple of tools, such as AVS and OHAC, but these seem rather
>> unmaintained, so it got me wondering what others use for ZFS clustering,
>> HA and sync replication. Can somebody please point me in the right
>> direction?
> 
> Architecturally, replicating in this way is a bad idea. Past efforts to do 
> block-level replication suffer from one or more of:
> 	1. coherency != consistency
> 	2. performance sux without nonvolatile write caches
> 	3. network bandwidth is not infinite
> 	4. the speed of light is too slow
> 	5. replicating big chunks of data exacerbates #3 and #4
> 
> AVS and friends worked ok for the time they were originally developed,
> when disks were 9, 18, or 36 GB. For a JBOD full of 4TB disks, it just isn't
> feasible.
> 
> In the market, where you do see successes for block-level replication, the
> systems are constrained to avoid #2 and #5 (eg TrueCopy or SRDF).
> 
> For most practical applications today, the biggest hurdle is #1, by far. 
> Fortunately, there are many solutions there: NoSQL, distributed databases,
> (HA-)IMDBs, etc.
> 
> Finally, many use cases for block-level replication are solved in the metro
> area choosing the right hardware and using mirrors thus solving #1 and KISS
> at the same time.

I understand that replication at the storage level is the Wrong Way(tm)
to do it, but I need to cover this scenario for the rare cases where the
application layer can't/won't do it themselves. Most specifically, I
need to replicate VM backing storage for VMs that can't do software
RAID-1 themselves (which is of course the best way).

In any case, I'm just looking at what's available in the market now.
Ultimate I might go for shared storage + two heads + corosync/pacemaker
(I got 'em to compile on Illumos).

Cheers,
--
Saso