[OpenIndiana-discuss] Comparison of mbuffer/tamp to ssh (A novice attempt)

[Timothy Coalson]

On Fri, Mar 31, 2017 at 7:29 AM, Harry Putnam <reader at newsguy.com> wrote:

>
> Sending end (29.7 GB):
>
> root # time zfs send p0/tst.2/isos at 170330 | tamp | mbuffer -s 128k -m
> 1000m -O oi0:31337
>

tamp is compression, which takes cpu time.  Since your network is gigabit
and you are running substantially below that, you would probably get better
speed without tamp (especially since the data isn't very compressible).


> Using ssh
>
> [...]
>
> root # time zfs send -v p1/tst.2/isos at 170330|ssh oit zfs recv -vFd p0
>

You have no buffering on this, which is a large disadvantage, likely
offsetting the removal of compression from the workload.  Add mbuffers of
the same size as your other test (but without -l and -O, obviously) on each
side of the ssh in order to do a fair comparison.


> Neither zfs fs is using compression so not sure why the reported
> difference in size of data. 29.7 send end, 28.7 recv end.
>

You mean 27.9 sent.  This is because the first sender is reporting the
amount sent through mbuffer *after* tamp compression, not the logical size
of the stream itself (mbuffer is reporting on the send side, zfs send is
quiet).  Both cases show "received" messages that show the same size
(because those come from zfs).


> You can see that the mbuffer/tramp transfer was 18 minutes quicker.
> So, with a much bigger batch of data, say 500 GB, the difference would
> be a very lot.
>
> As an idea, take it times 30.  That makes 861GB to transfer ... it would
> mean a saving of 9 hrs over using ssh.
>

I think tamp and lack of buffering on your ssh test are doing you a massive
disservice.  Here is an excerpt from my most recent backup, with mbuffers
on ssh with arcfour (of which I believe the mbuffers are the main speedup
over plain ssh with default cipher):

received 11.0GB stream in 112 seconds (101MB/sec)

However, I am doing this on bare metal, not in VMs, so the network
interface behavior could be could also be impairing your test.

Or 25.8 hrs as against 34.725 hrs.
> Of course these are only very loose figures because they do not take
> into account what the network traffic might be like.
> For the duration of this test net traffic other than this tranfer,
> should have been very light.
>
>
> _______________________________________________
> openindiana-discuss mailing list
> openindiana-discuss at openindiana.org
> https://openindiana.org/mailman/listinfo/openindiana-discuss
>