[OpenIndiana-discuss] Dead Horse Beating: zfs send doesn't await rem login

[Joshua M. Clulow]

On Fri, 22 Nov 2024 at 12:27, hput <hput3 at fastmail.fm> wrote:
> Novice question about `send recv'
> This commandline doesn't stop and wait for the login to complete or
> even start on remote (oi)  host.
>
>   # zfs send -v  p0/pub at 241122.200009-for-send-t0-oi |pv |ssh oi zfs recv -v p0/pub
>    full send of p0/pub at 241122.200009-for-send-t0-oi estimated size is
>    403M
>   total estimated size is 403M
>   (root at oi) Password: TIME        SENT   SNAPSHOT p0/pub at 241122.200009-for-send-t0-oi
>   20:13:48    228K   p0/pub at 241122.200009-for-send-t0-oi
>   20:13:49    228K   p0/pub at 241122.200009-for-send-t0-oi
>   [...]
>
> The pause at password prompt might be 1 second or a bit less.

There are a number of things going on here.  The "zfs send" command
has two output file descriptors: stdout, which you are redirecting
into the pipe, and stderr, which you are not.  When you use the "zfs
-v" command, I believe it emits regular progress reports about what
it's been able to send so far to stderr, once a second.  Because this
is not redirected anywhere it happily continues to write to the
terminal.  Note that it won't be making any forward progress, though,
because the pipe buffer between stdout of the "zfs send" process, and
stdin of the "ssh" process has already filled up with a relatively
small amount of data, so it's blocked waiting for that to make it
through to the other side.

Because you're not using SSH keys, SSH is asking you for a password to
connect to the other host.  It actually uses a wholly separate
mechanism to print that prompt and to ask for your input: it opens
"/dev/tty", which is a special virtual device that lets a program open
a new file descriptor talking to the controlling terminal of the
process.

The password prompt was emitted to your terminal, and SSH is still
waiting for you to type your password in.  It seems like the system
has moved on and isn't paused waiting anymore, but that's just because
"zfs send" is emitting output _over the top_ of everything else that
is happening, which it's able to do via the "stderr" descriptor it
has.

If you type your password and press enter, I expect the SSH connection
will then complete and data will start to flow through to the remote
site.  Otherwise, if you left it for, say, a half hour, I would expect
the "zfs send" process will continue to emit progress reports that say
nothing has moved, and SSH will continue to block waiting for a
password.

I would recommend using SSH keys, FWIW, not passwords.  They avoid
this issue, and are in general at least somewhat more secure!


Cheers.

-- 
Joshua M. Clulow
http://blog.sysmgr.org