Re: Support for transferring sparse files via scp/sftp correctly?

[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

 



On Sat, 29 Mar 2025 at 16:14, Ron Frederick <ronf@xxxxxxxxxxxxx> wrote:

> [...]
> If you don’t get all of the requested ranges in a single request,
> additional requests can be sent starting at just past the end of the last
> range previously returned.
>
> What do you think?
>

That seems like it'd work well for things with SEEK_HOLE or equivalent,
although there's always the chance of the underlying file changing between
mapping it out and doing the transfer.

Damien pointed out that it's possible to do a reasonable but not perfect
sparse file support by memcmp'ing your existing file buffer with a block of
zeros and skipping the write if it matches.  OpenBSD's cp(1) does this
(look for "skipholes"):
https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/bin/cp/utils.c?annotate=HEAD.

This seems surprisingly effective in the case where you already have the
file content in a buffer anyway, but it would be harder to do (or at least
more expensive) as part of a separate request type that returns the
ranges.  It'd be easier to implement if there was some kind of
"read-sparse" operation that could return a list of {offset, len, data}
instead of just the offsets and lengths.  This would reduce the time
between the sparse check and the read although it's still potentially racy.

-- 
Darren Tucker (dtucker at dtucker.net)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@xxxxxxxxxxx
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev




[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux