On Tue, 4 Mar 2025 at 21:22, Chris Rapier <rapier@xxxxxxx> wrote: > > > > On 3/4/25 05:34, Philipp Marek via openssh-unix-dev wrote: > >> Does OpenSSH scp/sftp mode transfer sparse files correctly, i.e. are > >> holes skipped and not transferred as chunks of 0 bytes? [1] > >> > >> We're asking about sparse files in the >= 1PB range, which consists of > >> multi-TB holes with around 600-2000GB of valid data. > > > > > > Perhaps rsync would be a good fit here, > > it supports --sparse. > > _______________________________________________ > > openssh-unix-dev mailing list > > openssh-unix-dev@xxxxxxxxxxx > > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev > > I think one of the issues you are going to face is that SEEK_DATA and > SEEK_HOLE don't seem to be currently supported under OpenBSD. Since > that's the home OS for OpenSSH this could create portability issues. > While you can get around that with the judicious use of defines it means > that the feature set will start to shift between different OSes. OpenBSD unfortunately does not implement so many other APIs. But other OS do implement SEEK_DATA+SEEK_HOLE, including FreeBSD, Linux, Solaris, Illumos and even Cygwin. Even NFS has a SEEK to lookup holes and data sections in files. SEEK_HOLE+SEEK_DATA are also now part of the POSIX standard, so IMO it is time to face the bug that sparse files are not handled correctly and fix it > > Personally I think it's a good idea and I may explore it for HPN-SSH but > I think it's going to be a hard sell for the OpenBSD community. Selling points: - Users need it - Sparse files were always part of UNIX - It's part of the POSIX standard: https://pubs.opengroup.org/onlinepubs/9799919799/functions/lseek.html Ced -- Cedric Blancher <cedric.blancher@xxxxxxxxx> [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur _______________________________________________ openssh-unix-dev mailing list openssh-unix-dev@xxxxxxxxxxx https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
