Disclaimer: haven't been tested well enough yet and needs more beating
For past couple of months David Wei, Vishal Verma and other folks
have been mentioning that it'd be great to have dmabuf support for
read/write and other operations in io_uring. The topic is not new,
it has been discussed many times in different contexts including
networking. The last relevant attempt was premapped dma tags by
Keith [1], and this patch set took a lot from it.
This series implements it for read/write io_uring requests. The uAPI
looks similar to normal registered buffers, the user will need to
register a dmabuf in io_uring first and then use it as any other
registered buffer. On registration the user also specifies a file
to map the dmabuf for.
// register
io_uring_update_buffers(ring, { dma_buf_fd, target_file_fd });
// use
reg_buf_idx = 0;
io_uring_prep_read_fixed(sqe, target_file_fd, buffer_offset,
buffer_size, file_offset, reg_buf_idx);
It's an RFC to discuss the overall direction. The series misses
parts like bio splitting and nvme sgl support, and otherwise
there are some rough edges and probably problems, which will
need more testing and attention.
[1] https://lore.kernel.org/io-uring/20220805162444.3985535-1-kbusch@xxxxxx/
simple liburing based example:
git: https://github.com/isilence/liburing.git dmabuf-rw
link: https://github.com/isilence/liburing/tree/dmabuf-rw
kernel branch:
git: https://github.com/isilence/linux.git dmabuf-rw-v1
Pavel Begunkov (12):
file: add callback returning dev for dma operations
iov_iter: introduce iter type for pre-registered dma
block: move around bio flagging helpers
block: introduce dmavec bio type
block: implement ->get_dma_device callback
nvme-pci: add support for user passed dma vectors
io_uring/rsrc: extended reg buffer registration
io_uring: add basic dmabuf helpers
io_uring/rsrc: add imu flags
io_uring/rsrc: add dmabuf-backed buffer registeration
io_uring/rsrc: implement dmabuf regbuf import
io_uring/rw: enable dma registered buffers
block/bdev.c | 11 ++
block/bio.c | 21 ++++
block/blk-merge.c | 32 +++++
block/blk.h | 2 +-
block/fops.c | 3 +
drivers/nvme/host/pci.c | 158 +++++++++++++++++++++++
include/linux/bio.h | 59 ++++++---
include/linux/blk-mq.h | 2 +
include/linux/blk_types.h | 6 +-
include/linux/blkdev.h | 2 +
include/linux/fs.h | 2 +
include/linux/uio.h | 14 +++
include/uapi/linux/io_uring.h | 13 +-
io_uring/Makefile | 1 +
io_uring/dmabuf.c | 60 +++++++++
io_uring/dmabuf.h | 34 +++++
io_uring/rsrc.c | 230 ++++++++++++++++++++++++++++++----
io_uring/rsrc.h | 23 +++-
io_uring/rw.c | 7 +-
lib/iov_iter.c | 70 ++++++++++-
20 files changed, 701 insertions(+), 49 deletions(-)
create mode 100644 io_uring/dmabuf.c
create mode 100644 io_uring/dmabuf.h
--
2.49.0
