This RFC series proposes the implementation of a new mechanism for
region mmap attributes using maple trees, based on Jason's suggested
maple tree and offset cookie approach[0]. The primary motivation is to
enable userspace applications to specify mmap attributes—such as Write
Combining (WC)—prior to invoking mmap on a VFIO region. While the
initial focus is on WC support, this framework can be extended to
support additional attributes (e.g., cachable) in the future.
Core concept is: a maple_tree instance is introduced per file
descriptor within vfio_device_file, allowing per-request ownership and
control of mmap attributes. Via new VFIO device operations (ioctl &
mmap), each vfio device populates its maple_tree, primarily during the
DEVICE_GET_REGION_INFO ioctl. The kernel returns a unique offset key
to userspace; userspace can then pass this offset to mmap, at which
point the kernel retrieves the correct maple_tree entry and invokes
the new mmap op on the vfio device to map the region with the desired
attributes.
This model also enables a new UAPI for userspace to set attributes on
a given mmap offset, allowing flexibility and room for future feature
expansion.
Because these changes alter both internal region offset handling and
the ioctl/mmap interfaces, a staged approach is necessary to manage
the large scope of the update.
This RFC implements:
- Integration of the maple_tree mechanism and new VFIO ops, along
with required helpers.
- Initial onboard support for vfio-pci.
- Introduction of the new UAPI supporting WC.
Planned follow-up work:
- Extending new ops support to all vfio-pci devices.
- Updating usages of VFIO_PCI_OFFSET_TO_INDEX and VFIO_PCI_INDEX_TO_OFFSET.
- Migrating additional VFIO devices to the new ops.
- Fully removing legacy ioctl and mmap ops, renaming the new ops
in their place once migration is complete.
For now, legacy and new VFIO ops coexist. Legacy ops will be removed
following full migration across all relevant devices.
This RFC marks the start of this transition. I am seeking feedback on
the core implementation to ensure the direction and design are correct
before proceeding with further conversion and cleanup work. Thank you
for your review and guidance.
[0]: https://lore.kernel.org/kvm/20240801141914.GC3030761@xxxxxxxx/
references:
- https://lore.kernel.org/kvm/20240731155352.3973857-1-kbusch@xxxxxxxx/T/#u
- https://lore.kernel.org/kvm/lrkyq4ivccb6x.fsf@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/
Mahmoud Adam (9):
vfio: add mmap maple tree to vfio
vfio: add transient ops to support vfio mmap mt
vfio-pci-core: rename vm operations
vfio-pci-core: remove redundant offset calculations
vfio-pci-core: add vfio_pci_mmap & helpers
vfio-pci-core: support the new vfio ops
vfio-pci: use the new vfio ops
vfio: UAPI for setting mmap attributes
vfio_pci_core: support mmap attrs uapi & WC
drivers/vfio/pci/vfio_pci.c | 4 +-
drivers/vfio/pci/vfio_pci_core.c | 165 +++++++++++++++++++++++++++----
drivers/vfio/vfio.h | 1 +
drivers/vfio/vfio_main.c | 42 ++++++++
include/linux/vfio.h | 22 +++++
include/linux/vfio_pci_core.h | 14 +++
include/uapi/linux/vfio.h | 19 ++++
7 files changed, 248 insertions(+), 19 deletions(-)
--
2.47.3
thanks,
MNAdam
Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
