On 7/22/2025 10:37 PM, Sean Christopherson wrote:
On Tue, Jul 22, 2025, Xiaoyao Li wrote:
On 7/21/2025 8:22 PM, Xiaoyao Li wrote:
On 7/18/2025 12:27 AM, Fuad Tabba wrote:
+/*
+ * CoCo VMs with hardware support that use guest_memfd only for
backing private
+ * memory, e.g., TDX, cannot use guest_memfd with userspace mapping
enabled.
+ */
+#define kvm_arch_supports_gmem_mmap(kvm) \
+ (IS_ENABLED(CONFIG_KVM_GMEM_SUPPORTS_MMAP) && \
+ (kvm)->arch.vm_type == KVM_X86_DEFAULT_VM)
I want to share the findings when I do the POC to enable gmem mmap in QEMU.
Actually, QEMU can use gmem with mmap support as the normal memory even
without passing the gmem fd to kvm_userspace_memory_region2.guest_memfd
on KVM_SET_USER_MEMORY_REGION2.
Since the gmem is mmapable, QEMU can pass the userspace addr got from
mmap() on gmem fd to kvm_userspace_memory_region(2).userspace_addr. It
works well for non-coco VMs on x86.
one more findings.
I tested with QEMU by creating normal (non-private) memory with mmapable
guest memfd, and enforcily passing the fd of the gmem to struct
kvm_userspace_memory_region2 when QEMU sets up memory region.
It hits the kvm_gmem_bind() error since QEMU tries to back different GPA
region with the same gmem.
So, the question is do we want to allow the multi-binding for shared-only
gmem?
Can you elaborate, maybe with code? I don't think I fully understand the setup.
well, I haven't fully sorted it out. Just share what I get so far.
the problem hit when SMM is enabled (which is enabled by default).
- The trace of "-machine q35,smm=off":
kvm_set_user_memory AddrSpace#0 Slot#0 flags=0x4 gpa=0x0 size=0x80000000
ua=0x7f5733fff000 guest_memfd=15 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#1 flags=0x4 gpa=0x100000000
size=0x80000000 ua=0x7f57b3fff000 guest_memfd=15
guest_memfd_offset=0x80000000 ret=0
kvm_set_user_memory AddrSpace#0 Slot#2 flags=0x2 gpa=0xffc00000
size=0x400000 ua=0x7f5840a00000 guest_memfd=-1 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#0 flags=0x0 gpa=0x0 size=0x0
ua=0x7f5733fff000 guest_memfd=15 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#0 flags=0x4 gpa=0x0 size=0xc0000
ua=0x7f5733fff000 guest_memfd=15 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#3 flags=0x2 gpa=0xc0000
size=0x20000 ua=0x7f5841000000 guest_memfd=-1 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#4 flags=0x2 gpa=0xe0000
size=0x20000 ua=0x7f5840de0000 guest_memfd=-1
guest_memfd_offset=0x3e0000 ret=0
kvm_set_user_memory AddrSpace#0 Slot#5 flags=0x4 gpa=0x100000
size=0x7ff00000 ua=0x7f57340ff000 guest_memfd=15
guest_memfd_offset=0x100000 ret=0
- The trace of "-machine q35"
kvm_set_user_memory AddrSpace#0 Slot#0 flags=0x4 gpa=0x0 size=0x80000000
ua=0x7f8faffff000 guest_memfd=15 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#1 flags=0x4 gpa=0x100000000
size=0x80000000 ua=0x7f902ffff000 guest_memfd=15
guest_memfd_offset=0x80000000 ret=0
kvm_set_user_memory AddrSpace#0 Slot#2 flags=0x2 gpa=0xffc00000
size=0x400000 ua=0x7f90bd000000 guest_memfd=-1 guest_memfd_offset=0x0 ret=0
kvm_set_user_memory AddrSpace#0 Slot#3 flags=0x4 gpa=0xfeda0000
size=0x20000 ua=0x7f8fb009f000 guest_memfd=15 guest_memfd_offset=0xa0000
ret=-22
qemu-system-x86_64: kvm_set_user_memory_region:
KVM_SET_USER_MEMORY_REGION2 failed, slot=3, start=0xfeda0000,
size=0x20000, flags=0x4, guest_memfd=15, guest_memfd_offset=0xa0000:
Invalid argument
kvm_set_phys_mem: error registering slot: Invalid argument
where QEMU tries to setup the memory region for [0xfeda0000, +0x20000],
which is back'ed by gmem (fd is 15) allocated for normal RAM, from
offset 0xa0000.
What I have tracked down in QEMU is mch_realize(), where it sets up some
memory region starting from 0xfeda0000.
If you want to reproduce it yourself, here is my QEMU branch
https://github.com/intel-staging/qemu-tdx.git lxy/gmem-mmap-poc
To boot a VM with guest memfd:
-object memory-backend-guest-memfd,id=gmem0,size=$mem
-machine memory-backend=gmem0
