On Wed, Jul 16, 2025 at 01:31:34PM -0300, Jason Gunthorpe wrote: > On Wed, Jul 16, 2025 at 11:41:12PM +0800, Xu Yilun wrote: > > > Except it doesn't work like that for MMIO. Shared/private is a TDI > > > operation only and effects the whole device. We shouldn't split it > > > into two actions. > > > > OK. IIUC you want 1 uAPI, TSM Bind, to finish all secure configuration, > > including MMIO sharebility. I think it is possible, the MMIO shareability > > is fixed after TSM Bind. iommufd could fetch TDI report to get the > > private/shared MMIO ranges and callback to VFIO. > > Am I wrong? Isn't this one action? Once you do bind you MUST have no You are right and I'm good to this one operation. > hypervisor mapping or x86 will explode. And once bind completes the > hypervisor mapping is unusable on all other arches. > > That's one operation as far as I can tell?? > > > > I also don't think it needs to be strictly 'in-place' as we expect the > > > > When I said "must be in-place", I mean the MMIO resource (hpa) for one gfn is > > fixed, can't have 2 copies of backend as the current private/shared > > memory does. > > Ok, I would not call that in place.. > > > > VM to be idle on the MMIO during this change over. Faulting would be OK. > > > > Sorry, I don't get your point about 'strictly in-place' here? > > Here I mean the iommu page table would have to atomically change "in > place" from shared to private in a way that that is hitless to the > guest. We don't need this, IMHO. I agree. > > > > Any case where we need to get back to vfio for something needs to be > > > managed with a callback of some kind. We need to get a list of what > > > those things are. > > > > > > What do all the arches need here? > > > - ARM I suspect has the TDI locking operation install the MMIO in the > > > secure S2, not KVM? > > > - AMD just leaves the MMIO mapped all the time? > > > - x86 presumably needs to carefully map/unmap to KVM and iommu in the > > > right sequence or you get a MCE? > > > > Yeah, for Intel TDX, basically 2 things, zap the mapping on opposite side > > page table, mark the correct shareability for correct fault in. > > I expect userspace to be doing this, which is why I asked about two DMABUFs.. > > > > So what is the plan? You want to wrap this in DMABUF, but will there > > > be two DMABUFS, one for secure and one for non-secure? Is userspace > > > > No I don't expect 2 dmabufs. I image shareability could be an physical > > attribute of dmabuf and the callback to VFIO changes the shareability. > > And VFIO, the dmabuf exporter, could notify KVM/iommufd about the > > shareability change. Then KVM/iommufd unmaps their page tables. > > How can this work exactly? > > Does Intel put shared/private MMIO at the same guest address when it > changes around? I understand other arches do not do this. No, The Shared bit is embedded in GPA, it toggles for shared/private change. > > Even if you do, the owning page table changes, right? In public mode > it has to be mapped into an ioas and in private mode there is no ioas > mapping? Yes. > > Seems to me that two DMABUFs is easier than trying to teach DMABUf > about some new attribute.. > > Userpsace can unmap all the shared DMABUFs, do a TDI BIND, then map > private DMABUFs. DMABUFs do not change from private to shared on the > fly. But the shareability of each MMIO pfn should still be recorded at the time of TDI BIND. Shareability is not only about hypervisor can map the MMIO or not, it is mainly about Guest should access it in a shared or private manner. According to this KVM map the pfn in EPT or S-EPT. For MMIO, the shareability layout is the TDI's inherence which can be get from TDI report. I.e. some MMIOs must still be accessed as shared by guest even if the device is converted to private. So I think a private DMABUF without sharebility layout can't support the TDI case. The existing shareability layout for all guest memory space is recorded in KVM via KVM_SET_MEMORY_ATTRIBUTES, but now Sean's suggestion is giving it back to resource owners when in-place conversion makes guest_memfd fully aware of the shareability layout. Thanks, Yilun > > Invalidation is only an error case situation to revoke if userspace > didn't do the above sequence right. > > Is that reasonable??? > > Jason
