Hi Nicolin, On 4/29/2025 1:32 AM, Nicolin Chen wrote: > On Mon, Apr 28, 2025 at 05:42:27PM +0530, Vasant Hegde wrote: >>> +/** >>> + * struct iommu_vcmdq_alloc - ioctl(IOMMU_VCMDQ_ALLOC) >>> + * @size: sizeof(struct iommu_vcmdq_alloc) >>> + * @flags: Must be 0 >>> + * @viommu_id: Virtual IOMMU ID to associate the virtual command queue with >>> + * @type: One of enum iommu_vcmdq_type >>> + * @index: The logical index to the virtual command queue per virtual IOMMU, for >>> + * a multi-queue model >>> + * @out_vcmdq_id: The ID of the new virtual command queue >>> + * @addr: Base address of the queue memory in the guest physical address space >> >> Sorry. I didn't get this part. >> >> So here `addr` is command queue base address like >> - NVIDIA's virtual command queue >> - AMD vIOMMU's command buffer >> >> .. and it will allocate vcmdq for each buffer type. Is that the correct >> understanding? > > Yes. For AMD "vIOMMU", it needs a new type for iommufd vIOMMU: > IOMMU_VIOMMU_TYPE_AMD_VIOMMU, > > For AMD "vIOMMU" command buffer, it needs a new type too: > IOMMU_VCMDQ_TYPE_AMD_VIOMMU, /* Kdoc it to be Command Buffer */ You are suggesting we define one type for AMD and use it for all buffers like command buffer, event log, PPR buffet etc? and use iommu_vcmdq_alloc->index to identity different buffer type? > > Then, use IOMMUFD_CMD_VIOMMU_ALLOC ioctl to allocate an vIOMMU > obj, and use IOMMUFD_CMD_VCMDQ_ALLOC ioctl(s) to allocate vCMDQ > objs. > >> In case of AMD vIOMMU, buffer base address is programmed in different register >> (ex: MMIO Offset 0008h Command Buffer Base Address Register) and buffer >> enable/disable is done via different register (ex: MMIO Offset 0018h IOMMU >> Control Register). And we need to communicate both to hypervisor. Not sure this >> API can accommodate this as addr seems to be mandatory. > > NVIDIA's CMDQV has all three of them too. What we do here is to > let VMM trap the buffer base address (in guest physical address > space) and forward it to kernel using this @addr. Then, kernel > will translate this @addr to host physical address space, and > program the physical address and size to the register. Right. For AMD IOMMU 1st 4K of MMIO space (which contains all buffer base address registers) is not accelerated. So we can trap it and pass GPA, size to iommufd. .. but programming base register (like Command buffer base addr) is not sufficient. We have to enable the command buffer by setting particular bit in Control register. So at high level flow is something like below (@Suravee, correct me if I missed something here).
