Jason, On 4/24/2025 6:01 PM, Jason Gunthorpe wrote: > On Thu, Apr 24, 2025 at 12:08:56PM +0530, Vasant Hegde wrote: > >>> What the iommu driver should do when set_dev_pasid is called for a non- >>> PASID device? > > That's a good point, maybe the core code should filter that out based > on max_pasids? I think we do run into trouble here because the drivers > are allocating PASID table space based on max_pasids so the non-pasid > device should fail to add the pasid. Tushar, you should have hit this > in your testing??? > > We also have a problem setting up the default domain - it won't > compute IOMMU_HWPT_ALLOC_PASID properly across the group. If the > no-pasid device probes first then PASID will be broken on the group. > > Tushar isn't hitting this because ARM always uses a PASID compatible > domain today, but it will not work on AMD. > > That's a huge pain to deal with :\ Agree. That will complicate things. Just to be clear, I gave some of the AMD GPU as an example of group where we have both PASID, non-PASID devices in same group. But currently AMDGPU is not using PASID. But currently I am not looking for supporting SVA for amdgpu with such configs. > >> Per device max_pasids check should cover that right? > > The driver shouldn't be doing this though, if the driver is told to > make a pasid then it should make a pasid.. The driver can fail > attaching a pasid to a device that is over the device's max_pasid. > >> FYI. One example of such device is some of the AMD GPUs which has >> both VGA and audio in same group. while VGA supports PASID, audio is >> not. This used to work fine when we had AMD IOMMU PASID specific >> driver. GPUs stopped using PASIDs in upstream kernel. So I didn't >> look into this part in details. > > Uhhh.. That sounds like a worse problem, the only way you should end > up with same group is if the ACS flags are missing on the GPU so Linux > assumes the VGA and audio can loopback to each other internally. > > That should completely block PASID support on the GPU side due the > wrong routing. We can't have a hole in the PASID address space where > the audio BAR is. > > I suppose the HW doesn't actually behave this way but since it doesn't > have the right ACS flags the SW doesn't know? Guessing.. Honestly I have no idea. Since they had stopped using PASID support I never digged into the details! -Vasant
