On Tue, Jul 01, 2025 at 04:01:34PM -0700, Nicolin Chen wrote: > On Tue, Jul 01, 2025 at 10:51:20PM +0000, Pranjal Shrivastava wrote: > > On Tue, Jul 01, 2025 at 03:07:57PM -0700, Nicolin Chen wrote: > > > On Tue, Jul 01, 2025 at 08:43:30PM +0000, Pranjal Shrivastava wrote: > > > > On Tue, Jul 01, 2025 at 01:23:17PM -0700, Nicolin Chen wrote: > > > > > Or perhaps calling them "non-accelerated commands" would be nicer. > > > > > > > > Uhh okay, so there'll be a separate driver in the VM issuing invalidation > > > > commands directly to the CMDQV thus we don't see any of it's part here? > > > > > > That's how it works. VM must run a guest-level VCMDQ driver that > > > separates accelerated and non-accelerated commands as it already > > > does: > > > > > > accelerated commands => VCMDQ (HW) > > > non-accelerated commands => SMMU CMDQ (SW) =iommufd=> SMMU CMDQ (HW) > > > > > > > Right exactly what got me confused. I was assuming the same CMDQV driver > > would run in the Guest kernel but seems like there's another driver for > > the Guest that's not in tree yet or maybe is a purely user-space thing? > > It's the same tegra241-cmdqv.c in the kernel, which is already > a part of mainline Linux. Both host and guest run the same copy > of software. The host kernel just has the user VINTF part (via > iommufd) additional to what the guest already has. > > > And the weird part was that "invalidation" commands are accelerated but > > we use the .cache_invalidate viommu op for `non-invalidation` commands. > > But I guess what you meant there could be non-accelerated invalidation > > commands (maybe something stage 2 TLBIs?) which would go through the > > .cache_invalidate op, right? > > I am talking about this: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c?h=v6.16-rc4#n305 > > Those commands returned "false" will be issued to smmu->cmdq in a > guest VM, which will be trapped by VMM as a standard SMMU nesting > and will be further forwarded via iommufd to the host kernel that > will invoke this cache_invalidate op in the arm-smmu-v3 driver. > > Those commands returned "true" will be issued to vcmdq->cmdq that > is HW-accelerated queue (setup by VMM via iommufd's hw_queue/mmap). Right, this brings me back to my original understanding, the arm-smmu-v3 driver checks for "supported commands" and figures out which queue shall they be issued to.. now there are commands which are "non-invalidation" commands which are non-acclerated like CMD_PRI_RESP, which would be issued through the trap => .cache_invalidate path. Thus, coming back to the two initial points: 1) Issuing "non-invalidation" commands through .cache_invalidate could be confusing, I'm not asking to change the op name here, but if we plan to label it, let's label them as "Trapped commands" OR "non-accelerated" commands as you suggested. 2) The "FIXME" confusion: The comment in arm_vsmmu_cache_invalidate mentions we'd like to "fix" the issuing of commands through the main cmdq and instead like to group by "type", if that "type" is the queue type (which I assume it is because IOMMU_TYPE has to be arm-smmu-v3), what do we plan to do differently there, given that the op is only for trapped commands *have* to go through the main CMDQ? If we were planning to do something based on the queue type, I was hoping for it to be addressed in this series as we've introduced the Tegra CMDQ type. That's all I wanted to say, sorry for if this was confusing. > > Nicolin Thanks, Praan
