On Mon, Jun 23, 2025, Tom Lendacky wrote:
> On 6/20/25 18:13, Sean Christopherson wrote:
> > On Wed, Mar 26, 2025, Yosry Ahmed wrote:
> > The more I think about all of this, the less it makes sense. The *entire* point
> > of an ASID is to tag TLB entries so that a flush isn't required when running code
> > for the same address space.
> >
> > The main problem I'm struggling with is that, as usual, the APM doesn't properly
> > document anything, and just gives "suggestions" for the VMM. *sigh*
> >
> > As I read it, these snippets from the APM are saying ASIDs tag only GPA=>PA entries
> > when NPT is in use.
> >
> > TLB entries are tagged with Address Space Identifier (ASID) bits to distinguish
> > different guest virtual address spaces when shadow page tables are used, or
> > different guest physical address spaces when nested page tables are used. The
> > VMM can choose a software strategy in which it keeps multiple shadow page tables,
> > and/or multiple nested page tables in processors that support nested paging,
> > up-to-date; the VMM can allocate a different ASID for each shadow or nested
> > page table. This allows switching to a new process in a guest under shadow
> > paging (changing CR3 contents), or to a new guest under nested paging (changing
> > nCR3 contents), without flushing the TLBs.
> >
> > Note that because an ASID is associated with the guest's physical address
> > space, it is common across all of the guest's virtual address spaces within a
> > processor. This differs from shadow page tables where ASIDs tag individual
> > guest virtual address spaces. Note also that the same ASID may or may not be
> > associated with the same address space across all processors in a
> > multiprocessor system, for either nested tables or shadow tables; this depends
> > on how the VMM manages ASID assignment.
> >
> > But then the "15.16.1 TLB Flush" section says this, without any qualification
> > whatsoever that it applies only to shadow paging.
> >
> > A MOV-to-CR3, a task switch that changes CR3, or clearing or setting CR0.PG or
> > bits PGE, PAE, PSE of CR4 affects only the TLB entries belonging to the current
> > ASID, regardless of whether the operation occurred in host or guest mode. The
> > current ASID is 0 when the CPU is not inside a guest context.
> >
> > And honestly, tagging only GPA=>PA entries doesn't make any sense, because
> > GVA=>GPA needs to be tagged with *something*. And the APM doesn't say anything
> > about caching GPA=>PA translations, only about caching VA=>PA.
>
> VA=>PA translations are always tagged with a TLB tag value. Outside of
> SEV-SNP, the TLB tag value is ASID.
>
> So for those guests, VA=>PA translation are tagged with the ASID. For
> SEV-SNP guests, see below.
>
> >
> > The thing that doesn't fit is that SEV+ uses ASIDs on a per-VM basis. I suggested
> > per-VM ASIDs for all VM types based solely on that fact, but now I'm wondering if
> > it's SEV+ that crazy and broken. Because if ASIDs also tag GVA=>GPA, then SEV has
> > a massive architectural security hole, e.g. a malicious hypervisor can coerce the
> > CPU into using a stale GVA=>GPA TLB entry by switching vCPUs and letting guest
> > process with CR3=x access memory for guest process with CR3=y. But again, if
> > ASIDs don't tag GVA=>GPA, then what provides isolation between vCPUs!?!?!
>
> No.
>
> For SEV/SEV-ES guests, the HV (which remains partially trusted) must do a
> TLB flush before running a different VMCB of the same guest, in order to
> avoid this problem. This code is in pre_sev_run().
>
> For SEV-SNP guests, this is handled automatically by hardware through the
> PCPU_ID and TLB_ID VMSA fields (documented somewhat in APM 15.36.15).
Aha! I knew I had to be missing something. Rule #1: don't doubt Kaplan ;-)
> In short, the TLB is tagged with {TLB_ID, ASID} and TLB_ID is managed by
> HW and guaranteed to be different for each vCPU of the guest running on a
> physical core. This ensures that the TLB tag is unique for each guest and
> for each vCPU of the guest.
Thanks Tom, very much appreciated!
