On 7/2/2025 8:20 PM, Bagas Sanjaya wrote: > On Wed, Jul 02, 2025 at 08:00:41PM +0900, Alexandre Courbot wrote: >> diff --git a/Documentation/gpu/nova/core/vbios.rst b/Documentation/gpu/nova/core/vbios.rst >> new file mode 100644 >> index 0000000000000000000000000000000000000000..55d7dd4a6658c2a20cc5617f96b278bc4ec2ba17 >> --- /dev/null >> +++ b/Documentation/gpu/nova/core/vbios.rst >> @@ -0,0 +1,180 @@ >> +.. SPDX-License-Identifier: (GPL-2.0+ OR MIT) >> + >> +========== >> +VBIOS >> +========== >> +This document describes the layout of the VBIOS image which is a series of concatenated >> +images in the ROM of the GPU. The VBIOS is mirrored onto the BAR 0 space and is read >> +by both Boot ROM firmware (also known as IFR or init-from-rom firmware) on the GPU to >> +boot strap various microcontrollers (PMU, SEC, GSP) with critical initialization before > > bootstrap > >> +the driver loads, as well as by the nova-core driver in the kernel to boot the GSP. >> + >> +The format of the images in the ROM follow the "BIOS Specification" part of the >> +PCI specification, with Nvidia-specific extensions. The ROM images of type FwSec >> +are the ones that contain Falcon ucode and what we are mainly looking for. >> + >> +As an example, the following are the different image types that can be found in the >> +VBIOS of an Ampere GA102 GPU which is supported by the nova-core driver. >> + >> +- PciAt Image (Type 0x00) - This is the standard PCI BIOS image, whose name >> + likely comes from the "IBM PC/AT" architecture. >> + >> +- EFI Image (Type 0x03) - This is the EFI BIOS image. It contains the UEFI GOP >> + driver that is used to display UEFI graphics output. >> + >> +- First FwSec Image (Type 0xE0) - The first FwSec image (Secure Firmware) >> + >> +- Second FwSec Image (Type 0xE0) - The second FwSec image (Secure Firmware) >> + contains various different microcodes (also known as an applications) that do > > various microcodes? The above 2 fixups sounds good to me, I'm guessing Danilo if you're pulling this then maybe apply the fixups directly to the patch since it is trivial? > >> + >> +.. note:: >> + This diagram is created based on an GA-102 Ampere GPU as an example and could >> + vary for future or other GPUs. >> + >> +Here is a block diagram of the VBIOS layout:: > > Above sentence (but not the note directive) is redundant, though. This suggestion I'd skip. But thanks for all the review! - Joel > >> + >> + +----------------------------------------------------------------------------+ >> + | VBIOS (Starting at ROM_OFFSET: 0x300000) | >> + +----------------------------------------------------------------------------+ >> + | +-----------------------------------------------+ | >> + | | PciAt Image (Type 0x00) | | >> + | +-----------------------------------------------+ | >> + | | +-------------------+ | | >> + | | | ROM Header | | | >> + | | | (Signature 0xAA55)| | | >> + | | +-------------------+ | | >> + | | | rom header's pci_data_struct_offset | | >> + | | | points to the PCIR structure | | >> + | | V | | >> + | | +-------------------+ | | >> + | | | PCIR Structure | | | >> + | | | (Signature "PCIR")| | | >> + | | | last_image: 0x80 | | | >> + | | | image_len: size | | | >> + | | | in 512-byte units | | | >> + | | +-------------------+ | | >> + | | | | | >> + | | | NPDE immediately follows PCIR | | >> + | | V | | >> + | | +-------------------+ | | >> + | | | NPDE Structure | | | >> + | | | (Signature "NPDE")| | | >> + | | | last_image: 0x00 | | | >> + | | +-------------------+ | | >> + | | | | >> + | | +-------------------+ | | >> + | | | BIT Header | (Signature scanning | | >> + | | | (Signature "BIT") | provides the location | | >> + | | +-------------------+ of the BIT table) | | >> + | | | header is | | >> + | | | followed by a table of tokens | | >> + | | V one of which is for falcon data. | | >> + | | +-------------------+ | | >> + | | | BIT Tokens | | | >> + | | | ______________ | | | >> + | | | | Falcon Data | | | | >> + | | | | Token (0x70)|---+------------>------------+--+ | >> + | | | +-------------+ | falcon_data_ptr() | | | >> + | | +-------------------+ | V | >> + | +-----------------------------------------------+ | | >> + | (no gap between images) | | >> + | +-----------------------------------------------+ | | >> + | | EFI Image (Type 0x03) | | | >> + | +-----------------------------------------------+ | | >> + | | Contains the UEFI GOP driver (Graphics Output)| | | >> + | | +-------------------+ | | | >> + | | | ROM Header | | | | >> + | | +-------------------+ | | | >> + | | | PCIR Structure | | | | >> + | | +-------------------+ | | | >> + | | | NPDE Structure | | | | >> + | | +-------------------+ | | | >> + | | | Image data | | | | >> + | | +-------------------+ | | | >> + | +-----------------------------------------------+ | | >> + | (no gap between images) | | >> + | +-----------------------------------------------+ | | >> + | | First FwSec Image (Type 0xE0) | | | >> + | +-----------------------------------------------+ | | >> + | | +-------------------+ | | | >> + | | | ROM Header | | | | >> + | | +-------------------+ | | | >> + | | | PCIR Structure | | | | >> + | | +-------------------+ | | | >> + | | | NPDE Structure | | | | >> + | | +-------------------+ | | | >> + | | | Image data | | | | >> + | | +-------------------+ | | | >> + | +-----------------------------------------------+ | | >> + | (no gap between images) | | >> + | +-----------------------------------------------+ | | >> + | | Second FwSec Image (Type 0xE0) | | | >> + | +-----------------------------------------------+ | | >> + | | +-------------------+ | | | >> + | | | ROM Header | | | | >> + | | +-------------------+ | | | >> + | | | PCIR Structure | | | | >> + | | +-------------------+ | | | >> + | | | NPDE Structure | | | | >> + | | +-------------------+ | | | >> + | | | | | >> + | | +-------------------+ | | | >> + | | | PMU Lookup Table | <- falcon_data_offset <----+ | >> + | | | +-------------+ | pmu_lookup_table | | >> + | | | | Entry 0x85 | | | | >> + | | | | FWSEC_PROD | | | | >> + | | | +-------------+ | | | >> + | | +-------------------+ | | >> + | | | | | >> + | | | points to | | >> + | | V | | >> + | | +-------------------+ | | >> + | | | FalconUCodeDescV3 | <- falcon_ucode_offset | | >> + | | | (FWSEC Firmware) | fwsec_header() | | >> + | | +-------------------+ | | >> + | | | immediately followed by... | | >> + | | V | | >> + | | +----------------------------+ | | >> + | | | Signatures + FWSEC Ucode | | | >> + | | | fwsec_sigs(), fwsec_ucode()| | | >> + | | +----------------------------+ | | >> + | +-----------------------------------------------+ | >> + | | >> + +----------------------------------------------------------------------------+ >> + >> +Falcon data Lookup >> +------------------ >> +A key part of the VBIOS extraction code (vbios.rs) is to find the location of the >> +Falcon data in the VBIOS which contains the PMU lookup table. This lookup table is >> +used to find the required Falcon ucode based on an application ID. >> + >> +The location of the PMU lookup table is found by scanning the BIT (`BIOS Information Table`_) >> +tokens for a token with the id `BIT_TOKEN_ID_FALCON_DATA` (0x70) which indicates the >> +offset of the same from the start of the VBIOS image. Unfortunately, the offset >> +does not account for the EFI image located between the PciAt and FwSec images. >> +The `vbios.rs` code compensates for this with appropriate arithmetic. >> + >> +.. _`BIOS Information Table`: https://download.nvidia.com/open-gpu-doc/BIOS-Information-Table/1/BIOS-Information-Table.html > > The rest looks good. > > Thanks. >
