> On May 12, 2025, at 5:19 AM, Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: > On Fri, May 09, 2025 at 07:29:04PM +0000, Matt Ochs wrote: >> >> Would it make sense to just use the existing pcihole64 since [I think] >> it more or less represents the same concept (setting 64bit MMIO window)? > > I'm not sure. I've been struggling to reproduce an effect wit hseting > the existing -global q35-pcihost.pci-hole64-size=1048576K settings > on x86, and also wondering how it interacts with the previously > mentioned -fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=262144 > > Possibly the former only works with SeaBIOS, and the latter only > works with EDK2, but I've not figured out how to prove this. The qemu docs mention opt/ovmf is specifically for OVMF firmware: https://github.com/qemu/qemu/blob/7be29f2f1a3f5b037d27eedbd5df9f441e8c8c16/docs/specs/fw_cfg.rst#L279 The pcihole64 setting can be used with OVMF (see below) and with SEABIOS: https://github.com/libvirt/libvirt/blob/master/docs/formatdomain.rst (see pcihole64) The X-PciMmio64Mb parameter isn't directly supported in libvirt IIUC. The libvirt XML would need to directly pass qemu command line arguments to use it. > > I'm curious if there's a good way to identify the guest memory > map impact, as I'm not finding a clear marker in 'dmesg' that > correlates ? We were able to test this by using OVMF without the dynamic mmio window size patch (i.e. a version older than edk2-stable202211) and guest kernel parameters that are not set to allow re-calculating the MMIO window size by deferring guest resource allocations to the guest kernel (i.e. pci=realloc and pci=nocrs aren't set). With this we could reproduce a 4 GPU VM launch with guest BARs not mapped properly due to running out of space/resources. The BAR mapping failures will be clear in dmesg, with no BAR region mappings in /proc/iomem or output of lspci for the GPUs.
