Skip to content

Define a prefetchable memory window and allocate virtio BARs from mmio32 instead of mmio64 - #6163

Open
ilstam wants to merge 3 commits into
firecracker-microvm:mainfrom
ilstam:non-prefetchable
Open

Define a prefetchable memory window and allocate virtio BARs from mmio32 instead of mmio64#6163
ilstam wants to merge 3 commits into
firecracker-microvm:mainfrom
ilstam:non-prefetchable

Conversation

@ilstam

@ilstam ilstam commented Aug 28, 2026

Copy link
Copy Markdown
Contributor
The virtio-pci capability BAR is a non-prefetchable BAR currently
allocated from the [256 GiB, 512 GiB) range managed by the mmio64
allocator.

Future patches will add PCIe root ports to the bus topology. Each root
port defines its own prefetchable and non-prefetchable memory windows
and all resources behind a root port must be within that window. The
non-prefetchable memory registers of root ports can only describe memory
below 4GiB (there is no restriction for prefetchable memory). Therefore
non-prefetchable BARs must be allocated from the mmio32 allocator.

Use the mmio32 allocator instead of the mmio64 allocator for virtio-pci
non-prefetchable BARs.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkbuild --all to verify that the PR passes
    build checks on all supported architectures.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

ilstam added 3 commits August 28, 2026 15:35
The virtio-pci capability BAR is a non-prefetchable BAR currently
allocated from the [256 GiB, 512 GiB) range managed by the mmio64
allocator.

Future patches will add PCIe root ports to the bus topology. Each root
port defines its own prefetchable and non-prefetchable memory windows
and all resources behind a root port must be within that window. The
non-prefetchable memory registers of root ports can only describe memory
below 4GiB (there is no restriction for prefetchable memory). Therefore
non-prefetchable BARs must be allocated from the mmio32 allocator.

Use the mmio32 allocator instead of the mmio64 allocator for virtio-pci
non-prefetchable BARs.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
Firecracker advertises two host bridge memory windows: a 32-bit one
below 4 GiB and a 64-bit one at 256 GiB. Both are currently advertised
as non-prefetchable.

In future commits we will add support for pass-through PCIe devices with
prefetchable BARs. Mark the 64-bit MMIO window as prefetchable in
preparation for that.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
PciRoot models the host bridge that sits at 00:00.0. Rename it to
PciHostBridge so it is not confused with the PCIe root ports added in
subsequent patches.

While at it rename PciRootError to PciBusError too. Its variants are
about allocating device slots on a bus, not about the host bridge, and
"root" would be ambiguous once root ports exist.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.61538% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.06%. Comparing base (cc535f0) to head (2b1e34b).

Files with missing lines Patch % Lines
src/vmm/src/pci/bus.rs 75.00% 2 Missing ⚠️
src/vmm/src/devices/pci/pci_segment.rs 80.00% 1 Missing ⚠️
src/vmm/src/devices/virtio/transport/pci/device.rs 85.71% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6163   +/-   ##
=======================================
  Coverage   83.06%   83.06%           
=======================================
  Files         277      277           
  Lines       31019    31019           
=======================================
  Hits        25767    25767           
  Misses       5252     5252           
Flag Coverage Δ
5.10-m5n.metal 83.34% <81.81%> (-0.01%) ⬇️
5.10-m6a.metal 82.71% <81.81%> (+<0.01%) ⬆️
5.10-m6g.metal 80.18% <87.50%> (ø)
5.10-m6i.metal 83.33% <81.81%> (-0.01%) ⬇️
5.10-m7a.metal-48xl 82.70% <81.81%> (+<0.01%) ⬆️
5.10-m7g.metal 80.18% <87.50%> (ø)
5.10-m7i.metal-24xl 83.31% <81.81%> (-0.01%) ⬇️
5.10-m7i.metal-48xl 83.31% <81.81%> (-0.01%) ⬇️
5.10-m8g.metal-24xl 80.18% <87.50%> (ø)
5.10-m8g.metal-48xl 80.18% <87.50%> (ø)
5.10-m8i.metal-48xl 83.31% <81.81%> (+<0.01%) ⬆️
5.10-m8i.metal-96xl 83.31% <81.81%> (ø)
5.10-m9g.metal-48xl 80.18% <87.50%> (ø)
6.1-m5n.metal 83.36% <81.81%> (+<0.01%) ⬆️
6.1-m6a.metal 82.73% <81.81%> (-0.01%) ⬇️
6.1-m6g.metal 80.18% <87.50%> (-0.01%) ⬇️
6.1-m6i.metal 83.36% <81.81%> (-0.01%) ⬇️
6.1-m7a.metal-48xl 82.72% <81.81%> (ø)
6.1-m7g.metal 80.18% <87.50%> (-0.01%) ⬇️
6.1-m7i.metal-24xl 83.38% <81.81%> (+<0.01%) ⬆️
6.1-m7i.metal-48xl 83.37% <81.81%> (-0.01%) ⬇️
6.1-m8g.metal-24xl 80.18% <87.50%> (ø)
6.1-m8g.metal-48xl 80.18% <87.50%> (ø)
6.1-m8i.metal-48xl 83.38% <81.81%> (+<0.01%) ⬆️
6.1-m8i.metal-96xl 83.38% <81.81%> (ø)
6.1-m9g.metal-48xl 80.18% <87.50%> (ø)
6.18-m5n.metal 83.36% <81.81%> (ø)
6.18-m6a.metal 82.73% <81.81%> (ø)
6.18-m6g.metal 80.20% <87.50%> (ø)
6.18-m6i.metal 83.36% <81.81%> (ø)
6.18-m7a.metal-48xl 82.72% <81.81%> (ø)
6.18-m7g.metal 80.20% <87.50%> (ø)
6.18-m7i.metal-24xl 83.38% <81.81%> (+<0.01%) ⬆️
6.18-m7i.metal-48xl 83.38% <81.81%> (ø)
6.18-m8g.metal-24xl 80.20% <87.50%> (ø)
6.18-m8g.metal-48xl 80.20% <87.50%> (ø)
6.18-m8i.metal-48xl 83.38% <81.81%> (ø)
6.18-m8i.metal-96xl 83.38% <81.81%> (ø)
6.18-m9g.metal-48xl 80.20% <87.50%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant