Skip to content

Initial Arm CCA enablement for OpenHCL and TMK test infrastructure#3455

Open
jiong-microsoft wants to merge 3 commits into
microsoft:mainfrom
jiong-microsoft:cca-dev
Open

Initial Arm CCA enablement for OpenHCL and TMK test infrastructure#3455
jiong-microsoft wants to merge 3 commits into
microsoft:mainfrom
jiong-microsoft:cca-dev

Conversation

@jiong-microsoft
Copy link
Copy Markdown

@jiong-microsoft jiong-microsoft commented May 11, 2026

This PR adds the initial Arm Confidential Compute Architecture (CCA) enablement
across OpenHCL, the mshv_vtl backend, and the TMK test path.

At a high level, this wires CCA into OpenHCL as a new hardware-isolated AArch64
CVM backend, parallel to the existing SNP/TDX hardware-isolated paths on x86-64.
The implementation introduces the CCA-specific HCL ioctl surface, models the RSI
plane-run ABI used to enter and resume lower-privilege planes, extends OpenHCL
memory setup for CCA protected memory, and adds a TMK/FVP-based test flow for
exercising the initial realm execution path. FVP (Fixed Virtual Platform) is a
software emulator released by Arm. It aims helping software developement before
the silicon is avaiable and we are using it for CCA OpenHCL testing.

OpenHCL Implementation

CCA is integrated at the same architectural layer as the other OpenHCL
hardware-isolated backends:

  • IsolationType::Cca is added to the platform abstraction so higher layers can
    select CCA as a hardware-isolated partition type.
  • openhcl/hcl adds the CCA-specific /dev/mshv_vtl interface:
    • realm configuration query
    • RSI system register write support
    • RSI memory permission programming
    • CCA VP backing state using a private DMA-allocated plane_run page
  • vm/aarch64/rsi defines the RSI data structures shared with the kernel/RMM
    interface, including realm config and the plane entry/exit/run page layouts.
  • openhcl/underhill_mem treats CCA as a hardware-isolated memory configuration
    and routes lower-VTL memory permission setup through the CCA/RSI path.
  • openhcl/virt_mshv_vtl adds CcaBacked and CcaBackedShared, making CCA a
    first-class backend alongside the existing hypervisor, SNP, and TDX
    backends:
    • Per-partition state is stored in the shared CVM partition state.
    • Per-VP state owns the CCA plane state.
    • VP entry uses the RSI plane-run page.
    • Register state is translated through the RSI plane entry/exit
      structures.
    • The initial exit handling supports the TMK path, including basic
      synchronous exit with MMIO handling supported
  • TMK gains a --hv cca mode that runs inside the paravisor environment,
    allocates private CCA RAM through DMA/PFN-backed memory, constructs an
    OpenHCL CCA partition, binds VPs with CcaBacked, and runs the test
    microkernel inside the realm.

Test Infrastructure

This PR also adds Flowey-based local CCA test support around the Arm CCA FVP
environment:

  • installs/checks the CCA emulation prerequisites
  • builds or updates TF-A, TF-RMM, Plane0 Linux, kvmtool, and rootfs assets
  • injects tmk_vmm and simple_tmk into the emulated realm environment
  • launches the CCA 3-world shrinkwrap configuration ('shrinkwrap' is an python
    based tool released by Arm to simply FVP deployment)
  • adds test data overlays for CCA planes and realm auto-launch

A new user can run CCA OpenHCL test by the following steps:

  • install the env: cargo xflowey cca-tests --install-emu
  • run tests: cargo xflowey cca-tests

If any components (firmwares, plane0 linux kernel etc.) are update, can update
the env using:

cargo xflowey cca-tests --update-emu --rebuild-plane0-linux --rebuild-rootfs

Scope

This is intended as initial enablement rather than full CCA feature completion.
Several paths are deliberately minimal and focused on bringing up the first
OpenHCL/TMK realm execution flow. Follow-up work is expected for fuller
interrupt/GIC handling, complete system register state, broader exit handling,
and production hardening.

Testing Done

  • cargo xflowey cca-tests passed on ARM64

Signed-off-by: Ben Aram b-bearam@microsoft.com
Signed-off-by: Wei Ding b-weiding@microsoft.com
Signed-off-by: Jiong Wang b-jiongwang@microsoft.com

Copilot AI review requested due to automatic review settings May 11, 2026 17:49
@jiong-microsoft jiong-microsoft requested a review from a team as a code owner May 11, 2026 17:49
weiding-msft and others added 2 commits May 11, 2026 10:53
The hardware that support ARM64 CCA is not widely available yet, so the
OpenVMM enablement work is tested using ARM's emulator, Fixed Virtual
Platform (FVP) now.

Arm also has release a python framework, 'shrinkwrap', which simplifies
the deployement of FVP.

This commit add initial test pipeline into flowey
  - add a new `cca-fvp` pipeline entry in `flowey_hvlite`
  - add a local install job to clone/update shrinkwrap, prepare its config,
    install required host dependencies, and fetch/build the external pieces
    needed by the flow
  - add a local shrinkwrap build job which runs `shrinkwrap build` and saves
    the output log under the pipeline working directory
  - add a local shrinkwrap run job which updates `rootfs.ext2` with the
    generated/local artifacts and then invokes `shrinkwrap run`

The new pipeline also provides basic argument handling for platform,
overlays, build-time variables, runtime variables, and output directory
layout, so the whole shrinkwrap environment can live under one self-contained
working directory.

After this commit, the user can use the follow command to build and
launch OpenVMM CCA test

  cargo xflowey cca-fvp

Signed-off-by: Wei Ding <b-weiding@microsoft.com>
Signed-off-by: Jiong Wang <b-jiongwang@microsoft.com>
Signed-off-by: Ben Aram <b-bearam@microsoft.com>
This commit introduces initial ARM CCA enablement across the stack.

Key changes:
- Add CCA ioctl support and related protocol updates
- Introduce CCA processor support in virt_mshv_vtl
- Add RSI support for AArch64
- Update hypervisor definitions and emulator logic
- Extend memory initialization and mapping for CCA
- Integrate changes into VMM and TMK components
- General fixes, cleanup, and refactoring

This represents the initial end-to-end integration of CCA support.

Credit to Ionut Mihalcea <ionut.mihalcea@arm.com> who original explored
and prototyped OpenVMM CCA support. This commit is heavily based on
ideas from his prototype.

Signed-off-by: Ben Aram <b-bearam@microsoft.com>
Signed-off-by: Jiong Wang <b-jiongwang@microsoft.com>
Signed-off-by: Wei Ding <b-weiding@microsoft.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds initial Arm CCA (Confidential Compute Architecture) enablement across OpenHCL and the TMK/FVP test flow, wiring CCA as a new hardware-isolated backend on AArch64.

Changes:

  • Introduces IsolationType::Cca end-to-end (virt, hvdef, OpenHCL backends, and HCL ioctl surface) and adds RSI/plane-run ABI structs (rsi crate).
  • Implements initial CCA-backed VP execution via RSI plane-run pages and routes lower-VTL memory protections through CCA/RSI.
  • Adds Flowey cca-tests pipeline plus shrinkwrap/FVP assets and overlays for local CCA emulation testing.

Reviewed changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
vmm_tests/vmm_tests/test_data/cca_realm_overlay.yaml Buildroot overlay to auto-launch realm and inject TMK startup script
vmm_tests/vmm_tests/test_data/cca_planes.yaml Shrinkwrap overlay for TF-RMM/TF-A/Linux/kvmtool inputs and FVP run params
vmm_core/virt_whp/src/lib.rs Unifies vendor detection across x86_64/aarch64 WHP capabilities
vmm_core/virt_mshv/src/aarch64/mod.rs Adds AArch64 vendor caps; reworks ESR ISS extraction for data aborts
vmm_core/virt_kvm/src/arch/aarch64/mod.rs Adds AArch64 vendor caps for KVM backend
vmm_core/virt_hvf/src/lib.rs Adds AArch64 vendor caps for HVF backend
vmm_core/virt/src/generic.rs Adds IsolationType::Cca and updates mappings/hardware-isolated check
vmm_core/virt/src/aarch64/mod.rs Extends AArch64 partition capabilities with vendor
vm/hv1/hvdef/src/lib.rs Adds HvPartitionIsolationType::CCA
vm/hv1/hv1_emulator/src/hv.rs Gates x86-only hypercall page logic; prepares for AArch64 vendor usage
vm/hv1/hv1_emulator/Cargo.toml Adds aarch64defs dependency
vm/aarch64/rsi/src/lib.rs New RSI/plane-run ABI structs and constants
vm/aarch64/rsi/Cargo.toml New rsi crate definition
vm/aarch64/aarch64emu/src/emulator.rs Updates data-abort ISS reconstruction for new ESR layout
vm/aarch64/aarch64defs/src/lib.rs Refactors EsrEl2 ISS layout; adds Vendor type and helpers
tmk/tmk_vmm/src/run.rs Adds Linux/AArch64 CCA private DMA allocation and memory layout path
tmk/tmk_vmm/src/paravisor_vmm.rs Passes VTOM and CVM late params; binds CCA VPs to CcaBacked
tmk/tmk_vmm/src/main.rs Adds --hv cca option and options finalization
tmk/tmk_vmm/src/load.rs Splits x86/aarch64 loader paths; adds load offset for aarch64
tmk/tmk_vmm/Cargo.toml Adds deps for CCA memory ranges and user_driver
support/safe_intrinsics/src/lib.rs Adds AArch64 read_cntfrq_el0() helper
openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Adds explicit expect()s for required DMA clients/allocations
openhcl/virt_mshv_vtl/src/processor/snp/mod.rs Adds explicit expect()s for required direct-overlay DMA allocations
openhcl/virt_mshv_vtl/src/processor/mshv/arm64.rs Uses let ... else { unreachable!() } for backing match
openhcl/virt_mshv_vtl/src/processor/mod.rs Adds CCA processor module; adjusts trait gating and imports
openhcl/virt_mshv_vtl/src/processor/hardware_cvm/mod.rs Adds Cca match arm (currently unreachable) in x64 initial context
openhcl/virt_mshv_vtl/src/processor/cca/mod.rs New CCA-backed processor implementation using RSI plane-run
openhcl/virt_mshv_vtl/src/lib.rs Adds CCA backing shared state; optional DMA clients; aarch64 timing path
openhcl/virt_mshv_vtl/Cargo.toml Adds rsi dependency
openhcl/underhill_mem/src/lib.rs Adds CCA memory permission plumbing and RSI perm ioctl usage
openhcl/underhill_mem/src/init.rs Refactors VTL0 protections into helper; treats CCA like SNP for VTOM mapping
openhcl/underhill_mem/Cargo.toml Adds rsi dependency
openhcl/openhcl_boot/src/host_params/shim_params.rs Marks CCA as hardware-backed isolation
openhcl/hcl/src/ioctl/register.rs Adds CCA VSM caps default; adjusts aarch64 privilege query for hardware isolation
openhcl/hcl/src/ioctl/cca.rs New HCL ioctl surface for realm config, RSI sysreg write, and mem perms
openhcl/hcl/src/ioctl.rs Registers new CCA ioctls/types; adds CCA backing state allocation
openhcl/hcl/Cargo.toml Adds aarch64defs and rsi deps
flowey/flowey_lib_hvlite/src/_jobs/mod.rs Wires new local CCA emu/test jobs
flowey/flowey_lib_hvlite/src/_jobs/local_update_cca_emu.rs Adds emu update job (rebuild plane0 linux/rootfs)
flowey/flowey_lib_hvlite/src/_jobs/local_run_cca_test.rs Adds job to inject artifacts into rootfs and launch shrinkwrap/FVP run
flowey/flowey_lib_hvlite/src/_jobs/local_install_cca_emu.rs Adds job to install shrinkwrap + plane0 linux build + overlay syncing
flowey/flowey_lib_hvlite/src/_jobs/local_check_cca_emu_prereq.rs Adds env prerequisite checks (packages, docker group)
flowey/flowey_hvlite/src/pipelines/mod.rs Adds cca-tests pipeline entrypoint
flowey/flowey_hvlite/src/pipelines/cca_tests.rs New cargo xflowey cca-tests pipeline (install/update/run)
Guide/src/dev_guide/dev_tools/xflowey.md Documents cargo xflowey cca-tests
Cargo.toml Adds rsi workspace member
Comments suppressed due to low confidence (3)

tmk/tmk_vmm/src/paravisor_vmm.rs:1

  • cvm_params is now always Some(...) and unconditionally unwraps m.cvm_memory(). For non-hardware-isolated runs (e.g., IsolationType::None used by KVM/HVF/WHP paths), cvm_memory() is expected to be None, causing a panic. Gate cvm_params creation on m.cvm_memory().is_some() (or on isolation == IsolationType::Cca) and avoid unwrap() by using if let Some(cvm) = m.cvm_memory() { ... }.
    vmm_core/virt_mshv/src/aarch64/mod.rs:1
  • The reconstructed ISS value omits the new EsrEl2 bits a/b/c/d (bits 21–24). For data aborts, this includes flags such as ISV (bit 24), so iss.isv() can become false even when the syndrome is valid, potentially turning valid data abort handling into a fatal error. Reconstruct the full 25-bit ISS by OR-ing in a/b/c/d at bits 21–24 (matching the approach used in vm/aarch64/aarch64emu/src/emulator.rs), or add an EsrEl2::iss_u32() helper in aarch64defs and use it consistently.
    vm/aarch64/rsi/src/lib.rs:1
  • size_of is used but not imported/qualified in this new crate, which will fail to compile. Add use core::mem::size_of; near the top, or fully qualify (core::mem::size_of::<...>()) at each call site.

Comment thread openhcl/underhill_mem/src/init.rs Outdated
Comment thread openhcl/hcl/src/ioctl/cca.rs
Comment thread openhcl/hcl/src/ioctl.rs
Comment thread openhcl/virt_mshv_vtl/src/lib.rs Outdated
Comment thread flowey/flowey_lib_hvlite/src/_jobs/local_run_cca_test.rs
@github-actions github-actions Bot added Guide unsafe Related to unsafe code labels May 11, 2026
@github-actions
Copy link
Copy Markdown

⚠️ Unsafe Code Detected

This PR modifies files containing unsafe Rust code. Extra scrutiny is required during review.

For more on why we check whole files, instead of just diffs, check out the Rustonomicon

@jiong-microsoft
Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree [company="Microsoft"]

@jiong-microsoft
Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

  - fix all regresssions on x86-64
  - fix all typos in comments
  - fix all 'cargo doc' build errors
  - further remove unnecessary cfgs
  - remove packed structure to avoid unaligned access
  - remove unused method
  - remove unused dependency for 'rsi' crate
  - remove duplicated code blocks inside underhill_mem
  - use 'expect' rather than 'allow' across the code
  - remove unnecessary introducing of Option
  - remove unnecessary touch on TDX and SNP code
  - enable existing tmk aarch64 test sequence rather hacked version
  - redesign flowey based test flow, now we have:
    - cargo xflowey cca-tests --install-emu
    - cargo xflowey cca-tests --update-emu [subcommands]
      [subcommands] can be --rebuild-plane0-linux --rebuild-rootfs
  - also automate flowey testflow by packing some test scrips into
    guest rootfs and init process

Signed-off-by: Jiong Wang <b-jiongwang@microsoft.com>
Signed-off-by: Ben Aram <b-bearam@microsoft.com>
Signed-off-by: Wei Ding <b-weiding@microsoft.com>
Copilot AI review requested due to automatic review settings May 11, 2026 21:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Guide unsafe Related to unsafe code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants