Skip to content

Register AK via KBS API#273

Draft
iroykaufman wants to merge 7 commits into
trusted-execution-clusters:mainfrom
iroykaufman:ak-sync
Draft

Register AK via KBS API#273
iroykaufman wants to merge 7 commits into
trusted-execution-clusters:mainfrom
iroykaufman:ak-sync

Conversation

@iroykaufman

@iroykaufman iroykaufman commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Following PR#248
This PR introduces the registration of AK via the KBS API.

  • Add a register_ak function to add the AK to the KBS as a reference value.
  • Update the attestation policy with a check to verify that the AK is registered.
  • Add a test to confirm the creation and synchronization of the AK upon trustee restarts.

Summary by Sourcery

Register trustee machine secrets and attestation keys with the KBS API and keep them synchronized across trustee restarts.

New Features:

  • Send machine LUKS secrets to the KBS via its API instead of mounting them into the trustee pod.
  • Register attestation keys in the KBS as trusted reference values and enforce their presence in the attestation policy.
  • Introduce JWT-based admin authentication and authorization for the KBS using an operator-managed Ed25519 key pair.
  • Add a controller to resync machine LUKS secrets and attestation keys to the KBS when the trustee deployment becomes ready.

Enhancements:

  • Upgrade the trustee (KBS) image and reconfigure it to use KV storage, authenticated admin API, and updated attestation token handling.
  • Generate and manage trustee auth keys as a Kubernetes Secret and mount the public key into the trustee pod for KBS admin auth.
  • Ensure TLS/crypto providers are explicitly installed to avoid conflicts and make tests compatible with the new crypto stack.

Build:

  • Pin compute-pcrs-lib to a specific revision for reproducible builds.
  • Update the default trustee image tag used in Makefile to a newer KBS version.

Tests:

  • Add integration tests verifying that machine LUKS secrets are synced to the KBS, resynced after trustee restarts, and cleaned up after machine deletion.
  • Add integration tests verifying that attestation keys are registered with the KBS and re-registered after trustee restarts.
  • Add unit tests for Ed25519 key pair generation utilities.

Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
- Replacing the patch mechanism with API calls
- Add a reconcile loop that watches for changes in the trustee deployment and sync LUKS key that was lost.

Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
This test checks that the luks key is first sent to tustee and
validates that after trustee restarts, the keys are sent again.

Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
…n trustee.

Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
@openshift-ci

openshift-ci Bot commented Jun 4, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci

openshift-ci Bot commented Jun 4, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: iroykaufman

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sourcery-ai

sourcery-ai Bot commented Jun 4, 2026

Copy link
Copy Markdown

Reviewer's Guide

Implements KBS-backed secret and attestation key (AK) registration with authenticated admin API, adds trustee restart synchronization logic for LUKS keys and AKs, and extends tests and configuration to validate end-to-end behavior.

Sequence diagram for machine LUKS key registration via KBS API

sequenceDiagram
    actor User
    participant MachineController as keygen_reconcile
    participant Trustee as trustee
    participant K8sAPI as Kubernetes_API
    participant KBS as kbs_client

    User->>MachineController: Create Machine
    MachineController->>Trustee: generate_secret(client, id, owner_reference)
    Trustee->>K8sAPI: create Secret {id}
    K8sAPI-->>Trustee: Secret created
    MachineController->>Trustee: send_secret(client, id)
    Trustee->>Trustee: get_auth_key_token(client)
    Trustee->>Trustee: get_kbs_connection(client)
    Trustee->>K8sAPI: get Secret {id}
    K8sAPI-->>Trustee: Secret {id}
    Trustee->>KBS: set_resource(url, auth_key_token, resource_bytes, secret_path(id), certs)
    KBS-->>Trustee: OK
    Trustee-->>MachineController: Action::await_change()

    User->>MachineController: Delete Machine
    MachineController->>Trustee: delete_secret(client, id)
    Trustee->>Trustee: get_auth_key_token(client)
    Trustee->>Trustee: get_kbs_connection(client)
    Trustee->>KBS: delete_resource(url, auth_key_token, secret_path(id), certs)
    KBS-->>Trustee: OK
    Trustee-->>MachineController: Action::await_change()
Loading

Sequence diagram for trustee restart synchronization and AK registration via KBS API

sequenceDiagram
    participant Main as main
    participant SyncCtrl as launch_trustee_sync_controller
    participant K8sAPI as Kubernetes_API
    participant Reconcile as trustee_deployment_reconcile
    participant Trustee as trustee
    participant KBS as kbs_client

    Main->>SyncCtrl: launch_trustee_sync_controller(client)
    SyncCtrl->>K8sAPI: watch Deployments label app=TRUSTEE_APP_LABEL
    K8sAPI-->>Reconcile: Deployment {TRUSTEE_DEPLOYMENT} update

    Reconcile->>Reconcile: check status.ready_replicas >= desired
    alt Trustee ready
        Reconcile->>Trustee: sync_all_machine_luks_key(client)
        Trustee->>K8sAPI: list Machines
        loop for each Machine id
            Trustee->>Trustee: send_secret(client, id)
            Trustee->>KBS: set_resource(url, auth_key_token, resource_bytes, secret_path(id), certs)
        end

        Reconcile->>Trustee: update_attestation_keys(client)
        Trustee->>K8sAPI: list Secrets label type=AttestationKey
        loop for each ak in ak_secrets
            Trustee->>Trustee: register_ak(client, ak)
            Trustee->>KBS: set_sample_rv(url, "trusted_aks", ak, auth_key_token, certs)
        end
    end

    Reconcile-->>SyncCtrl: Action::await_change()
Loading

File-Level Changes

Change Details Files
Replace local filesystem-based secret mounting with KBS API-backed secret storage and lifecycle for machine LUKS keys.
  • Introduce get_auth_key_token and get_kbs_connection helpers to authenticate and connect to the KBS admin API using an Ed25519 key stored in a Kubernetes Secret.
  • Add send_secret and delete_secret functions that push and remove machine secrets in KBS instead of mounting/unmounting them into the trustee Deployment.
  • Update keygen_reconcile to call send_secret on creation and delete_secret on deletion, mapping errors into controller finalizer errors.
  • Adjust kbs-config.toml resource plugin to use kvstorage instead of LocalFs and change admin/attestation_token configuration to match the KBS API usage.
operator/src/trustee.rs
operator/src/register_server.rs
operator/src/kbs-config.toml
Register attestation keys (AKs) as reference values in KBS and ensure they are kept in sync, including across trustee restarts.
  • Add register_ak helper that stores AK identifiers as trusted_aks sample reference values in KBS via kbs_client.
  • Update update_attestation_keys to iterate over AK Secrets and call register_ak for each instead of patching the trustee Deployment volumes.
  • Extend OPA policy (tpm.rego) to require the AK public key to exist in the trusted_aks reference value set.
  • Introduce sync_all_machine_luks_key and trustee_deployment_reconcile plus launch_trustee_sync_controller to watch the trustee Deployment and resync machine secrets and AK registrations once the Deployment is ready.
operator/src/trustee.rs
operator/src/tpm.rego
operator/src/main.rs
Introduce JWT-based admin authentication/authorization for KBS using operator-managed Ed25519 key material.
  • Add Ed25519KeyPair struct and generate_ed25519_key_pair function to create JWT signing keys.
  • Add generate_trustee_auth_keys_secret to create a TRUSTEE_AUTH_SECRET containing private/public Ed25519 keys, and mount the public key into the KBS pod via generate_kbs_volume_templates.
  • Implement get_auth_key_token to sign an EdDSA JWT with role=admin loaded from TRUSTEE_AUTH_SECRET and use it in KBS API calls.
  • Update main to install the jsonwebtoken CryptoProvider to avoid provider conflicts.
  • Reconfigure kbs-config.toml admin section to use bearer_jwt authentication and regex_acl authorization keyed by the mounted public key.
operator/src/trustee.rs
operator/src/main.rs
operator/src/kbs-config.toml
Align trustee Deployment labeling and add runtime controller to observe readiness for sync operations.
  • Use TRUSTEE_APP_LABEL as the app label for the KBS Deployment and apply it to metadata.labels and selector.
  • Add launch_trustee_sync_controller to start a kube-runtime Controller watching Deployments with the trustee app label and triggering sync_all_machine_luks_key and update_attestation_keys when ready_replicas meets desired.
  • Import futures_util::StreamExt and controller utilities to drive the controller loop.
operator/src/trustee.rs
operator/src/main.rs
Extend end-to-end tests and utilities to validate LUKS key and AK synchronization via KBS and trustee restarts.
  • Add test_luks_key_sync to create Machines, wait for secrets, assert they are sent to KBS, restart the trustee Deployment, confirm resync, and verify deletion from KBS and Kubernetes on Machine deletion via operator logs.
  • Add test_attestation_key_sync to create Machines and AttestationKeys, wait for approval and Secret creation, verify AK registration and re-registration after trustee restart via operator logs.
  • Expose TestContext::wait_for_deployment_ready and initialize a rustls crypto provider in tests for TLS-related behavior.
  • Update tests Cargo.toml to depend on chrono and test_utils Cargo.toml to depend on rustls with ring backend.
tests/trusted_execution_cluster.rs
test_utils/src/lib.rs
tests/Cargo.toml
test_utils/Cargo.toml
Miscellaneous dependency and version updates to support new KBS client and crypto behavior.
  • Add kbs-client and jsonwebtoken dependencies to operator/Cargo.toml with native-tls and aws_lc_rs/use_pem features enabled.
  • Pin compute-pcrs-lib to a specific git revision in the workspace Cargo.toml and bump TRUSTEE_IMAGE version in the Makefile to v0.20.0.
  • Add small unit tests for Ed25519 key generation and remove obsolete mount/unmount tests tied to Deployment volume manipulation.
operator/Cargo.toml
Cargo.toml
Makefile
operator/src/trustee.rs
Cargo.lock

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@iroykaufman iroykaufman force-pushed the ak-sync branch 2 times, most recently from 6ca5e47 to 5039c64 Compare June 4, 2026 13:40
- Add a register_ak function to add the AK to the KBS as a reference value.
- Update the attestation policy with a check to verify that the AK is registered.
- Add a test to confirm the creation and synchronization of the AK upon trustee restarts.

Signed-off-by: Roy Kaufman <rkaufman@redhat.com>
@openshift-ci

openshift-ci Bot commented Jun 10, 2026

Copy link
Copy Markdown

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant