Skip to content

test(release): bind lifecycle evidence to attested runs - #61

Draft
ahrav wants to merge 1 commit into
stack/mc-host-13-client-compatibilityfrom
stack/mc-host-14-release-evidence
Draft

test(release): bind lifecycle evidence to attested runs#61
ahrav wants to merge 1 commit into
stack/mc-host-13-client-compatibilityfrom
stack/mc-host-14-release-evidence

Conversation

@ahrav

@ahrav ahrav commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

Release checks now bind proof bytes, target reports, the installed evidence record, checkout commit, signer workflow, and workflow run into one attested chain. Cross-harness smoke exercises two owners, stop plus demand-start, real request completion, replay, and cleanup.

Verification

  • bun run test:release: 135 tests passed
  • Attestation mutation coverage for commit, run, workflow, artifact, and target-report drift

Remaining Qualification

The gate stays fail-closed until the protected qualification workflow, exact Linux/macOS campaigns, installed registry flows, and production Synapse evidence exist. npm publication remains intentionally out of scope.

Stack

  1. fix(mc-host): authenticate daemon version transcripts #58 authentication transcript
  2. fix(mc-host): bind runtime execution to verified identity #59 native runtime identity
  3. fix(plugin): fence lifecycle compatibility by daemon #60 client compatibility
  4. test(release): bind lifecycle evidence to attested runs #61 release evidence
  5. fix(ci): restore portable lifecycle source checks #62 CI portability

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

Comment on lines +644 to +646
if (
!/^https:\/\/github\.com\/ahrav\/magic-context\/actions\/runs\/\d+$/.test(runUrl) ||
repository !== "ahrav/magic-context" ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit: "ahrav/magic-context" is now hardcoded twice here (the runUrl regex and this repository check), in addition to the existing `https://github.com/${source.repository}` templating used elsewhere in attestationCertificateMatches. Consider hoisting it into a single EXPECTED_REPOSITORY constant next to QUALIFICATION_WORKFLOW_PATH so a future repo rename/fork can't leave one occurrence stale.

@claude

claude Bot commented Aug 26, 2026

Copy link
Copy Markdown

Review

Scope: scripts/build-mc-host-payload.ts, scripts/verify-mc-host-release-evidence.ts, scripts/qualify-mc-host-production-inputs.ts, and scripts/smoke-mc-host-cross-harness.ts (plus their tests). Reviewed by reading the full diff and the surrounding source, and tracing the attestation/verification chain end to end. (bun test required interactive approval that wasn't available in this sandbox, so I couldn't execute the suite — static review only.)

Overall: this is a solid, well-scoped hardening of the release-evidence gate. Highlights:

  • attestationCertificateMatches/attestationMatchesWorkflowSource binds proof and installed-evidence attestations to the exact artifact digest, commit SHA, repo, workflow path, and run ID — closing what looks like a real gap where the old code only ran gh attestation verify --repo ahrav/magic-context without checking the returned certificate fields at all.
  • test_report_path/test_report_sha256 observations are now actually re-hashed against on-disk bytes (sha256File) instead of trusting an arbitrary claimed hash — good, this was previously unverified ("b".repeat(64) in the old test fixture).
  • proof_artifacts[].path is now pinned to tmp/mc-host-release-proofs/ in addition to the existing traversal check — reasonable extra containment.
  • The "all qualified proofs must share one workflow source" check (qualifiedSource comparison) and the separate "installed evidence bytes match what's on disk and is itself attested" check are good anti-tampering additions — they stop a caller from validating an in-memory object while the on-disk/attested file says something else.
  • loadReleaseContext's evidence-citation refactor in build-mc-host-payload.ts looks behavior-preserving (just deduplicating the synthesized-vs-loaded evidence path) — didn't find a regression there.
  • Test changes (freshRoot() instead of repoRoot in several build-mc-host-payload.test.ts cases) look like a legitimate test-isolation fix rather than a behavior change.

Nits (posted inline where applicable):

  • "ahrav/magic-context" is hardcoded in a couple of places in verify-mc-host-release-evidence.ts (the runUrl regex and the repository equality check) alongside the templated `https://github.com/${source.repository}` used elsewhere — worth a shared constant to avoid drift.

Things I couldn't fully verify from this diff alone (not blocking, just flagging for awareness):

  • QUALIFICATION_WORKFLOW_PATH (.github/workflows/mc-host-release-qualification.yml) doesn't exist in this repo yet, and the default verifyWorkflowRun/verifyAttestation paths don't check the triggering event/actor (e.g. fork-safety) — but this matches the PR description's stated remaining work ("gate stays fail-closed until the protected qualification workflow ... exist"), so it reads as an intentional, disclosed gap rather than an oversight in this diff.

No correctness or security bugs found in the changed logic itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant