Split supply chain factors and fix several other failing factors#116
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors Teep’s supply-chain verification reporting by splitting “name recognition” from “signature/provenance verification”, aiming to fix regressions like provider image renames/new components (Issue #114) without weakening enforced provenance checks. It also updates Tinfoil supply-chain reporting, NearCloud gateway REPORTDATA binding, policy entries for newly observed components/signers, and various formatting/tests/docs to match the new factor model.
Changes:
- Split compose supply-chain evaluation into separate factors:
component_recognition(default allow-fail) and enforced signer/signature recognition factors, while keepingbuild_transparency_logfocused on provenance/log integrity. - Add Rekor fetching helpers that can prefer entries matching a component’s signature policy; update compose providers and verify CLI to pass digest→repo + supply-chain policy into Rekor fetching.
- Expand provider policies/tests (e.g., OpenTelemetry collector signer fingerprint; NearCloud alpine provenance; Tinfoil factor applicability and per-component tracking) and update CLI help + integration/unit tests.
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/verify/verify.go | Thread supply-chain policy + digest→repo mapping into Sigstore/Rekor checks; pass HTTP client into Tinfoil supply-chain verify. |
| internal/verify/verify_test.go | Add regression test ensuring formatted report columns align for longer factor names. |
| internal/verify/attest.go | Extend checkSigstore to fetch Rekor provenance with policy/digest→repo context; enhance Tinfoil supply-chain result population. |
| internal/verify/attest_test.go | Update call sites for new checkSigstore signature; adjust Tinfoil supply-chain tests. |
| internal/proxy/proxy.go | Ensure missing Tinfoil repo produces a concrete failing supply-chain result; record per-component Sigstore results. |
| internal/provider/tinfoil/attester_test.go | Update expected inapplicable/applicable factor set for Tinfoil after factor split. |
| internal/provider/tinfoil/applicability.go | Make build_transparency_log applicable for Tinfoil; keep other non-applicable compose factors. |
| internal/provider/neardirect/policy.go | Add OpenTelemetry collector signing key fingerprint to tighten signer recognition. |
| internal/provider/neardirect/policy_test.go | Add tests asserting OpenTelemetry signer trust + alpine absence in neardirect policy. |
| internal/provider/neardirect/pinned.go | Fetch Rekor provenances with policy + digest→repo mapping (policy-aware selection). |
| internal/provider/nearcloud/reportdata.go | Update gateway REPORTDATA binding scheme to include signing address + TLS fingerprint; validate lengths/hex. |
| internal/provider/nearcloud/reportdata_test.go | Update tests for new REPORTDATA scheme; add coverage for empty signing address and fingerprint validation. |
| internal/provider/nearcloud/policy.go | Strengthen alpine policy from “SigstorePresent” to explicit Fulcio identity/source allowlist. |
| internal/provider/nearcloud/policy_test.go | Add tests for OpenTelemetry signer trust and alpine signer identity allowlist. |
| internal/provider/nearcloud/pinned.go | Reuse a single computed scPolicy; pass policy + digest→repo into Rekor fetching; use signing address in REPORTDATA verify. |
| internal/provider/nearcloud/nearcloud.go | Parse and propagate gateway signing_address from provider response into raw attestation for verification. |
| internal/provider/chutes/applicability.go | Mark new supply-chain recognition factors as inapplicable for Chutes. |
| internal/integration/verify_run_test.go | Update expectations for Tinfoil direct fixture: recognition passes but enforced Sigstore/replay-dependent factors fail closed. |
| internal/integration/tinfoil_test.go | Provide synthetic Tinfoil supply-chain result in integration test and assert new supply-chain factors. |
| internal/config/config_test.go | Update allow-fail defaults expectations (notably response_schema for Tinfoil). |
| internal/attestation/report.go | Add new factor constants + evaluators; split supply-chain checks; add Tinfoil per-component tracking and policy helpers. |
| internal/attestation/report_test.go | Update factor counts and multiple supply-chain behavior tests for new factor model. |
| internal/attestation/rekor.go | Add policy-aware Rekor fetch APIs and implement preference for policy-matching entries where possible. |
| internal/attestation/rekor_test.go | Add coverage ensuring policy-aware Rekor fetch prefers a policy-matching entry. |
| internal/attestation/policy_test.go | Update/extend tests for new supply-chain factor behaviors (rename scenarios, third-party signer recognition, compose-only cases, Tinfoil cases). |
| internal/attestation/export_test.go | Export new evaluator funcs for tests. |
| internal/attestation/attestation.go | Add GatewaySigningAddress to raw attestation state. |
| docs/plans/supply_chain_factors.md | Document the intended factor split, enforcement defaults, and test plan. |
| cmd/teep/help.go | Register new supply-chain factors and document their semantics in CLI help. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
internal/attestation/report.go:1724
- In the build_transparency_log Rekor evaluation, several failure details still use the "image" term (e.g., "image %q: Rekor SET verification failed"). With the PR introducing component_* factors and switching policy language to components, these user-facing strings should be updated to "component" for consistency.
if r.SETErr != nil {
return FactorResult{Tier: TierSupplyChain, Name: FactorBuildTransparency, Status: Fail,
Detail: fmt.Sprintf("image %q: Rekor SET verification failed: %v", imageRepo, r.SETErr)}
}
if r.InclusionErr != nil {
return FactorResult{Tier: TierSupplyChain, Name: FactorBuildTransparency, Status: Fail,
Detail: fmt.Sprintf("image %q: Rekor inclusion proof verification failed: %v", imageRepo, r.InclusionErr)}
}
The nvswitch field is only present for some models
Make Rekor provenance selection policy-aware even when an attested component repo is unknown or renamed. Unknown repos now search provider-wide trusted signature policies before falling back to arbitrary Fulcio/raw-key entries, and known repos keep exact component signature matches preferred while retaining provider-wide signer matches as fallback so provider_signer_recognition can remain orthogonal to component_signature_recognition. Add regression coverage for unknown repo raw-key selection and known repo provider-signer fallback.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2a924e7 to
9a5c550
Compare
9a5c550 to
ff768a7
Compare
- NEAR pinned report builders skipped provider inapplicability, so `nvswitch_binding` stayed `Skip`, got promoted to enforced `Fail`, and produced the misleading “Tinfoil supply chain not verified” block. Fixed in [neardirect pinned.go](/Users/rainwater/Source/study/TEEs/teep/internal/provider/neardirect/pinned.go:370) and [nearcloud pinned.go](/Users/rainwater/Source/study/TEEs/teep/internal/provider/nearcloud/pinned.go:429). - Live inference integration configs for `neardirect`, `nearcloud`, `tinfoil_v3_cloud`, and `tinfoil_v3_direct` were using `AllowFail: attestation.KnownFactors`, so inference tests allowed every factor to fail. Removed those blanket allow-fail overrides so provider defaults are exercised. - Plaintext pinned requests could reuse a global SPKI cache hit with no provider/model report cache entry, allowing a cross-provider/shared-domain nil-report path. The pinned pre-dispatch guard now evicts SPKI on any pinned report-cache miss, not only E2EE misses, forcing fresh attestation. See [proxy.go](/Users/rainwater/Source/study/TEEs/teep/internal/proxy/proxy.go:1918). Added regressions: - NEAR Direct and NearCloud tests enforce only `nvswitch_binding` and assert it becomes N/A, not a block. - Proxy test covers plaintext pinned cache-miss SPKI eviction.
- Fixed Tinfoil capture/replay by passing the recording client into Tinfoil supply-chain verification in [verify.go](/Users/rainwater/Source/study/TEEs/teep/internal/verify/verify.go). - Updated fixture selection to choose the newest capture by manifest timestamp, not lexicographic directory name, in [helpers_test.go](/Users/rainwater/Source/study/TEEs/teep/internal/integration/helpers_test.go). - Fixture tests now reuse captured E2EE outcomes, so no-key replay reflects the API-key capture result. - Chutes fixture test now exercises the GPU evidence path instead of only checking `NvidiaPayload`. - Tinfoil fixture tests now run through `verify.Run`, so they exercise the same supply-chain replay path as `teep verify`. - Refreshed captures for NearDirect, NearCloud, Chutes, Tinfoil cloud, and Tinfoil direct. The latest useful Tinfoil captures are: - `internal/integration/testdata/tinfoil_v3_cloud_llama3-3-70b_20260629_021249/` - `internal/integration/testdata/tinfoil_v3_direct_gemma4-31b_20260629_021304/`
This may not be enough; this provider is highly regarded garbo.
Another highly regarded provider.
Fixes #114 as per #114 (comment) and the following comment about tinfoil.