test: add container-based acceptance test infrastructure

[trevor-vaughan]

Add acceptance tests that validate the full complyctl lifecycle (init,
get, scan, export) against a real OCI registry (zot) using
podman-compose/docker-compose orchestration.

Infrastructure:

• Zot OCI registry container with Gemara test policy artifacts
• Seed container that pushes catalog + policy to the registry via oras
• SUT (system under test) container running complyctl against the registry
• Compose orchestration with lifecycle profile and health checks

Test coverage:

• Registry connectivity and policy pull (complyctl get)
• Workspace initialization (complyctl init)
• Policy scanning with test provider (complyctl scan)
• Report generation in multiple formats (complyctl scan --format)
• Doctor diagnostics (complyctl doctor)

Build integration:

• make test-acceptance / make test-acceptance-clean targets
• CI workflow (.github/workflows/acceptance_test.yml)
• AGENTS.md and CHANGELOG.md updated

Refs: #653

Assisted-by: Claude Opus 4.6
Signed-off-by: Trevor Vaughan tvaughan@redhat.com

[github-actions]

✅ CRAP Load Analysis: PASS

Summary

Metric	Value
Functions analysed	91
Avg complexity	4.4
Avg line coverage	67.1%
Avg CRAP score	8.7
CRAPload (>= 15)	11
Avg contract coverage	86.4%
Avg GazeCRAP score	5.2
GazeCRAPload (>= 15)	0
Regressions	6
Improvements	3
New functions	23

Quadrant Distribution

Quadrant	Count
Q1 Safe	22
Q2 Complex but Tested	0
Q3 Simple but Underspecified	0
Q4 Dangerous	0

Regressions

Function	Baseline CRAP	Current CRAP	Delta	Baseline GazeCRAP	Current GazeCRAP	Delta
internal/cache/cachetest/mock_source.go:(*MockPolicySource).DefinitionVersion	6	12	6	N/A	N/A	N/A
internal/cache/complypack_sync.go:NewComplypackSync	1	2	1	N/A	N/A	N/A
internal/cache/complypack_sync.go:(*ComplypackSync).SyncComplypack	13.587834265520216	36.644352000000005	23.05651773447979	N/A	N/A	N/A
internal/cache/state.go:SaveState	4.592592592592593	16.998912	12.406319407407409	N/A	N/A	N/A
internal/cache/sync.go:NewSync	1	2	1	N/A	N/A	N/A
internal/cache/sync.go:(*Sync).SyncPolicy	12.666666666666668	14.17763754153939	1.5109708748727222	N/A	N/A	N/A

Improvements

Function	Baseline CRAP	Current CRAP	Delta	Baseline GazeCRAP	Current GazeCRAP	Delta
internal/cache/cachetest/mock_source.go:isDuplicateErr	6	2	-4	N/A	N/A	N/A
internal/cache/complypack.go:(*ComplypackCache).LookupByEvaluatorID	10.462962962962964	7.036814425244177	-3.4261485377187864	N/A	N/A	N/A
internal/policy/loader.go:(*Loader).ListCachedPolicies	30	5.150262960180315	-24.849737039819686	N/A	5	N/A

New Functions

Status	Function	CRAP	GazeCRAP	Note
+	internal/cache/cachetest/mock_source.go:(*MockPolicySource).SetError	2	N/A	new
+	internal/cache/complypack.go:evictOldVersions	21.80353117956424	N/A	new
+	internal/cache/complypack.go:buildTrackedVersionSet	6.104956268221574	N/A	new
+	internal/cache/complypack.go:sortByLastUpdated	6	N/A	new
+	internal/cache/complypack.go:(*ComplypackCache).lookupByDirScan	9.486851990984222	N/A	new
+	internal/cache/complypack_sync.go:(*ComplypackSync).cacheExistsForState	3.140625	N/A	new
+	internal/cache/complypack_sync.go:(*ComplypackSync).tryLocalCacheHit	6.104956268221574	N/A	new
+	internal/cache/retention.go:CacheRetentionCount	4	N/A	new
+	internal/cache/state.go:(*State).UpdatePolicyStateWithVerification	3.009	12	new
+	internal/cache/state.go:(*State).UpdateComplypackStateWithVerification	3.009	12	new
+	internal/cache/state.go:(*State).EvaluatorIDToVersion	6	N/A	new
+	internal/cache/sync.go:WithVerifier	1	N/A	new
!+	internal/cache/verify.go:NewKeylessVerifier	42	N/A	new violation
!+	internal/cache/verify.go:NewKeyedVerifier	56	N/A	new violation
!+	internal/cache/verify.go:bundleFromCosignOCI	42	N/A	new violation
+	internal/cache/verify.go:resolveArtifactDigest	12	N/A	new
+	internal/cache/verify.go:fetchCosignSignature	20	N/A	new
+	internal/cache/verify.go:findSigningLayer	4.03125	N/A	new
+	internal/cache/verify.go:buildProtobufBundle	6.141711619769645	N/A	new
+	internal/cache/verify.go:buildVerificationMaterial	5.072886297376093	N/A	new
+	internal/cache/verify.go:parseCertificateChain	6	N/A	new
+	internal/cache/verify.go:parseRekorBundle	5.2459912536443145	N/A	new
+	internal/cache/verify.go:extractVerificationResult	6	N/A	new

View full analysis logs

[marcusburghardt]

PR Review: #669 — test: add container-based acceptance test infrastructure

Solid acceptance test infrastructure with clean compose orchestration (zot + oras seed + SUT). Two well-structured tests cover the lifecycle and OSCAL format interop paths. One blocking issue with exit code propagation in the Makefile target.

Alignment

• PR uses Refs: #653 (partial coverage acknowledged). Missing coverage for init, list, --show-passing, error exit codes, SARIF/Markdown formats — expected as follow-up work.
• Several unrelated changes included (.DEFAULT_GOAL, .jscpd.json, .mega-linter.yml zizmor config, generation_state.go error wrapping). Per Constitution III, unrelated changes should be in separate commits.

This review was generated by /review-pr (AI-assisted).

[marcusburghardt]

LGTM

[marcusburghardt]

LGTM

[hbraswelrh]

This PR looks good. I left two comments regarding the Gemara aritfacts in the testdata. I'd be happy to open a PR to fix the files.

[marcusburghardt]

@trevor-vaughan , same case from #663 where multiple PRs change the same file. So when the first is merged the others are automatically blocked with a conflict that requires manual intervention. This can happen, but it would be great if we can limit the scope of PRs to specific changes. This way, a simple rebase should unblock all PRs to be merged once approved.

[trevor-vaughan]

@trevor-vaughan , same case from #663 where multiple PRs change the same file. So when the first is merged the others are automatically blocked with a conflict that requires manual intervention. This can happen, but it would be great if we can limit the scope of PRs to specific changes. This way, a simple rebase should unblock all PRs to be merged once approved.

I make the changes necessary to get the tests to pass. Unfortunately, this wasn't set correctly the first time so I got caught in the tailwind with changes that were already in-flight. I also happened to mis-type this one 😮‍💨 .

The PR would have needed to be rebased anyway.

[marcusburghardt]

@trevor-vaughan , same case from #663 where multiple PRs change the same file. So when the first is merged the others are automatically blocked with a conflict that requires manual intervention. This can happen, but it would be great if we can limit the scope of PRs to specific changes. This way, a simple rebase should unblock all PRs to be merged once approved.

I make the changes necessary to get the tests to pass. Unfortunately, this wasn't set correctly the first time so I got caught in the tailwind with changes that were already in-flight. I also happened to mis-type this one 😮‍💨 .

The PR would have needed to be rebased anyway.

I understood the intention. But it is also fine to just open a tiny PR fixing the file and mentioning it in other PRs so reviewers can quickly see the dependency relationship and we can merge the related PRs with rebases. The #663 has the necessary approvals but is blocked by the same reason.

[marcusburghardt]

#663 (comment)