diff --git a/.gitignore b/.gitignore index 82555792..3f55ee2e 100644 --- a/.gitignore +++ b/.gitignore @@ -77,6 +77,7 @@ gha-creds-*.json # Runtime data under .uf/ (databases, caches, locks, logs) .uf/workflows/ .uf/artifacts/ +.uf/feedback/ .uf/dewey/graph.db .uf/dewey/graph.db-shm .uf/dewey/graph.db-wal diff --git a/AGENTS.md b/AGENTS.md index 05289804..07e0f64d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -285,6 +285,7 @@ packages organized by domain responsibility. ## Recent Changes - complypack-cache-versioning: `COMPLYTIME_CACHE_VERSIONS` env var configures retention count (default 1) for complypack cache versions per evaluator-id; `NewComplypackCache()` gains `*State` parameter for state-driven lookup and timestamp-based eviction ordering; `evictOldVersions()` becomes retention-count-aware (orphaned dirs first, then oldest by `LastUpdated`); `LookupByEvaluatorID()` resolves from state.json with directory-scan fallback; `SyncComplypack()` checks local cache before remote fetch with re-verification when verifier is configured, returns `(true, nil)` for cache hits to trigger generation invalidation; `EvaluatorIDToVersion()` reverse lookup on `*State`; `CacheRetentionCount()` in `internal/cache/retention.go`; `CheckComplypacks()` extended with `walkCacheSize()` and `findOrphanedVersions()` for cache health reporting; `complyctl doctor` reports cache size and orphaned/untracked versions (#676) +- bundle-metadata-cli: `complyctl list` gains EVALUATOR and CONTROLS columns sourced from `PolicyState` metadata in `state.json`; `complyctl get` prints post-sync summary to stderr after fresh fetch showing policy title, evaluator, control count, and assessment count; `PolicyState` gains `PolicyTitle`/`PolicyEvaluator`/`ControlCount`/`AssessmentCount` fields populated at sync time via `SetPolicyMetadata()`; `policy.Resolver.ExtractPolicyMetadata()` extracts display metadata without building full `DependencyGraph`; `policyLayerResult.Title` added to `parsePolicyLayer()` output; upgrade backfill for pre-existing caches without metadata; `formatPolicySummary()` helper for testable stderr output (#506) - per-entry-verification: Per-entry `verification:` and `skip_verify:` fields on `PolicyEntry` in `internal/complytime/config.go` for policy-level signature verification overrides; `resolveVerifier()` in `cmd/complyctl/cli/get.go` with verifier cache and resolution priority chain (entry → workspace → none); error collection in `syncAllPolicies()`/`syncAllComplypacks()` via `errors.Join()` for partial-failure resilience; cross-group error collection in `syncAll()` so policy failures do not block complypack sync; per-entry WARNING to stderr on sync failure for real-time feedback; `validateEntries()` extended for per-entry verification validation and mutual exclusivity of `verification:` and `skip_verify:` (#680) - debug-visible-output: `--debug` flag now tees all log messages to stderr via `teeWriter` in `pkg/log/log.go`; `enableDebug()` in `cmd/complyctl/cli/root.go` reconstructs logger with `NewTeeWriter(stderr, logFile)` and forces `termenv.ANSI256` color profile when stderr is a TTY (respects `NO_COLOR`); `Debug log: ` hint printed to stderr after workspace resolution; flag description updated to `"output debug logs to stderr and log file"`; `SetColorProfile()` method added to `CharmHclog` adapter (#614) - sigstore-verification: `complyctl get` verifies OCI artifact signatures via `sigstore-go` when `verification:` configured in `complytime.yaml`; keyless (OIDC issuer + identity) and keyed (public key) modes; pre-copy verification via registry API before `oras.Copy()`; `--skip-verify` flag; `VerificationConfig` in `WorkspaceConfig`; `VerifyFunc`/`NewKeylessVerifier`/`NewKeyedVerifier` in `internal/cache/verify.go`; `PolicyState` gains `Verified`/`SignerIdentity`/`Issuer`/`VerifiedAt` fields; `SyncOption`/`WithVerifier()` functional options on `Sync`/`ComplypackSync`; `complyctl list` VERIFIED column; `complyctl doctor` `CheckVerification` diagnostic; `sigstore-go` v1.2.1 + `go-containerregistry` dependencies added diff --git a/CHANGELOG.md b/CHANGELOG.md index c8532187..7be36954 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,24 @@ ## Unreleased +### Added + +- `complyctl list` now displays EVALUATOR and CONTROLS columns + showing the evaluator type and control count for each cached + policy. Metadata is sourced from cached state populated during + `complyctl get`. Pre-existing caches show "-" until re-fetched. + (#506) + +- `complyctl get` now prints a post-sync summary to stderr after + fetching a policy, showing the policy title, evaluator, control + count, and assessment count. The summary appears only for freshly + fetched policies. (#506) + +- `PolicyState` in `state.json` gains `policy_title`, + `policy_evaluator`, `control_count`, and `assessment_count` fields + populated at sync time. Backward compatible via `omitempty` JSON + tags. (#506) + ### Fixed - `complyctl get` recorded the OCI tag version (e.g., `v1.0.0`) in diff --git a/cmd/complyctl/cli/cli_test.go b/cmd/complyctl/cli/cli_test.go index 946b72ad..0292a304 100644 --- a/cmd/complyctl/cli/cli_test.go +++ b/cmd/complyctl/cli/cli_test.go @@ -573,7 +573,7 @@ func TestListOptions_Run_ShowsDigestColumn(t *testing.T) { Policies: map[string]cache.PolicyState{ "policies/test-policy": { Version: "v1.0", - Digest: "sha256:9f86d081884c7d65", + Digest: "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", }, }, } @@ -609,13 +609,359 @@ func TestListOptions_Run_UncachedPolicyShowsDash(t *testing.T) { output := buf.String() assert.Contains(t, output, "DIGEST") - // Should show "-" for digest when uncached. Version column also shows "-". + // Should show "-" for digest when uncached. Version, evaluator, + // controls, and digest columns all show "-". lines := strings.Split(strings.TrimSpace(output), "\n") require.GreaterOrEqual(t, len(lines), 2, "should have header + at least one data row") dataRow := lines[1] - // Count occurrences of "-" in the data row (version, digest = 2) + // Count occurrences of "-" in the data row + // (version, evaluator, controls, digest = 4 minimum) dashCount := strings.Count(dataRow, "-") - assert.GreaterOrEqual(t, dashCount, 2, "uncached policy should show '-' for version and digest") + assert.GreaterOrEqual(t, dashCount, 4, + "uncached policy should show '-' for version, evaluator, controls, and digest") +} + +func TestListOptions_Run_ColumnOrder(t *testing.T) { + chdirTemp(t) + writeWorkspaceConfig(t, minimalConfig) + + cacheDir := t.TempDir() + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "policies/test-policy": { + Version: "v1.0", + Digest: "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + PolicyEvaluator: "openscap", + ControlCount: 42, + }, + }, + } + require.NoError(t, cache.SaveState(state, cacheDir)) + + var buf bytes.Buffer + o := &listOptions{ + Common: &Common{Output: Output{Out: &buf}}, + cacheDir: cacheDir, + } + err := o.run(context.Background()) + require.NoError(t, err) + + output := buf.String() + lines := strings.Split(strings.TrimSpace(output), "\n") + require.GreaterOrEqual(t, len(lines), 1, "should have header row") + header := lines[0] + + // Verify column order: POLICY ID, VERSION, EVALUATOR, CONTROLS, + // DIGEST, VERIFIED per spec requirement. + versionIdx := strings.Index(header, "VERSION") + evaluatorIdx := strings.Index(header, "EVALUATOR") + controlsIdx := strings.Index(header, "CONTROLS") + digestIdx := strings.Index(header, "DIGEST") + assert.Greater(t, evaluatorIdx, versionIdx, + "EVALUATOR must appear after VERSION") + assert.Greater(t, controlsIdx, evaluatorIdx, + "CONTROLS must appear after EVALUATOR") + assert.Greater(t, digestIdx, controlsIdx, + "DIGEST must appear after CONTROLS") +} + +func TestListOptions_Run_ShowsMetadata(t *testing.T) { + chdirTemp(t) + writeWorkspaceConfig(t, minimalConfig) + + cacheDir := t.TempDir() + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "policies/test-policy": { + Version: "v1.0", + Digest: "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + PolicyEvaluator: "openscap", + ControlCount: 42, + }, + }, + } + require.NoError(t, cache.SaveState(state, cacheDir)) + + var buf bytes.Buffer + o := &listOptions{ + Common: &Common{Output: Output{Out: &buf}}, + cacheDir: cacheDir, + } + err := o.run(context.Background()) + require.NoError(t, err) + + output := buf.String() + assert.Contains(t, output, "EVALUATOR") + assert.Contains(t, output, "CONTROLS") + assert.Contains(t, output, "openscap") + assert.Contains(t, output, "42") +} + +func TestListOptions_Run_NoMetadataShowsDash(t *testing.T) { + chdirTemp(t) + writeWorkspaceConfig(t, minimalConfig) + + cacheDir := t.TempDir() + // Simulate pre-upgrade cache: policy state exists but no metadata + // fields are populated (all zero values). + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "policies/test-policy": { + Version: "v1.0", + Digest: "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + }, + }, + } + require.NoError(t, cache.SaveState(state, cacheDir)) + + var buf bytes.Buffer + o := &listOptions{ + Common: &Common{Output: Output{Out: &buf}}, + cacheDir: cacheDir, + } + err := o.run(context.Background()) + require.NoError(t, err) + + output := buf.String() + assert.Contains(t, output, "EVALUATOR") + assert.Contains(t, output, "CONTROLS") + // Data row should show "-" for evaluator and controls since + // no metadata fields are populated. + lines := strings.Split(strings.TrimSpace(output), "\n") + require.GreaterOrEqual(t, len(lines), 2) + dataRow := lines[1] + // The data row should NOT contain "openscap" or numeric controls. + assert.NotContains(t, dataRow, "openscap") +} + +// multiPolicyListConfig has two policies for list --policy-id filter tests. +const multiPolicyListConfig = `policies: + - url: registry.example.com/policies/nist-policy:v1.0 + id: nist-policy + - url: registry.example.com/policies/cis-policy:v2.0 + id: cis-policy +targets: + - id: local + policies: + - nist-policy + variables: + profile: test +` + +func TestListOptions_Run_PolicyIDFilter(t *testing.T) { + chdirTemp(t) + writeWorkspaceConfig(t, multiPolicyListConfig) + + cacheDir := t.TempDir() + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "policies/nist-policy": { + Version: "v1.0", + Digest: "sha256:aaa111222333444555666777888999000aaabbbcccdddeeefff0001112223334", + PolicyEvaluator: "openscap", + ControlCount: 100, + }, + "policies/cis-policy": { + Version: "v2.0", + Digest: "sha256:bbb444555666777888999000aaabbbcccdddeeefff0001112223344556677889", + PolicyEvaluator: "opa", + ControlCount: 50, + }, + }, + } + require.NoError(t, cache.SaveState(state, cacheDir)) + + var buf bytes.Buffer + o := &listOptions{ + Common: &Common{Output: Output{Out: &buf}}, + cacheDir: cacheDir, + policyID: "cis-policy", + } + err := o.run(context.Background()) + require.NoError(t, err) + + output := buf.String() + // Only cis-policy should appear. + assert.Contains(t, output, "cis-policy") + assert.Contains(t, output, "opa") + assert.Contains(t, output, "50") + // nist-policy should NOT appear. + assert.NotContains(t, output, "nist-policy") + assert.NotContains(t, output, "openscap") +} + +func TestListOptions_Run_MultiEvaluatorShowsDash(t *testing.T) { + chdirTemp(t) + writeWorkspaceConfig(t, minimalConfig) + + cacheDir := t.TempDir() + // Multi-evaluator policy: PolicyEvaluator is empty but + // ControlCount is populated (metadata exists). + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "policies/test-policy": { + Version: "v1.0", + Digest: "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + PolicyTitle: "Multi-Eval Policy", + ControlCount: 25, + }, + }, + } + require.NoError(t, cache.SaveState(state, cacheDir)) + + var buf bytes.Buffer + o := &listOptions{ + Common: &Common{Output: Output{Out: &buf}}, + cacheDir: cacheDir, + } + err := o.run(context.Background()) + require.NoError(t, err) + + output := buf.String() + // Controls should show "25" since metadata exists. + assert.Contains(t, output, "25") + // The data row should show "-" for evaluator (multi-evaluator). + lines := strings.Split(strings.TrimSpace(output), "\n") + require.GreaterOrEqual(t, len(lines), 2) + dataRow := lines[1] + // Parse the EVALUATOR column position from the header and + // verify the data row has "-" in that position. + header := lines[0] + evalIdx := strings.Index(header, "EVALUATOR") + require.GreaterOrEqual(t, evalIdx, 0, + "header must contain EVALUATOR column") + ctrlIdx := strings.Index(header, "CONTROLS") + require.Greater(t, ctrlIdx, evalIdx) + evalField := strings.TrimSpace( + dataRow[evalIdx:ctrlIdx]) + assert.Equal(t, "-", evalField, + "multi-evaluator policy should show '-' for evaluator") +} + +func TestListOptions_Run_ZeroControlsWithMetadata(t *testing.T) { + chdirTemp(t) + writeWorkspaceConfig(t, minimalConfig) + + cacheDir := t.TempDir() + // Valid metadata with zero controls — should show "0", not "-". + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "policies/test-policy": { + Version: "v1.0", + Digest: "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + PolicyEvaluator: "opa", + ControlCount: 0, + }, + }, + } + require.NoError(t, cache.SaveState(state, cacheDir)) + + var buf bytes.Buffer + o := &listOptions{ + Common: &Common{Output: Output{Out: &buf}}, + cacheDir: cacheDir, + } + err := o.run(context.Background()) + require.NoError(t, err) + + output := buf.String() + assert.Contains(t, output, "opa") + // Should show "0" for controls, not "-", because metadata exists + // (PolicyEvaluator is non-empty). + lines := strings.Split(strings.TrimSpace(output), "\n") + require.GreaterOrEqual(t, len(lines), 2) + dataRow := lines[1] + assert.Contains(t, dataRow, "0") +} + +func TestPolicyMetadataFields_NoPolicyState(t *testing.T) { + state := &cache.State{ + Policies: make(map[string]cache.PolicyState), + } + eval, ctrl := policyMetadataFields(state, "nonexistent") + assert.Equal(t, "-", eval) + assert.Equal(t, "-", ctrl) +} + +func TestPolicyMetadataFields_NoMetadata(t *testing.T) { + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "repo": {Version: "v1.0", Digest: "sha256:abc123def4567890123456789012345678901234567890123456789012340000"}, + }, + } + eval, ctrl := policyMetadataFields(state, "repo") + assert.Equal(t, "-", eval) + assert.Equal(t, "-", ctrl) +} + +func TestPolicyMetadataFields_WithEvaluator(t *testing.T) { + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "repo": { + Version: "v1.0", + PolicyEvaluator: "openscap", + ControlCount: 42, + }, + }, + } + eval, ctrl := policyMetadataFields(state, "repo") + assert.Equal(t, "openscap", eval) + assert.Equal(t, "42", ctrl) +} + +func TestPolicyMetadataFields_MultiEvaluator(t *testing.T) { + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "repo": { + Version: "v1.0", + PolicyTitle: "Multi-Eval", + ControlCount: 10, + }, + }, + } + eval, ctrl := policyMetadataFields(state, "repo") + assert.Equal(t, "-", eval) + assert.Equal(t, "10", ctrl) +} + +func TestPolicyMetadataFields_ZeroControlsWithEvaluator(t *testing.T) { + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "repo": { + Version: "v1.0", + PolicyEvaluator: "opa", + ControlCount: 0, + }, + }, + } + eval, ctrl := policyMetadataFields(state, "repo") + assert.Equal(t, "opa", eval) + assert.Equal(t, "0", ctrl) +} + +func TestHasMetadata_AllZero(t *testing.T) { + ps := cache.PolicyState{Version: "v1.0", Digest: "sha256:abc123def4567890123456789012345678901234567890123456789012340000"} + assert.False(t, hasMetadata(ps)) +} + +func TestHasMetadata_TitleOnly(t *testing.T) { + ps := cache.PolicyState{PolicyTitle: "My Policy"} + assert.True(t, hasMetadata(ps)) +} + +func TestHasMetadata_EvaluatorOnly(t *testing.T) { + ps := cache.PolicyState{PolicyEvaluator: "opa"} + assert.True(t, hasMetadata(ps)) +} + +func TestHasMetadata_ControlCountOnly(t *testing.T) { + ps := cache.PolicyState{ControlCount: 5} + assert.True(t, hasMetadata(ps)) +} + +func TestHasMetadata_AssessmentCountOnly(t *testing.T) { + ps := cache.PolicyState{AssessmentCount: 3} + assert.True(t, hasMetadata(ps)) } func TestAbbreviateDigest(t *testing.T) { @@ -2194,3 +2540,81 @@ func TestEnableDebug_NoColor(t *testing.T) { assert.NotContains(t, output.String(), "\x1b[", "output should not contain ANSI escape sequences when NO_COLOR is set") } + +// --- formatPolicySummary tests --- + +func TestFormatPolicySummary_TitleAndEvaluator(t *testing.T) { + meta := policy.PolicyMetadata{ + Title: "CIS Fedora Linux", + EvaluatorID: "openscap", + ControlCount: 42, + AssessmentCount: 38, + } + result := formatPolicySummary(meta) + assert.Contains(t, result, "Policy: CIS Fedora Linux (openscap)") + assert.Contains(t, result, "Controls: 42 | Assessments: 38") +} + +func TestFormatPolicySummary_TitleOnly(t *testing.T) { + meta := policy.PolicyMetadata{ + Title: "Multi-Evaluator Policy", + EvaluatorID: "", + ControlCount: 10, + AssessmentCount: 5, + } + result := formatPolicySummary(meta) + assert.Contains(t, result, "Policy: Multi-Evaluator Policy") + assert.NotContains(t, result, "()") + assert.Contains(t, result, "Controls: 10 | Assessments: 5") +} + +func TestFormatPolicySummary_EvaluatorOnly(t *testing.T) { + meta := policy.PolicyMetadata{ + Title: "", + EvaluatorID: "opa", + ControlCount: 7, + AssessmentCount: 3, + } + result := formatPolicySummary(meta) + assert.Contains(t, result, "Policy: opa") + assert.NotContains(t, result, "(opa)") + assert.Contains(t, result, "Controls: 7 | Assessments: 3") +} + +func TestFormatPolicySummary_BothEmpty(t *testing.T) { + meta := policy.PolicyMetadata{ + Title: "", + EvaluatorID: "", + ControlCount: 0, + AssessmentCount: 0, + } + result := formatPolicySummary(meta) + assert.Empty(t, result, + "should return empty string when no metadata is available") +} + +func TestFormatPolicySummary_ZeroCounts(t *testing.T) { + meta := policy.PolicyMetadata{ + Title: "Empty Policy", + EvaluatorID: "ampel", + ControlCount: 0, + AssessmentCount: 0, + } + result := formatPolicySummary(meta) + assert.Contains(t, result, "Policy: Empty Policy (ampel)") + assert.Contains(t, result, "Controls: 0 | Assessments: 0") +} + +func TestFormatPolicySummary_CountsOnlyNoTitleNoEval(t *testing.T) { + // Edge case: counts are non-zero but title and evaluator + // are empty. Should still show the counts line. + meta := policy.PolicyMetadata{ + Title: "", + EvaluatorID: "", + ControlCount: 5, + AssessmentCount: 3, + } + result := formatPolicySummary(meta) + assert.Contains(t, result, "Controls: 5 | Assessments: 3") + assert.NotContains(t, result, "Policy:") +} diff --git a/cmd/complyctl/cli/get.go b/cmd/complyctl/cli/get.go index f62cbead..07600d72 100644 --- a/cmd/complyctl/cli/get.go +++ b/cmd/complyctl/cli/get.go @@ -291,6 +291,8 @@ func (o *getOptions) syncComplypacks( // syncAllPolicies iterates all policy entries, resolving per-entry // verification and collecting errors (D5: errors.Join). All entries // are attempted regardless of individual failures (FR-006). +// Creates a policy.Resolver once (D9) and passes it to each +// syncSinglePolicy call for metadata extraction. func syncAllPolicies( ctx context.Context, cacheMgr *cache.Cache, @@ -303,12 +305,18 @@ func syncAllPolicies( logger.Info("Starting policy synchronization", "policy_count", len(policies)) + // Create resolver once per invocation (D9) for metadata + // extraction in syncSinglePolicy. + loader := policy.NewLoader(cacheMgr) + resolver := policy.NewResolver(loader) + total := len(policies) var errs []error for i, entry := range policies { if err := syncSinglePolicy( ctx, cacheMgr, state, credFunc, entry, i+1, total, wsCfg, vfCache, + resolver, ); err != nil { eid := entry.EffectiveID() fmt.Fprintf(os.Stderr, @@ -319,6 +327,15 @@ func syncAllPolicies( } } + // Batch-flush metadata state after all policies are synced, + // rather than writing to disk per-policy. + if saveErr := cache.SaveState( + state, cacheMgr.Dir(), + ); saveErr != nil { + logger.Warn("Failed to save metadata state", + "error", saveErr) + } + if len(errs) > 0 { return errors.Join(errs...) } @@ -337,6 +354,7 @@ func syncSinglePolicy( index, total int, wsCfg *complytime.VerificationConfig, vfCache map[complytime.VerificationConfig]cache.VerifyFunc, + resolver *policy.Resolver, ) error { ref, err := complytime.ParsePolicyRef(entry.URL) if err != nil { @@ -395,9 +413,77 @@ func syncSinglePolicy( "policy", entry.EffectiveID()) } + // Extract and cache metadata after sync (D7 ordering). + // Needed for fresh fetches or upgrade backfill (D8). + // hasMetadata() (not just PolicyEvaluator=="") avoids redundant + // re-extraction for multi-evaluator policies where PolicyEvaluator + // is intentionally empty. + ps, _ := state.GetPolicyState(ref.Repository) + extractMetadata := fetched || !hasMetadata(ps) + + if extractMetadata { + // Use the concrete version SyncPolicy persisted, not the config + // tag: a ":latest" pin is stored under the resolved version, so + // extracting by "latest" would fail store.Resolve() and silently + // drop all metadata. + meta, metaErr := resolver.ExtractPolicyMetadata( + ref.Repository, ps.Version, + ) + if metaErr != nil { + // D5: metadata extraction failure is non-fatal. + logger.Warn( + "Failed to extract policy metadata", + "policy", entry.EffectiveID(), + "error", metaErr, + ) + } else { + state.SetPolicyMetadata( + ref.Repository, + meta.Title, + meta.EvaluatorID, + meta.ControlCount, + meta.AssessmentCount, + ) + summary := formatPolicySummary(meta) + if summary != "" { + fmt.Fprint(os.Stderr, summary) + } + } + } + return nil } +// formatPolicySummary builds a human-readable summary string from +// extracted policy metadata. Returns empty string when no meaningful +// content is available to display. +func formatPolicySummary(meta policy.PolicyMetadata) string { + var sb strings.Builder + + // Build the "Policy:" line based on available fields. + hasTitle := meta.Title != "" + hasEval := meta.EvaluatorID != "" + switch { + case hasTitle && hasEval: + fmt.Fprintf(&sb, " Policy: %s (%s)\n", + meta.Title, meta.EvaluatorID) + case hasTitle: + fmt.Fprintf(&sb, " Policy: %s\n", meta.Title) + case hasEval: + fmt.Fprintf(&sb, " Policy: %s\n", meta.EvaluatorID) + } + + // Always print counts when we have metadata to show. + if hasTitle || hasEval || + meta.ControlCount > 0 || meta.AssessmentCount > 0 { + fmt.Fprintf(&sb, + " Controls: %d | Assessments: %d\n", + meta.ControlCount, meta.AssessmentCount) + } + + return sb.String() +} + func resolveLatestVersion(ctx context.Context, client *registry.Client, repository, policyID string) string { logger.Info("Resolving latest version", "policy", policyID) _, resolvedVersion, resolveErr := client.DefinitionVersion(ctx, repository) diff --git a/cmd/complyctl/cli/list.go b/cmd/complyctl/cli/list.go index 6b21e92e..31d41543 100644 --- a/cmd/complyctl/cli/list.go +++ b/cmd/complyctl/cli/list.go @@ -7,6 +7,7 @@ import ( "fmt" "io" "sort" + "strconv" "strings" "github.com/spf13/cobra" @@ -107,14 +108,53 @@ func (o *listOptions) run(_ context.Context) error { versionStr = "-" } + evaluatorStr, controlsStr := policyMetadataFields( + state, ref.Repository, + ) digestStr := policyDigestField(state, ref.Repository) verifiedStr := verificationStatus(state, ref.Repository) - rows = append(rows, []string{eid, versionStr, digestStr, verifiedStr}) + rows = append(rows, []string{ + eid, versionStr, evaluatorStr, controlsStr, + digestStr, verifiedStr, + }) } return printGemaraPolicyTable(o.Out, rows) } +// policyMetadataFields returns the EVALUATOR and CONTROLS column +// values for a policy from the cache state. Returns "-" for both +// when the policy has no cached state or metadata has not been +// populated (pre-upgrade cache without metadata fields). +func policyMetadataFields( + state *cache.State, repository string, +) (evaluator, controls string) { + ps, ok := state.GetPolicyState(repository) + if !ok { + return "-", "-" + } + if !hasMetadata(ps) { + return "-", "-" + } + evaluator = ps.PolicyEvaluator + if evaluator == "" { + evaluator = "-" + } + controls = strconv.Itoa(ps.ControlCount) + return evaluator, controls +} + +// hasMetadata returns true when any display-oriented metadata field +// on the PolicyState has been populated. This distinguishes a +// pre-upgrade cache (all zero values) from a policy that was synced +// with metadata extraction. +func hasMetadata(ps cache.PolicyState) bool { + return ps.PolicyTitle != "" || + ps.PolicyEvaluator != "" || + ps.ControlCount > 0 || + ps.AssessmentCount > 0 +} + // policyDigestField returns the abbreviated digest for a policy from the // cache state. Returns "-" when the policy has no cached state. func policyDigestField(state *cache.State, repository string) string { @@ -146,7 +186,10 @@ func abbreviateDigest(dgst string) string { func printGemaraPolicyTable(w io.Writer, rows [][]string) error { sort.SliceStable(rows, func(i, j int) bool { return rows[i][0] < rows[j][0] }) - headers := []string{"POLICY ID", "VERSION", "DIGEST", "VERIFIED"} + headers := []string{ + "POLICY ID", "VERSION", "EVALUATOR", "CONTROLS", + "DIGEST", "VERIFIED", + } terminal.ShowPlainTable(w, headers, rows) return nil } diff --git a/internal/cache/state.go b/internal/cache/state.go index 7f003afa..449ec35b 100644 --- a/internal/cache/state.go +++ b/internal/cache/state.go @@ -20,8 +20,11 @@ type State struct { Complypacks map[string]PolicyState `json:"complypacks,omitempty"` } -// PolicyState holds version, digest, verification status, and timestamp for -// a single cached policy or complypack. +// PolicyState holds version, digest, verification status, timestamp, and +// display-oriented metadata for a single cached policy or complypack. +// The metadata fields (PolicyTitle, PolicyEvaluator, ControlCount, +// AssessmentCount) are populated at sync time by ExtractPolicyMetadata +// and are used by the list and get commands for display purposes. type PolicyState struct { Version string `json:"version"` Digest string `json:"digest"` @@ -31,6 +34,12 @@ type PolicyState struct { SignerIdentity string `json:"signer_identity,omitempty"` Issuer string `json:"issuer,omitempty"` VerifiedAt time.Time `json:"verified_at,omitempty"` + + // Display-oriented metadata extracted from Gemara policy YAML. + PolicyTitle string `json:"policy_title,omitempty"` + PolicyEvaluator string `json:"policy_evaluator,omitempty"` + ControlCount int `json:"control_count"` + AssessmentCount int `json:"assessment_count"` } // LoadState reads and parses the state.json file from the given cache directory. @@ -186,3 +195,21 @@ func (s *State) EvaluatorIDToVersion(evaluatorID string) (string, bool, error) { } return version, found, nil } + +// SetPolicyMetadata updates display-oriented metadata fields on an +// existing PolicyState entry without overwriting sync fields. No-ops +// when the repository key does not exist in the Policies map. +func (s *State) SetPolicyMetadata( + repository, title, evaluator string, + controls, assessments int, +) { + ps, exists := s.Policies[repository] + if !exists { + return + } + ps.PolicyTitle = title + ps.PolicyEvaluator = evaluator + ps.ControlCount = controls + ps.AssessmentCount = assessments + s.Policies[repository] = ps +} diff --git a/internal/cache/state_test.go b/internal/cache/state_test.go index d956db16..f6f6ea79 100644 --- a/internal/cache/state_test.go +++ b/internal/cache/state_test.go @@ -7,6 +7,7 @@ import ( "os" "path/filepath" "testing" + "time" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -129,3 +130,159 @@ func TestPolicyState_BackwardCompatibility_NoVerifiedField(t *testing.T) { assert.Equal(t, "v1.0.0", ps.Version) assert.Equal(t, "sha256:legacy", ps.Digest) } + +func TestPolicyState_JSONRoundTrip_WithMetadata(t *testing.T) { + original := cache.PolicyState{ + Version: "v2.0.0", + Digest: "sha256:abc123", + EvaluatorID: "openscap", + LastUpdated: time.Date(2026, 7, 10, 12, 0, 0, 0, time.UTC), + Verified: true, + SignerIdentity: "user@example.com", + Issuer: "https://accounts.google.com", + VerifiedAt: time.Date(2026, 7, 10, 12, 0, 0, 0, time.UTC), + PolicyTitle: "CIS Fedora Linux - Level 1", + PolicyEvaluator: "openscap", + ControlCount: 42, + AssessmentCount: 5, + } + + data, err := json.Marshal(original) + require.NoError(t, err) + + var roundTripped cache.PolicyState + err = json.Unmarshal(data, &roundTripped) + require.NoError(t, err) + + assert.Equal(t, original, roundTripped) +} + +func TestPolicyState_JSONBackwardCompatibility(t *testing.T) { + // JSON from an older complyctl version that lacks metadata fields. + oldJSON := `{ + "version": "v1.0.0", + "digest": "sha256:old", + "last_updated": "2025-06-01T00:00:00Z" + }` + + var ps cache.PolicyState + err := json.Unmarshal([]byte(oldJSON), &ps) + require.NoError(t, err) + + assert.Equal(t, "v1.0.0", ps.Version) + assert.Equal(t, "sha256:old", ps.Digest) + // Metadata fields default to zero values. + assert.Equal(t, "", ps.PolicyTitle) + assert.Equal(t, "", ps.PolicyEvaluator) + assert.Equal(t, 0, ps.ControlCount) + assert.Equal(t, 0, ps.AssessmentCount) +} + +func TestPolicyState_JSONRoundTrip_ZeroControlCount(t *testing.T) { + // Verify that ControlCount:0 survives a JSON round-trip. + // Without omitempty the field is explicitly serialized as 0, + // so "metadata extracted with 0 controls" is distinguishable + // from "metadata never extracted" after deserialization. + original := cache.PolicyState{ + Version: "v1.0.0", + Digest: "sha256:abc", + PolicyTitle: "Zero Controls Policy", + PolicyEvaluator: "opa", + ControlCount: 0, + AssessmentCount: 1, + } + + data, err := json.Marshal(original) + require.NoError(t, err) + + // Confirm control_count is present in JSON (explicit zero). + assert.Contains(t, string(data), `"control_count":0`) + + var roundTripped cache.PolicyState + err = json.Unmarshal(data, &roundTripped) + require.NoError(t, err) + + assert.Equal(t, 0, roundTripped.ControlCount) + assert.Equal(t, "opa", roundTripped.PolicyEvaluator) + assert.Equal(t, 1, roundTripped.AssessmentCount) +} + +func TestState_SetPolicyMetadata_PreservesSyncFields(t *testing.T) { + verifiedAt := time.Date(2026, 7, 1, 10, 0, 0, 0, time.UTC) + lastUpdated := time.Date(2026, 7, 1, 9, 0, 0, 0, time.UTC) + + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "example.com/policies/cis-fedora": { + Version: "v3.1.0", + Digest: "sha256:syncdigest", + EvaluatorID: "complypack-eval", + LastUpdated: lastUpdated, + Verified: true, + SignerIdentity: "signer@example.com", + Issuer: "https://issuer.example.com", + VerifiedAt: verifiedAt, + }, + }, + } + + state.SetPolicyMetadata( + "example.com/policies/cis-fedora", + "CIS Fedora Linux", + "openscap", + 42, + 5, + ) + + ps, exists := state.GetPolicyState( + "example.com/policies/cis-fedora", + ) + require.True(t, exists) + + // Sync fields MUST be unchanged. + assert.Equal(t, "v3.1.0", ps.Version) + assert.Equal(t, "sha256:syncdigest", ps.Digest) + assert.Equal(t, "complypack-eval", ps.EvaluatorID) + assert.Equal(t, lastUpdated, ps.LastUpdated) + assert.True(t, ps.Verified) + assert.Equal(t, "signer@example.com", ps.SignerIdentity) + assert.Equal(t, "https://issuer.example.com", ps.Issuer) + assert.Equal(t, verifiedAt, ps.VerifiedAt) + + // Metadata fields MUST be set. + assert.Equal(t, "CIS Fedora Linux", ps.PolicyTitle) + assert.Equal(t, "openscap", ps.PolicyEvaluator) + assert.Equal(t, 42, ps.ControlCount) + assert.Equal(t, 5, ps.AssessmentCount) +} + +func TestState_SetPolicyMetadata_NoOpForMissingKey(t *testing.T) { + state := &cache.State{ + Policies: map[string]cache.PolicyState{ + "existing-policy": { + Version: "v1.0.0", + Digest: "sha256:exists", + }, + }, + } + + // Call with a key that does not exist — must not panic. + state.SetPolicyMetadata( + "nonexistent-policy", + "Some Title", + "ampel", + 10, + 3, + ) + + // Map must be unchanged: only the original entry exists. + assert.Len(t, state.Policies, 1) + _, exists := state.GetPolicyState("nonexistent-policy") + assert.False(t, exists) + + // Original entry must be untouched. + ps, exists := state.GetPolicyState("existing-policy") + require.True(t, exists) + assert.Equal(t, "v1.0.0", ps.Version) + assert.Equal(t, "sha256:exists", ps.Digest) +} diff --git a/internal/policy/resolver.go b/internal/policy/resolver.go index 77477def..5d89092e 100644 --- a/internal/policy/resolver.go +++ b/internal/policy/resolver.go @@ -57,6 +57,17 @@ type PolicyTimeline struct { EnforcementNotes string } +// PolicyMetadata holds display-oriented metadata extracted from a +// cached Gemara policy, suitable for CLI summary and list output. +// EvaluatorID maps to PolicyState.PolicyEvaluator when persisted +// (not PolicyState.EvaluatorID, which is reserved for complypacks). +type PolicyMetadata struct { + Title string + EvaluatorID string + ControlCount int + AssessmentCount int +} + // PolicyLoader abstracts the Loader methods used by Resolver, enabling // mock injection for unit tests without coupling to OCI store internals. type PolicyLoader interface { @@ -85,6 +96,130 @@ func (r *Resolver) ResolveVersion(policyID, configVersion string) (string, error return r.loader.ResolveVersion(policyID, configVersion) } +// ExtractPolicyMetadata extracts display-oriented metadata from a +// cached policy without building a full DependencyGraph. The policyID +// parameter is the OCI repository path (cache key), following the +// same convention as ResolvePolicyGraph. +func (r *Resolver) ExtractPolicyMetadata( + policyID, version string, +) (PolicyMetadata, error) { + if policyID == "" { + return PolicyMetadata{}, fmt.Errorf( + "policy ID cannot be empty") + } + if version == "" { + return PolicyMetadata{}, fmt.Errorf( + "version cannot be empty") + } + if !r.loader.PolicyExists(policyID, version) { + return PolicyMetadata{}, fmt.Errorf( + "policy not found: %s@%s", policyID, version) + } + + isBundle, detectErr := r.loader.DetectManifestShape( + policyID, version) + if detectErr != nil { + return PolicyMetadata{}, fmt.Errorf( + "failed to detect manifest shape for %s@%s: %w", + policyID, version, detectErr) + } + + if isBundle { + return r.extractBundleMetadata(policyID, version) + } + return r.extractSplitMetadata(policyID, version) +} + +// extractBundleMetadata extracts metadata from a Gemara bundle-format +// policy. Parses only the Policy and ControlCatalog artifacts — +// guidance is not needed for display metadata. +func (r *Resolver) extractBundleMetadata( + policyID, version string, +) (PolicyMetadata, error) { + files, err := r.loader.LoadBundleFiles(policyID, version) + if err != nil { + return PolicyMetadata{}, fmt.Errorf( + "bundle unpack failed for %s@%s: %w", + policyID, version, err) + } + + policyData, ok := files["Policy"] + if !ok { + return PolicyMetadata{}, fmt.Errorf( + "bundle for %s@%s: missing required Policy artifact", + policyID, version) + } + + policyResult, parseErr := parsePolicyLayer( + policyID, policyData) + if parseErr != nil { + return PolicyMetadata{}, fmt.Errorf( + "failed to parse policy layer for %s: %w", + policyID, parseErr) + } + + meta := PolicyMetadata{ + Title: policyResult.Title, + EvaluatorID: policyResult.EvaluatorID, + AssessmentCount: len(policyResult.Assessments), + } + + if catalogData, catalogOK := files["ControlCatalog"]; catalogOK { + catalog, catErr := parseControlCatalog(catalogData) + if catErr != nil { + return PolicyMetadata{}, fmt.Errorf( + "policy %s: catalog layer is not valid Gemara: %w", + policyID, catErr) + } + meta.ControlCount = len(catalog.Controls) + } + + return meta, nil +} + +// extractSplitMetadata extracts metadata from a split-layer format +// policy. Parses only the Policy and ControlCatalog layers — +// guidance is not needed for display metadata. +func (r *Resolver) extractSplitMetadata( + policyID, version string, +) (PolicyMetadata, error) { + policyData, policyLoadErr := r.loader.LoadLayerByMediaType( + policyID, version, complytime.MediaTypePolicy) + if policyLoadErr != nil { + return PolicyMetadata{}, fmt.Errorf( + "failed to load policy layer for %s@%s: %w", + policyID, version, policyLoadErr) + } + + policyResult, parseErr := parsePolicyLayer( + policyID, policyData) + if parseErr != nil { + return PolicyMetadata{}, fmt.Errorf( + "failed to parse policy layer for %s: %w", + policyID, parseErr) + } + + meta := PolicyMetadata{ + Title: policyResult.Title, + EvaluatorID: policyResult.EvaluatorID, + AssessmentCount: len(policyResult.Assessments), + } + + catalogData, catalogLoadErr := r.loader.LoadLayerByMediaType( + policyID, version, complytime.MediaTypeCatalog) + if catalogLoadErr == nil { + catalog, catErr := parseControlCatalog(catalogData) + if catErr != nil { + return PolicyMetadata{}, fmt.Errorf( + "policy %s: catalog layer is not valid Gemara: %w", + policyID, catErr) + } + meta.ControlCount = len(catalog.Controls) + } + + return meta, nil +} + // ResolvePolicyGraph builds a DependencyGraph from cached OCI layers. // It detects the manifest shape (bundle vs split-layer) and delegates // to the appropriate loading path. @@ -238,6 +373,7 @@ func parseGuidanceCatalog(data []byte) (*gemara.GuidanceCatalog, error) { } type policyLayerResult struct { + Title string EvaluatorID string Assessments []Assessment Timeline *PolicyTimeline @@ -306,6 +442,7 @@ func extractFromGemaraPolicy(p *gemara.Policy) policyLayerResult { } return policyLayerResult{ + Title: p.Title, EvaluatorID: resultEvalID, Assessments: assessments, Timeline: timeline, diff --git a/internal/policy/resolver_test.go b/internal/policy/resolver_test.go index 4ad3d0a9..6ebc4e07 100644 --- a/internal/policy/resolver_test.go +++ b/internal/policy/resolver_test.go @@ -714,6 +714,300 @@ guidelines: [] assert.Equal(t, "guide-parsed", graph.Guidelines[0].Parsed.Metadata.Id) } +// --- ExtractPolicyMetadata tests --- + +func TestResolver_ExtractPolicyMetadata_EmptyPolicyID(t *testing.T) { + r := NewResolver(newMockLoader()) + _, err := r.ExtractPolicyMetadata("", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), "policy ID cannot be empty") +} + +func TestResolver_ExtractPolicyMetadata_EmptyVersion(t *testing.T) { + r := NewResolver(newMockLoader()) + _, err := r.ExtractPolicyMetadata("test-policy", "") + require.Error(t, err) + assert.Contains(t, err.Error(), "version cannot be empty") +} + +func TestResolver_ExtractPolicyMetadata_PolicyNotInCache(t *testing.T) { + r := NewResolver(newMockLoader()) + _, err := r.ExtractPolicyMetadata("missing-policy", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), "policy not found") +} + +func TestResolver_ExtractPolicyMetadata_DetectShapeError(t *testing.T) { + ml := newMockLoader() + ml.exists["shape-err/v1"] = true + ml.bundleShapeErr["shape-err/v1"] = fmt.Errorf( + "corrupt manifest") + + r := NewResolver(ml) + _, err := r.ExtractPolicyMetadata("shape-err", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), + "failed to detect manifest shape") + assert.Contains(t, err.Error(), "corrupt manifest") +} + +func TestResolver_ExtractPolicyMetadata_BundleMissingPolicyArtifact( + t *testing.T, +) { + ml := newMockLoader() + ml.exists["no-policy/v1"] = true + ml.bundleShape["no-policy/v1"] = true + ml.bundleFiles["no-policy/v1"] = map[string][]byte{ + "ControlCatalog": validCatalogYAML(), + } + + r := NewResolver(ml) + _, err := r.ExtractPolicyMetadata("no-policy", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), + "missing required Policy artifact") +} + +func TestResolver_ExtractPolicyMetadata_BundleInvalidCatalog( + t *testing.T, +) { + ml := newMockLoader() + ml.exists["bad-cat/v1"] = true + ml.bundleShape["bad-cat/v1"] = true + ml.bundleFiles["bad-cat/v1"] = map[string][]byte{ + "Policy": validPolicyYAML(), + "ControlCatalog": []byte("{not: valid: catalog: [}"), + } + + r := NewResolver(ml) + _, err := r.ExtractPolicyMetadata("bad-cat", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), + "catalog layer is not valid Gemara") +} + +func TestResolver_ExtractPolicyMetadata_SplitPolicyLoadError( + t *testing.T, +) { + ml := newMockLoader() + ml.exists["split-err/v1"] = true + // bundleShape defaults to false (split-layer). + // No policy layer registered -> LoadLayerByMediaType fails. + + r := NewResolver(ml) + _, err := r.ExtractPolicyMetadata("split-err", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), + "failed to load policy layer") +} + +func TestResolver_ExtractPolicyMetadata_SplitMissingCatalog( + t *testing.T, +) { + ml := newMockLoader() + ml.exists["split-nocat/v1"] = true + // bundleShape defaults to false (split-layer). + policyMediaType := "application/vnd.gemara.policy.v1+yaml" + ml.layers["split-nocat/v1/"+policyMediaType] = validPolicyYAML() + // No catalog layer registered -> catalog load fails silently. + + r := NewResolver(ml) + meta, err := r.ExtractPolicyMetadata("split-nocat", "v1") + require.NoError(t, err) + assert.Equal(t, "Test Policy", meta.Title) + assert.Equal(t, "openscap", meta.EvaluatorID) + assert.Equal(t, 0, meta.ControlCount, + "missing catalog should yield zero controls") + assert.Equal(t, 1, meta.AssessmentCount) +} + +func TestResolver_ExtractPolicyMetadata_BundleFormat(t *testing.T) { + ml := newMockLoader() + ml.exists["bundle-meta/v1"] = true + ml.bundleShape["bundle-meta/v1"] = true + ml.bundleFiles["bundle-meta/v1"] = map[string][]byte{ + "Policy": validPolicyYAML(), + "ControlCatalog": validCatalogYAML(), + } + + r := NewResolver(ml) + meta, err := r.ExtractPolicyMetadata("bundle-meta", "v1") + require.NoError(t, err) + assert.Equal(t, "Test Policy", meta.Title) + assert.Equal(t, "openscap", meta.EvaluatorID) + assert.Equal(t, 2, meta.ControlCount) + assert.Equal(t, 1, meta.AssessmentCount) +} + +func TestResolver_ExtractPolicyMetadata_SplitLayer(t *testing.T) { + ml := newMockLoader() + ml.exists["split-meta/v1"] = true + // bundleShape defaults to false (split-layer) + + policyMediaType := "application/vnd.gemara.policy.v1+yaml" + catalogMediaType := "application/vnd.gemara.catalog.v1+yaml" + + ml.layers["split-meta/v1/"+policyMediaType] = validPolicyYAML() + ml.layers["split-meta/v1/"+catalogMediaType] = validCatalogYAML() + + r := NewResolver(ml) + meta, err := r.ExtractPolicyMetadata("split-meta", "v1") + require.NoError(t, err) + assert.Equal(t, "Test Policy", meta.Title) + assert.Equal(t, "openscap", meta.EvaluatorID) + assert.Equal(t, 2, meta.ControlCount) + assert.Equal(t, 1, meta.AssessmentCount) +} + +func TestResolver_ExtractPolicyMetadata_MissingCatalog(t *testing.T) { + ml := newMockLoader() + ml.exists["no-catalog/v1"] = true + ml.bundleShape["no-catalog/v1"] = true + ml.bundleFiles["no-catalog/v1"] = map[string][]byte{ + "Policy": validPolicyYAML(), + } + + r := NewResolver(ml) + meta, err := r.ExtractPolicyMetadata("no-catalog", "v1") + require.NoError(t, err) + assert.Equal(t, "Test Policy", meta.Title) + assert.Equal(t, "openscap", meta.EvaluatorID) + assert.Equal(t, 0, meta.ControlCount) + assert.Equal(t, 1, meta.AssessmentCount) +} + +func TestResolver_ExtractPolicyMetadata_MultiEvaluator(t *testing.T) { + multiEvalPolicy := []byte(` +title: Multi Evaluator Policy +metadata: + id: pol-multi + version: "1.0" +contacts: + responsible: + - name: team-a + accountable: + - name: team-b +scope: + in: + technologies: + - linux +imports: + catalogs: + - reference-id: cat-1 +adherence: + assessment-plans: + - id: ap-1 + requirement-id: req-1 + frequency: daily + evaluation-methods: + - type: Behavioral + executor: + id: openscap + - id: ap-2 + requirement-id: req-2 + frequency: weekly + evaluation-methods: + - type: Behavioral + executor: + id: opa +`) + + ml := newMockLoader() + ml.exists["multi-eval/v1"] = true + ml.bundleShape["multi-eval/v1"] = true + ml.bundleFiles["multi-eval/v1"] = map[string][]byte{ + "Policy": multiEvalPolicy, + } + + r := NewResolver(ml) + meta, err := r.ExtractPolicyMetadata("multi-eval", "v1") + require.NoError(t, err) + assert.Equal(t, "Multi Evaluator Policy", meta.Title) + assert.Empty(t, meta.EvaluatorID, + "multi-evaluator policy should have empty EvaluatorID") + assert.Equal(t, 2, meta.AssessmentCount) +} + +func TestResolver_ExtractPolicyMetadata_EmptyTitle(t *testing.T) { + emptyTitlePolicy := []byte(` +title: "" +metadata: + id: pol-notitle + version: "1.0" +contacts: + responsible: + - name: team-a + accountable: + - name: team-b +scope: + in: + technologies: + - linux +imports: + catalogs: + - reference-id: cat-1 +adherence: + assessment-plans: + - id: ap-1 + requirement-id: req-1 + frequency: daily + evaluation-methods: + - type: Behavioral + executor: + id: ampel +`) + + ml := newMockLoader() + ml.exists["empty-title/v1"] = true + ml.bundleShape["empty-title/v1"] = true + ml.bundleFiles["empty-title/v1"] = map[string][]byte{ + "Policy": emptyTitlePolicy, + } + + r := NewResolver(ml) + meta, err := r.ExtractPolicyMetadata("empty-title", "v1") + require.NoError(t, err) + assert.Empty(t, meta.Title) + assert.Equal(t, "ampel", meta.EvaluatorID) + assert.Equal(t, 1, meta.AssessmentCount) +} + +func TestResolver_ExtractPolicyMetadata_ParseError(t *testing.T) { + ml := newMockLoader() + ml.exists["bad-yaml/v1"] = true + ml.bundleShape["bad-yaml/v1"] = true + ml.bundleFiles["bad-yaml/v1"] = map[string][]byte{ + "Policy": []byte("{not: valid: yaml: [}"), + } + + r := NewResolver(ml) + _, err := r.ExtractPolicyMetadata("bad-yaml", "v1") + require.Error(t, err) + assert.Contains(t, err.Error(), "not valid Gemara Policy YAML") +} + +func validCatalogYAML() []byte { + return []byte(` +title: Test Catalog +metadata: + id: cat-1 + version: "1.0" +controls: + - id: ctrl-1 + title: First Control + objective: Test objective 1 + assessment-requirements: + - id: ar-1 + description: Check first + - id: ctrl-2 + title: Second Control + objective: Test objective 2 + assessment-requirements: + - id: ar-2 + description: Check second +`) +} + func validPolicyYAML() []byte { return []byte(` title: Test Policy diff --git a/openspec/changes/bundle-metadata-cli/.openspec.yaml b/openspec/changes/bundle-metadata-cli/.openspec.yaml new file mode 100644 index 00000000..9943806c --- /dev/null +++ b/openspec/changes/bundle-metadata-cli/.openspec.yaml @@ -0,0 +1,2 @@ +schema: unbound-force +created: 2026-07-09 diff --git a/openspec/changes/bundle-metadata-cli/design.md b/openspec/changes/bundle-metadata-cli/design.md new file mode 100644 index 00000000..2bbd5aa8 --- /dev/null +++ b/openspec/changes/bundle-metadata-cli/design.md @@ -0,0 +1,269 @@ +## Context + +After `complyctl get` fetches a Gemara bundle from an OCI +registry, the CLI retains only OCI-level metadata (digest, +version, verification status) in `state.json`. All policy +content metadata (title, evaluator, controls, assessments) is +discarded until `complyctl scan` parses the cached YAML via +`policy.Resolver.ResolvePolicyGraph()`. + +This means `complyctl list` shows only 4 columns (POLICY ID, +VERSION, DIGEST, VERIFIED) with no insight into what the +policy actually contains. Users must run `oras blob fetch` or +similar tools to discover the evaluator type, control count, +or policy title. + +The relevant code paths are: + +1. `syncSinglePolicy()` in `cmd/complyctl/cli/get.go` calls + `sync.SyncPolicy()` and updates `state.json` with version, + digest, and verification status. It never opens the policy + layers. +2. `policy.Resolver.ResolvePolicyGraph()` in + `internal/policy/resolver.go` parses all Gemara YAML and + builds a `DependencyGraph` with evaluator ID, assessments, + controls, and guidelines. This is only called during `scan`. +3. `listOptions.run()` in `cmd/complyctl/cli/list.go` reads + `complytime.yaml` and `state.json` to build the table. It + uses `policy.Loader` only for version tags, never for + content. +4. `parsePolicyLayer()` in `internal/policy/resolver.go` + unmarshals the full `gemara.Policy` struct (which has + `Title`, `Metadata`, etc.) but `extractFromGemaraPolicy()` + only extracts `EvaluatorID`, `Assessments`, and `Timeline` + -- discarding `Title` and other display-oriented metadata. + +## Goals / Non-Goals + +### Goals + +- Surface policy metadata (title, evaluator, control count, + assessment count) via `complyctl list` and `complyctl get` + summary +- Cache metadata in `state.json` at sync time so `list` does + not require YAML parsing or OCI store access +- Provide a lightweight extraction path that does not build a + full `DependencyGraph` +- Backfill metadata for pre-existing caches on upgrade + +### Non-Goals + +- Pre-pull discovery (`complyctl inspect `) -- deferred + to follow-up issue +- Policy title in `list` table (too long for terminal width) +- ID validation (warning when config `id` differs from bundle + policy ID) -- separate concern +- Changes to scan output, report formatters, or provider + protocol +- Changes to complypack metadata (already has `EvaluatorID` + in state) +- Machine-readable `list` output (e.g., `--output json`) -- + composability enhancement deferred to separate change + +## Decisions + +### D1: Cache metadata in `state.json` + +**Decision**: Add `policy_title`, `policy_evaluator`, +`control_count`, and `assessment_count` fields to +`PolicyState` with `omitempty` JSON tags. Populate at sync +time when a fresh policy is fetched. + +**Rationale**: `state.json` is already the metadata store +for policies. `list` already reads it. Adding fields is +backward compatible (Go's JSON unmarshalling ignores unknown +fields, `omitempty` avoids noise for old entries). The +alternative (on-demand parsing) would require opening OCI +stores, resolving manifests, and parsing YAML for every +policy on every `list` call. + +**Staleness risk**: Near-zero. The OCI layout directory is +an internal implementation detail managed exclusively by +`complyctl get`. The metadata is display-only -- `scan` +always re-parses fresh YAML via the Resolver. Any +`complyctl get` run refreshes the metadata. + +### D2: Lightweight `ExtractPolicyMetadata()` method + +**Decision**: Add `ExtractPolicyMetadata(policyID, version)` +to `policy.Resolver` that returns a `PolicyMetadata` struct +with `Title`, `EvaluatorID`, `ControlCount`, +`AssessmentCount`. The `policyID` parameter follows the +existing `ResolvePolicyGraph(policyID, version)` convention +where it represents the OCI repository path used as the +cache key (not the user-facing `EffectiveID()` alias). + +**Rationale**: Reuses existing YAML parsing functions +(`parsePolicyLayer`, `parseControlCatalog`) without building +a full `DependencyGraph`. Avoids parsing guidance catalogs +(not needed for display metadata). The method follows the +same bundle-vs-split-layer detection pattern as +`ResolvePolicyGraph()` but with a smaller output struct. + +**Alternative considered**: Extending `ResolvePolicyGraph()` +to return metadata. Rejected because it would couple display +metadata to the scan pipeline and parse guidance catalogs +unnecessarily. + +**Alternative considered**: Extracting metadata in `get.go` +directly (inline parsing). Rejected because it would +duplicate YAML unmarshalling logic that already lives in the +`policy` package, violating DRY. + +### D3: Post-sync summary on stderr + +**Decision**: After a fresh policy fetch, print a two-line +summary to stderr: + +``` + Policy: (<evaluator>) + Controls: <N> | Assessments: <M> +``` + +When the title is empty, omit the title portion and show +only the evaluator. When the evaluator is empty +(multi-evaluator policy), omit the evaluator parenthetical. + +**Rationale**: Gives immediate "what did I pull" feedback. +Matches existing stderr convention (all `get` output goes +to stderr). Two lines keep it concise. The title + evaluator +combination tells users whether they pulled the right policy +for their provider setup. + +**Alternative considered**: Printing a full table with all +metadata. Rejected as too verbose for a fetch operation -- +the detailed view is `list`. + +### D4: `list` columns: EVALUATOR and CONTROLS + +**Decision**: Add two columns to `complyctl list`: +- EVALUATOR: evaluator ID (e.g., "openscap", "ampel", "opa") +- CONTROLS: control count (e.g., "42") + +Column order: POLICY ID, VERSION, EVALUATOR, CONTROLS, +DIGEST, VERIFIED. + +**Rationale**: Evaluator ID is the most actionable metadata +-- it tells users which provider they need installed. Control +count gives a sense of policy scope. Both are short values +that fit in a terminal table. + +**Alternative considered**: Including POLICY NAME (title) +column. Rejected because policy titles are often 50+ +characters (e.g., "CIS Fedora Linux - Level 1 Workstation +Policy") and would blow out terminal width. The title is +shown in the `get` summary instead. + +**Alternative considered**: Including ASSESSMENTS column. +Deferred -- assessments and controls are often 1:1, and +adding too many columns makes the table hard to read. Can +be added later if needed. + +### D5: Metadata extraction failure is non-fatal + +**Decision**: If `ExtractPolicyMetadata()` fails (e.g., +corrupted YAML in cache), the error is logged as a warning +and the sync still succeeds. The metadata fields remain at +zero values, and `list` shows "-". + +**Rationale**: Metadata is display-only. A parse failure +should not prevent a successful OCI sync from being recorded. +The policy can still be used for `scan` (which has its own +error handling for parse failures). This follows the existing +pattern where unverified policies emit a NOTE but do not fail +the sync. + +### D6: Naming `PolicyEvaluator` vs reusing `EvaluatorID` + +**Decision**: Use `PolicyEvaluator` as the new field name, +not reusing the existing `EvaluatorID` field. + +**Rationale**: `EvaluatorID` already exists on `PolicyState` +and is used exclusively for complypack entries (populated +from the complypack config). A policy entry and a complypack +entry are stored in separate maps (`Policies` vs +`Complypacks`), but they share the same `PolicyState` struct. +Using a distinct field name avoids semantic confusion and +makes intent clear: `EvaluatorID` is the complypack's +evaluator binding, while `PolicyEvaluator` is the policy's +declared evaluator from the Gemara YAML. + +### D7: State update ordering and double-save + +**Decision**: Metadata extraction MUST happen after +`SyncPolicy()` returns. The call sequence is: + +1. `SyncPolicy()` -- internally calls + `UpdatePolicyStateWithVerification()` and `SaveState()` +2. `ExtractPolicyMetadata()` -- parses cached YAML +3. `SetPolicyMetadata()` -- updates metadata fields on + existing `PolicyState` entry (read-modify-write) +4. `SaveState()` -- second save with metadata included + +**Rationale**: `UpdatePolicyStateWithVerification()` creates +a new `PolicyState` struct and replaces the map entry, +zeroing out any previously set metadata. By running metadata +extraction *after* sync, we ensure the metadata is added to +the freshly created entry rather than being overwritten. +The double-save of `state.json` is intentional and +acceptable for a small JSON file (~1KB typical). + +### D8: Upgrade backfill for pre-existing caches + +**Decision**: When `complyctl get` runs and `SyncPolicy()` +returns `fetched=false` (no digest change), check whether +the existing `PolicyState` has empty metadata fields +(`PolicyEvaluator == ""`). If so, run metadata extraction +and save -- this backfills metadata for policies cached by +older complyctl versions without requiring a manual cache +clear. + +**Rationale**: Without backfill, upgrading users would see +"-" in EVALUATOR and CONTROLS columns indefinitely for +unchanged policies, requiring an undocumented manual cache +clear to fix. The backfill is a one-time cost per policy +(subsequent no-change syncs skip it because metadata is +already populated). + +### D9: Resolver instantiation in get command + +**Decision**: Create a `policy.Resolver` once in +`syncAllPolicies()` (via `policy.NewLoader(cacheMgr)` + +`policy.NewResolver(loader)`) and pass it to +`syncSinglePolicy()` as a parameter. + +**Rationale**: Follows the existing pattern in `scan.go` +where the resolver is created once per command invocation. +Avoids creating a new `Resolver` per policy in the loop, +which would be wasteful (though harmless since `Resolver` +is stateless). + +## Risks / Trade-offs + +- **[Backward compatibility]** Adding fields to + `PolicyState` is safe due to `omitempty` JSON tags and + Go's permissive unmarshalling. Older complyctl versions + reading a new `state.json` will silently ignore unknown + fields. + +- **[Parse cost at sync time]** + `ExtractPolicyMetadata()` parses the policy and catalog + YAML layers. This adds ~10ms per policy to the `get` + operation (local I/O only, no network), which is + negligible compared to network I/O for OCI fetch. The + parsing is only done for freshly fetched policies and + one-time backfill. + +- **[Double-save of state.json]** `SyncPolicy()` saves + state internally, then metadata extraction triggers a + second save. This is two writes of a small JSON file + (~1KB) per fresh policy -- acceptable and simpler than + modifying `SyncPolicy()`'s internal save behavior. + +- **[Multi-evaluator policies]** Policies with assessment + plans bound to different evaluators will show "-" in the + EVALUATOR column. This matches the existing behavior of + `extractFromGemaraPolicy()` which sets `EvaluatorID` to + empty string for multi-evaluator policies. Documenting + this edge case is sufficient -- multi-evaluator policies + are uncommon in practice. diff --git a/openspec/changes/bundle-metadata-cli/proposal.md b/openspec/changes/bundle-metadata-cli/proposal.md new file mode 100644 index 00000000..57ab0676 --- /dev/null +++ b/openspec/changes/bundle-metadata-cli/proposal.md @@ -0,0 +1,143 @@ +## Why + +After `complyctl get` pulls a Gemara bundle from an OCI registry, the +CLI discards all content metadata. Users cannot discover the policy +title, evaluator type, required target variables, or control count +without manually inspecting OCI blobs via `oras blob fetch`. The +`policies[].id` in `complytime.yaml` is a disconnected local alias +with no validation against bundle content -- `id: banana` works +identically to `id: cis-fedora-l1-workstation-policy`. + +As reported in [#506](https://github.com/complytime/complyctl/issues/506), +this problem worsens with multiple providers (OPA, Ampel, OpenSCAP) +because users need to know which evaluator a policy requires to +configure the correct target variables. + +The OCI config blob and policy YAML layers already contain all the +metadata needed (title, evaluator ID, assessment plans, control +catalog). This data is parsed during `scan` via `ResolvePolicyGraph()` +but is never surfaced to users outside of scan reports. + +## What Changes + +- **Option B**: Enrich `complyctl list` output with policy metadata + columns (EVALUATOR, CONTROLS) sourced from cached state +- **Option C**: `complyctl get` prints a post-sync summary after + each freshly fetched policy, showing title, evaluator, and counts + +Option A (`complyctl inspect <url>` for pre-pull discovery) is +deferred to a follow-up issue. + +## Capabilities + +### New Capabilities +- `get-sync-summary`: After a fresh policy fetch, `complyctl get` + prints a human-readable summary to stderr showing the policy + title, evaluator, control count, and assessment count +- `policy-metadata-cache`: `PolicyState` in `state.json` gains + metadata fields (`policy_title`, `policy_evaluator`, + `control_count`, `assessment_count`) populated at sync time +- `extract-policy-metadata`: `policy.Resolver` gains + `ExtractPolicyMetadata()` for lightweight metadata extraction + without building a full `DependencyGraph` + +### Modified Capabilities +- `list-output`: `complyctl list` table gains EVALUATOR and + CONTROLS columns sourced from cached `PolicyState` metadata + +### Removed Capabilities +None. + +## Impact + +- `internal/cache/state.go`: Add metadata fields to `PolicyState` +- `internal/policy/resolver.go`: Add `PolicyMetadata` type and + `ExtractPolicyMetadata()` method +- `cmd/complyctl/cli/get.go`: Call metadata extraction after fresh + sync, print summary to stderr, persist metadata to state +- `cmd/complyctl/cli/list.go`: Add EVALUATOR and CONTROLS columns +- Tests: New tests for metadata extraction, state round-trip, + list output, and get summary output + +## Documentation Impact + +- `CHANGELOG.md`: Entry under `### Added` for enriched `list` + output and `get` post-sync summary +- `AGENTS.md`: Update Recent Changes section + +## Constitution Alignment + +Assessed against the Unbound Force org constitution. + +### I. Autonomous Collaboration + +**Assessment**: PASS + +The metadata is persisted in `state.json` (an existing +artifact) using the existing JSON serialization pattern. +Both `list` and `get` summary consume the same cached +metadata, producing self-describing output without +requiring interactive discovery. + +### II. Composability First + +**Assessment**: PASS + +`ExtractPolicyMetadata()` is a standalone method on +`Resolver` that reuses existing YAML parsing functions +(`parsePolicyLayer`, `parseControlCatalog`) without +introducing new dependencies. The metadata cache is +additive to `PolicyState` -- no existing fields or methods +change. `list` reads metadata from state without needing +the Resolver or OCI store at display time. + +### III. Observable Quality + +**Assessment**: PASS + +All metadata fields are machine-parseable (JSON in +`state.json`, plain-text table in `list`, structured +stderr lines in `get` summary). The `list` table format +is tab-aligned and consistent with existing output. No +provenance metadata is lost -- the existing digest, +version, and verification fields are preserved. + +### IV. Testability + +**Assessment**: PASS + +Each component is testable in isolation: +- `PolicyState` JSON round-trip with new fields (unit test) +- `ExtractPolicyMetadata()` with mock `PolicyLoader` + (unit test) +- `SetPolicyMetadata()` non-overwrite guarantee (unit test) +- `list` output with pre-populated state (unit test) +- `get` summary output via stderr capture (unit test) + +### ComplyTime Constitution Alignment + +Additionally assessed against the ComplyTime constitution +principles in `.specify/memory/constitution.md`: + +- **I. Single Source of Truth**: Metadata cached in + `state.json` (single source), not duplicated elsewhere. + `list` and `get` both read from the same state file. +- **II. Simplicity & Isolation**: `ExtractPolicyMetadata()` + is a focused method with single responsibility, separate + from `ResolvePolicyGraph()`. `SetPolicyMetadata()` uses + read-modify-write to preserve sync fields. +- **III. Incremental Improvement**: Single concern (metadata + surfacing), no unrelated refactoring or formatting changes. +- **IV. Readability First**: Clear naming distinguishes + `PolicyEvaluator` (policy metadata) from `EvaluatorID` + (complypack binding). Design decisions documented with + rationale and alternatives. +- **V. Do Not Reinvent the Wheel**: Reuses existing YAML + parsing functions and mock infrastructure. +- **VI. Composability**: `list` reads state without needing + Resolver; `get` extracts and caches independently. + Output is tab-aligned plain text, consumable by scripts. +- **VII. Convention Over Configuration**: No new + configuration required; metadata appears automatically + after `get`. Follows existing `omitempty` pattern for + backward compatibility. diff --git a/openspec/changes/bundle-metadata-cli/specs/bundle-metadata-surface/spec.md b/openspec/changes/bundle-metadata-cli/specs/bundle-metadata-surface/spec.md new file mode 100644 index 00000000..bffed662 --- /dev/null +++ b/openspec/changes/bundle-metadata-cli/specs/bundle-metadata-surface/spec.md @@ -0,0 +1,276 @@ +## ADDED Requirements + +### Requirement: Policy metadata cached at sync time + +When `complyctl get` successfully fetches a fresh policy (i.e., +`SyncPolicy()` returns `fetched=true`), OR when the existing +`PolicyState` entry has empty metadata fields (upgrade backfill), +the system MUST extract policy metadata from the cached OCI +layout and persist it in `state.json` alongside the existing +sync state fields. + +The following metadata fields MUST be extracted and cached: + +- `policy_title`: The `title` field from the Gemara Policy + YAML (`gemara.Policy.Title`) +- `policy_evaluator`: The evaluator ID from the policy's + evaluation methods. For single-evaluator policies, this is + the executor ID from the policy-level or plan-level + evaluation method. For multi-evaluator policies (assessment + plans bound to different evaluators), this MUST be empty + string (matching existing `extractFromGemaraPolicy()` + behavior where `len(evalIDSet) > 1` yields `""`) +- `control_count`: The number of top-level entries in + `ControlCatalog.Controls` (i.e., `len(catalog.Controls)`). + Sub-controls and assessment requirements are not counted + separately +- `assessment_count`: The total number of assessment plans in + the Policy (i.e., `len(policy.Adherence.AssessmentPlans)`), + regardless of evaluator binding + +All fields MUST use `omitempty` JSON tags for backward +compatibility with existing `state.json` files. + +**Call ordering**: Metadata extraction MUST happen after +`SyncPolicy()` returns. The sequence is: `SyncPolicy()` (which +internally calls `UpdatePolicyStateWithVerification` and +`SaveState`) -> `ExtractPolicyMetadata()` -> +`SetPolicyMetadata()` -> `SaveState()`. The double-save of +`state.json` (once inside `SyncPolicy`, once after metadata +is set) is intentional and acceptable for a small JSON file. + +**Field name mapping**: The `PolicyMetadata.EvaluatorID` field +returned by `ExtractPolicyMetadata()` maps to +`PolicyState.PolicyEvaluator` when persisted. This is distinct +from `PolicyState.EvaluatorID`, which is reserved for +complypack entries (see design decision D6). + +#### Scenario: Fresh fetch populates metadata +- **GIVEN** a workspace with a policy entry pointing to a + valid OCI registry +- **WHEN** `complyctl get` fetches the policy for the first + time +- **THEN** `state.json` MUST contain the `policy_title`, + `policy_evaluator`, `control_count`, and `assessment_count` + fields for that policy + +#### Scenario: Re-fetch updates metadata +- **GIVEN** a policy was previously fetched and has metadata + in `state.json` +- **WHEN** `complyctl get` detects a new digest, re-fetches + the policy, and `SyncPolicy()` returns `fetched=true` +- **THEN** the metadata fields in `state.json` MUST be + updated to reflect the new policy content + +#### Scenario: No-change sync preserves existing metadata +- **GIVEN** a policy was previously fetched with metadata in + `state.json` +- **WHEN** `complyctl get` determines the policy is already + up-to-date (no digest change) +- **THEN** the existing metadata fields MUST be preserved + unchanged in `state.json` + +#### Scenario: Existing state.json without metadata fields +- **GIVEN** a `state.json` created by an older version of + complyctl that does not contain metadata fields +- **WHEN** the state is loaded by the new version +- **THEN** the missing metadata fields MUST default to their + zero values (empty string for strings, 0 for integers) +- **AND** no error MUST be returned + +#### Scenario: Upgrade backfill for unchanged policies +- **GIVEN** a policy was fetched by an older complyctl version + (no metadata in `state.json`) and the policy digest has not + changed since the original fetch +- **WHEN** `complyctl get` runs and `SyncPolicy()` returns + `fetched=false` +- **THEN** metadata MUST still be extracted and cached + (backfill) because the existing `PolicyState` has empty + `PolicyEvaluator` field +- **AND** `state.json` MUST be saved with the backfilled + metadata + +#### Scenario: Multi-evaluator policy +- **GIVEN** a policy with assessment plans bound to different + evaluator IDs (e.g., one plan uses "opa", another uses + "ampel") +- **WHEN** metadata is extracted +- **THEN** `policy_evaluator` MUST be empty string +- **AND** `complyctl list` MUST show "-" in the EVALUATOR + column for that policy + +#### Scenario: Policy with empty title +- **GIVEN** a cached policy whose Gemara Policy YAML has an + empty `title` field +- **WHEN** metadata is extracted +- **THEN** `policy_title` MUST be set to empty string +- **AND** the `get` summary MUST omit the title portion + and show only the evaluator + +### Requirement: Lightweight metadata extraction + +The `policy.Resolver` MUST provide an +`ExtractPolicyMetadata()` method that extracts +display-oriented metadata from a cached policy without +building a full `DependencyGraph`. + +The method MUST return a `PolicyMetadata` struct containing +`Title`, `EvaluatorID`, `ControlCount`, and `AssessmentCount`. + +The method MUST support both bundle-format and split-layer +OCI manifest shapes. + +Note: `PolicyMetadata.EvaluatorID` maps to +`PolicyState.PolicyEvaluator` when persisted (not to +`PolicyState.EvaluatorID`, which is reserved for complypacks). +See design decision D6 for the naming rationale. + +#### Scenario: Bundle-format policy +- **GIVEN** a cached policy using the Gemara bundle format + (`application/vnd.gemara.manifest.v1+json` config media + type) +- **WHEN** `ExtractPolicyMetadata()` is called +- **THEN** the returned `PolicyMetadata` MUST contain the + correct title, evaluator ID, control count, and assessment + count from the bundle's policy and catalog artifacts + +#### Scenario: Split-layer policy +- **GIVEN** a cached policy using the split-layer format + (distinct media types per layer) +- **WHEN** `ExtractPolicyMetadata()` is called +- **THEN** the returned `PolicyMetadata` MUST contain the + correct title, evaluator ID, control count, and assessment + count from the respective layers + +#### Scenario: Policy without control catalog +- **GIVEN** a cached policy that contains a Policy YAML but + no ControlCatalog layer +- **WHEN** `ExtractPolicyMetadata()` is called +- **THEN** `ControlCount` MUST be 0 +- **AND** no error MUST be returned for the missing catalog + +#### Scenario: Metadata extraction failure +- **GIVEN** a cached policy with corrupted or unparseable + YAML +- **WHEN** `ExtractPolicyMetadata()` is called +- **THEN** a descriptive error MUST be returned +- **AND** the `get` command MUST NOT fail -- metadata + extraction errors MUST be logged as warnings and the sync + MUST still succeed + +### Requirement: Get command prints post-sync summary + +After successfully syncing a fresh policy, `complyctl get` +MUST print a human-readable summary to stderr showing the +policy's internal metadata. + +The summary MUST include: +- Policy title and evaluator ID on one line +- Control count and assessment count on one line + +The summary MUST be printed only for freshly fetched +policies or backfilled metadata (not for no-change syncs +where metadata already exists). + +Metadata extraction is local-only (no network I/O) -- +it reads from the already-cached OCI layout on disk. + +#### Scenario: Fresh fetch prints summary +- **GIVEN** a workspace with one policy entry +- **WHEN** `complyctl get` fetches the policy for the + first time +- **THEN** stderr MUST contain a summary line showing the + policy title and evaluator, and a line showing control + and assessment counts +- **AND** the summary MUST appear after the "done" + confirmation + +#### Scenario: No-change sync omits summary +- **GIVEN** a policy was previously fetched and is + up-to-date with metadata already cached +- **WHEN** `complyctl get` runs and detects no digest + change +- **THEN** no post-sync summary MUST be printed for that + policy + +#### Scenario: Metadata extraction fails gracefully +- **GIVEN** a policy was freshly fetched but metadata + extraction encounters a parse error +- **WHEN** the extraction error is handled +- **THEN** a warning MUST be logged +- **AND** the sync MUST still report success ("done") +- **AND** no summary MUST be printed for that policy + +### Requirement: List command displays policy metadata + +`complyctl list` MUST display EVALUATOR and CONTROLS +columns in the policy table, sourced from the cached +`PolicyState` metadata in `state.json`. + +Machine-readable `list` output (e.g., `--output json`) is +out of scope for this change. + +#### Scenario: List with metadata available +- **GIVEN** policies have been fetched and metadata is + cached in `state.json` +- **WHEN** `complyctl list` is run +- **THEN** the table MUST include EVALUATOR and CONTROLS + columns showing the evaluator ID and control count for + each policy + +#### Scenario: List with no metadata (pre-existing cache) +- **GIVEN** policies were fetched by an older version of + complyctl and `state.json` has no metadata fields +- **WHEN** `complyctl list` is run +- **THEN** the EVALUATOR and CONTROLS columns MUST show + "-" for policies without cached metadata + +#### Scenario: List with policy-id filter +- **GIVEN** multiple policies are cached with metadata +- **WHEN** `complyctl list --policy-id <id>` is run +- **THEN** only the matching policy MUST appear with its + EVALUATOR and CONTROLS values + +#### Scenario: Column order +- **GIVEN** policies are cached with metadata +- **WHEN** `complyctl list` is run +- **THEN** the table columns MUST appear in the order: + POLICY ID, VERSION, EVALUATOR, CONTROLS, DIGEST, + VERIFIED + +## MODIFIED Requirements + +### Requirement: PolicyState struct extended + +Previously: `PolicyState` contained `Version`, `Digest`, +`EvaluatorID` (complypacks only), `LastUpdated`, `Verified`, +`SignerIdentity`, `Issuer`, `VerifiedAt`. + +Now: `PolicyState` MUST additionally contain `PolicyTitle` +(string), `PolicyEvaluator` (string), `ControlCount` (int), +and `AssessmentCount` (int) fields with `omitempty` JSON +tags. + +Note: The existing `EvaluatorID` field is used for +complypacks. The new `PolicyEvaluator` field is used for +policies. Both coexist on the same struct without conflict +because a given `PolicyState` entry is either a policy or a +complypack, not both. + +### Requirement: SetPolicyMetadata preserves sync fields + +The `State.SetPolicyMetadata()` method MUST read the +existing `PolicyState` entry for the given key and update +only the four metadata fields (`PolicyTitle`, +`PolicyEvaluator`, `ControlCount`, `AssessmentCount`). It +MUST NOT overwrite or zero out the sync fields (`Version`, +`Digest`, `EvaluatorID`, `LastUpdated`, `Verified`, +`SignerIdentity`, `Issuer`, `VerifiedAt`). + +If the key does not exist in the `Policies` map, the +method MUST no-op (return without modification), since +metadata without sync fields is meaningless. + +## REMOVED Requirements + +None. diff --git a/openspec/changes/bundle-metadata-cli/tasks.md b/openspec/changes/bundle-metadata-cli/tasks.md new file mode 100644 index 00000000..950b36a7 --- /dev/null +++ b/openspec/changes/bundle-metadata-cli/tasks.md @@ -0,0 +1,130 @@ +<!-- + [P] marks tasks eligible for parallel execution. + Add [P] when a task: (a) touches different files from + other [P] tasks in the group, (b) has no dependency + on prior tasks in the group, (c) can safely execute + without ordering constraints. + Do NOT add [P] when tasks modify the same file -- + parallel workers will cause merge conflicts. + Tasks without [P] run sequentially first, then [P] + tasks run in parallel. +--> + +## 1. Extend PolicyState with metadata fields + +- [x] 1.1 Add `PolicyTitle string`, `PolicyEvaluator string`, + `ControlCount int`, `AssessmentCount int` fields to + `PolicyState` in `internal/cache/state.go` with + `json:"...,omitempty"` tags +- [x] 1.2 Add `SetPolicyMetadata(repository string, title, + evaluator string, controls, assessments int)` method to + `State` that reads the existing `PolicyState` entry for + the given repository key and updates only the four + metadata fields. MUST NOT overwrite sync fields + (`Version`, `Digest`, `Verified`, etc.). If the key does + not exist in the `Policies` map, no-op (return without + modification) +- [x] 1.3 Add unit tests for: (a) `PolicyState` JSON + round-trip with new fields including backward + compatibility test with old JSON (missing fields default + to zero values), (b) `SetPolicyMetadata()` preserves all + existing sync fields (`Version`, `Digest`, `Verified`, + `SignerIdentity`, `Issuer`, `VerifiedAt`, `LastUpdated`) + when updating metadata on a pre-existing entry (table- + driven test with fully-populated `PolicyState`), (c) + `SetPolicyMetadata()` no-ops when key does not exist + +## 2. Add ExtractPolicyMetadata to Resolver + +- [x] 2.1 Add `PolicyMetadata` struct to + `internal/policy/resolver.go` with `Title`, `EvaluatorID`, + `ControlCount`, `AssessmentCount` fields. Note: + `EvaluatorID` here maps to `PolicyState.PolicyEvaluator` + when persisted (see design D6) +- [x] 2.2 Implement `ExtractPolicyMetadata(policyID, version)` + method on `Resolver` that detects manifest shape and + extracts metadata from policy + catalog layers. The + `policyID` parameter is the OCI repository path (cache + key), following the same convention as + `ResolvePolicyGraph()` +- [x] 2.3 Extend `parsePolicyLayer()` return or add a helper + to extract `Title` from `gemara.Policy` alongside + existing `EvaluatorID` and `Assessments`. The title + extraction MUST be tested (assert specific title value + from fixture, not just non-empty) +- [x] 2.4 Add unit tests for `ExtractPolicyMetadata()` + covering: bundle-format, split-layer, missing catalog + (`ControlCount` = 0), multi-evaluator policy + (`EvaluatorID` = ""), empty title, and parse error + scenarios. Use the existing `mockPolicyLoader` pattern + from `internal/policy/resolver_test.go` (not + `MockBundlePolicySource` which implements `PolicySource`, + not `PolicyLoader`) + +## 3. Integrate metadata extraction into get command + +- [x] 3.1 Create a `policy.Resolver` once in + `syncAllPolicies()` (via `policy.NewLoader(cacheMgr)` + + `policy.NewResolver(loader)`) and pass it to + `syncSinglePolicy()` as a new parameter. This follows + the existing pattern in `scan.go` +- [x] 3.2 In `syncSinglePolicy()` in + `cmd/complyctl/cli/get.go`, after `SyncPolicy()` returns, + call `ExtractPolicyMetadata()` when either: (a) `fetched + == true` (fresh fetch), or (b) the existing `PolicyState` + has empty `PolicyEvaluator` (upgrade backfill). Call + `state.SetPolicyMetadata()` then `cache.SaveState()` to + persist. The call ordering is: `SyncPolicy()` -> + `ExtractPolicyMetadata()` -> `SetPolicyMetadata()` -> + `SaveState()` (see design D7) +- [x] 3.3 Print post-sync summary to stderr after metadata + extraction succeeds: ` Policy: <title> (<evaluator>)` + and ` Controls: <N> | Assessments: <M>`. Omit title + portion when empty. Omit evaluator parenthetical when + empty (multi-evaluator). Print only for fresh fetch or + backfill, not for no-change syncs with existing metadata +- [x] 3.4 Handle metadata extraction errors gracefully: log + warning, skip summary, do not fail the sync +- [x] 3.5 Add unit tests for: (a) get summary output on + stderr (capture stderr, assert it contains + `"Policy: <title> (<evaluator>)"` and + `"Controls: <N> | Assessments: <M>"` after `"done"`), + (b) graceful error handling (assert warning logged and + no summary lines), (c) no-change sync with existing + metadata (assert no summary lines), (d) re-fetch with + new content (seed state with old metadata, simulate + fresh fetch, assert metadata updated to new values) + +## 4. Enrich list command with metadata columns + +- [x] 4.1 Update `listOptions.run()` in + `cmd/complyctl/cli/list.go` to read `PolicyEvaluator` + and `ControlCount` from `PolicyState` +- [x] 4.2 Update `printGemaraPolicyTable()` headers to: + `POLICY ID | VERSION | EVALUATOR | CONTROLS | DIGEST | + VERIFIED` +- [x] 4.3 Show "-" when metadata fields are empty + (pre-existing cache without metadata) or zero + (`ControlCount` of 0 shows "0", but empty + `PolicyEvaluator` shows "-") +- [x] 4.4 Update existing list tests to verify new column + output. Include: (a) column order assertion (verify + "EVALUATOR" appears after "VERSION" and before + "CONTROLS" in header), (b) metadata values display + correctly, (c) "-" shown for missing metadata, (d) + `--policy-id` filter with metadata (multiple policies + in config, only matching policy appears with correct + EVALUATOR and CONTROLS values) + +## 5. Verification and documentation + +- [x] 5.1 Run `make test-unit` and verify all tests pass +- [x] 5.2 Run `make lint` and fix any issues +- [x] 5.3 Run `make vet` and fix any issues +- [x] 5.4 Run `make sanity` to ensure full CI parity +- [x] 5.5 Update `CHANGELOG.md` with entries for enriched + `list` output and `get` post-sync summary +- [x] 5.6 Update `AGENTS.md` Recent Changes section with + bundle-metadata-cli entry +<!-- spec-review: passed --> +<!-- code-review: passed -->