Skip to content

ci(af01): establish workflow trust baseline - #43

Open
TheHalfMoon wants to merge 17 commits into
mainfrom
feat/af01-stack-a-workflow-trust
Open

ci(af01): establish workflow trust baseline#43
TheHalfMoon wants to merge 17 commits into
mainfrom
feat/af01-stack-a-workflow-trust

Conversation

@TheHalfMoon

@TheHalfMoon TheHalfMoon commented Aug 26, 2026

Copy link
Copy Markdown
Owner

AF-01 Stack A — workflow trust baseline

Canonical base at PR creation:

main: eeecb0bc03c7040bb18b70bce8b69d618384f783
tree: d5abe932f1436a9612f45bf130ba29aadbc5a133

This PR implements AF-01 Phase 1 / Stack A only. It does not change commandF product semantics, CF-06 oracle identity, CF-10 corpus authority, package/diff/terminology/impact rules, or dependency policy.

T010 — inventory

Adds stack-a-inventory.md covering every tracked workflow and Action metadata file, external uses: authority, job runners, effective permissions, timeouts, proof containers, checkout credentials, and lockfile-consuming Cargo commands at the canonical planning base.

T011/T012 — machine-checkable policy + deterministic audit

Adds:

  • .github/workflow-trust-policy.json
  • .github/scripts/audit_workflow_trust.py

The audit discovers all tracked .github/workflows/*.yml|*.yaml and every tracked action.yml|action.yaml at any depth, rejects unplanned workflows/jobs, mutable external uses: references, mutable latest runner labels, missing/excessive timeouts, unresolved/default or overbroad token permissions, checkout credentials left enabled, mutable job/service container identities, and configured lockfile-consuming Cargo commands without --locked. Local ./ actions remain allowed.

Output is deterministic JSON and exceptions require explicit rule/path/reason/revisit evidence.

T013/T016 — counterexamples and future-file coverage

Adds a repository-owned test suite covering:

  • tag/branch/short-SHA external refs;
  • nested action.yaml mutable uses;
  • both Action metadata filename forms at arbitrary depth;
  • missing checkout persist-credentials: false;
  • unresolved/default and overbroad permissions;
  • mutable *-latest runner labels;
  • missing/excessive timeouts;
  • mutable job and service container images;
  • unlocked Cargo commands;
  • malformed workflow input;
  • newly tracked workflow coverage;
  • deterministic repeat output;
  • invalid exception metadata.

T014/T015 — current workflow hardening

ci.yml now uses full-SHA checkout/rust-toolchain references, credentialless checkout, ubuntu-24.04, explicit 30-minute timeout, and executes the AF-01 audit tests plus live repository audit before Rust gates.

cf06-oracle.yml preserves all existing oracle commands/evidence while fixing runners to ubuntu-24.04, bounding all three jobs, and reducing the aggregation-only oracle-proof job to permissions: {}.

registry-download-smoke.yml moves its runner from ubuntu-latest to ubuntu-24.04; existing timeout, pinned actions, credentials, and semantic smoke behavior remain unchanged.

The CF-11/CF-11G/CF-12/CF-13 proof workflows already satisfied the selected Stack A baseline and are not semantically rewritten.

Qualification required before merge

  • exact-head mandatory ci;
  • every path-applicable existing proof/oracle workflow terminal green;
  • fresh exact-head CodeRabbit and Qodo review;
  • zero unresolved substantive findings;
  • exact-head merge only;
  • post-merge main/tree verification.

No PASS is assumed from queue state or review launch.


Summary by cubic

Establishes a deterministic trust baseline for all tracked GitHub workflows and Action metadata. Previously CI allowed mutable runners and action references; now every workflow and job must match the policy and pass the audit before Rust gates run.

Changes

  • Adds the policy, inventory, audit script, and counterexample tests.
  • Audits full-SHA action references, nested action.yml/action.yaml metadata, checkout credentials, runners, permissions, timeouts, container digests, and lockfile-consuming Cargo commands.
  • Enforces the exact policy schema; unknown rules, malformed workflow syntax, and flow-style uses: keys fail closed.
  • Detects unlocked Cargo invocations across inline, multiline with backslash continuations, and chained shell commands, and fails closed on unsupported cargo global-option syntax.
  • Pins current runners and actions, bounds jobs with timeouts, removes unnecessary oracle-proof permissions, and disables checkout credentials where needed.
  • Keeps commandF semantics, CF-06 oracle identity, and CF-10 corpus authority unchanged.

Written for commit 3c84f9a. Summary will update on new commits.

Review in cubic

@qodo-code-review

Copy link
Copy Markdown

ⓘ Your Qodo trial ends soon. Ask your workspace admin to set up billing to keep reviews running after the trial. Manage billing

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Copy link
Copy Markdown
Owner Author

@qodo review

Please review exact current head e844f5b306cbf201cc31fa893539d2ed0875486c. Focus on AF-01 Stack A only: deterministic workflow/action discovery, full-SHA external uses: enforcement, checkout credential checks, effective permission normalization/allowlist behavior, fixed runner and timeout policy, job/service container digest enforcement, Cargo --locked detection, exception fail-closed behavior, counterexample adequacy, and whether the workflow hardening preserves existing CF-06/CI/registry semantics. Do not infer approval from prior PRs.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Please perform a fresh review of exact current head e844f5b306cbf201cc31fa893539d2ed0875486c. Focus on audit parser correctness and fail-closed behavior, security coverage gaps, false-positive/false-negative risks, test validity, workflow permission/runner/timeout hardening, immutable external references and containers, and accidental changes to existing product/oracle/proof semantics. No PASS is assumed.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Establish machine-checkable GitHub Actions trust baseline

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

Grey Divider

AI Description

• Defines policy-backed auditing for immutable dependencies, bounded authority, and reproducible
 Cargo execution.
• Adds counterexample tests and runs trust validation before Rust CI gates.
• Hardens runners, action pins, checkout credentials, timeouts, and oracle permissions.
Diagram

graph TD
  A["Tracked files"] --> C["Trust audit"] --> D["JSON findings"] --> E{"Policy passes?"} -->|Yes| F["Rust gates"]
  B["Trust policy"] --> C
  E -->|No| G["Fail CI"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Standard YAML parser plus policy checks
  • ➕ Handles broader valid YAML syntax and structures
  • ➕ Reduces risks from indentation and scalar edge cases
  • ➖ Adds a runtime dependency to the mandatory trust gate
  • ➖ Still requires custom GitHub Actions semantic validation
2. Third-party workflow security scanner
  • ➕ Provides established checks and broader ecosystem coverage
  • ➕ May identify additional GitHub Actions security risks
  • ➖ Cannot directly encode the repository's exact workflow and job allowlist
  • ➖ Introduces another external tool and update authority into the trust path

Recommendation: Keep the repository-owned policy-driven audit because it provides exact fail-closed authority, deterministic output, and no new dependency. The custom parser intentionally supports a constrained workflow subset; complementing it later with a standard scanner is preferable to replacing the repository-specific policy checks.

Files changed (7) +915 / -10

Enhancement (1) +411 / -0
audit_workflow_trust.pyAdd deterministic workflow trust auditor +411/-0

Add deterministic workflow trust auditor

• Introduces tracked-file discovery and fail-closed checks for planned workflows and jobs, immutable action and container identities, fixed runners, bounded timeouts, explicit permissions, credentialless checkout, and locked Cargo commands. Produces sorted JSON findings and supports narrowly documented policy exceptions.

.github/scripts/audit_workflow_trust.py

Tests (1) +263 / -0
test_audit_workflow_trust.pyCover workflow trust policy counterexamples +263/-0

Cover workflow trust policy counterexamples

• Adds repository-owned unit tests for deterministic success output and failures involving mutable references, new files, credentials, permissions, runners, timeouts, containers, Cargo commands, malformed input, and invalid exceptions.

.github/scripts/test_audit_workflow_trust.py

Documentation (1) +110 / -0
stack-a-inventory.mdDocument canonical workflow trust inventory +110/-0

Document canonical workflow trust inventory

• Records the planning-base workflow and Action metadata authority, job permissions, runners, timeouts, action references, checkout behavior, containers, and lockfile-consuming Cargo commands. Defines the Stack A machine-checkable target and scope boundary.

specs/015-af-01-trusted-development-baseline/stack-a-inventory.md

Other (4) +131 / -10
workflow-trust-policy.jsonDeclare workflow and job trust authority +110/-0

Declare workflow and job trust authority

• Defines global trust rules and the expected jobs, effective permissions, fixed runners, and timeout ceilings for every tracked GitHub Actions workflow. Starts with no policy exceptions.

.github/workflow-trust-policy.json

cf06-oracle.ymlBound CF-06 oracle workflow authority +8/-4

Bound CF-06 oracle workflow authority

• Pins all oracle jobs to Ubuntu 24.04 and adds job-specific timeout limits. Removes repository token permissions from the aggregation-only proof job while preserving oracle validation behavior.

.github/workflows/cf06-oracle.yml

ci.ymlEnforce trust auditing before Rust gates +11/-4

Enforce trust auditing before Rust gates

• Pins the runner and external actions, disables persisted checkout credentials, and bounds the CI job duration. Runs the audit test suite and live repository audit before existing Rust checks.

.github/workflows/ci.yml

registry-download-smoke.ymlPin registry smoke runner image +2/-2

Pin registry smoke runner image

• Replaces the mutable Ubuntu latest runner label with Ubuntu 24.04 while retaining the existing timeout and registry smoke behavior.

.github/workflows/registry-download-smoke.yml

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== HEAD =='
git rev-parse HEAD
printf '%s\n' '== STATUS =='
git status --short
printf '%s\n' '== MERGE BASE =='
git merge-base HEAD main || true
printf '%s\n' '== CHANGED FILES VS MAIN =='
git diff --name-status "$(git merge-base HEAD main)" HEAD
printf '%s\n' '== AF-01 CANDIDATE FILES =='
fd -HI -t f . .github | sort
printf '%s\n' '== PYTHON OUTLINE =='
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true
printf '%s\n' '== TEST OUTLINES =='
fd -HI -t f -e py .github | while IFS= read -r f; do
  case "$f" in
    *test*|*Test*) printf '%s\n' "--- $f"; ast-grep outline "$f" --items all 2>/dev/null || true ;;
  esac
done
printf '%s\n' '== WORKFLOW DIFF STAT =='
git diff --stat "$(git merge-base HEAD main)" HEAD -- .github

Length of output: 4380


🏁 Script executed:

#!/bin/bash
set -euo pipefail

base="$(git merge-base HEAD main)"

printf '%s\n' '== AUDIT IMPLEMENTATION: 1-230 =='
sed -n '1,230p' .github/scripts/audit_workflow_trust.py
printf '%s\n' '== AUDIT IMPLEMENTATION: 230-430 =='
sed -n '230,430p' .github/scripts/audit_workflow_trust.py
printf '%s\n' '== POLICY =='
cat .github/workflow-trust-policy.json
printf '%s\n' '== TEST FILE =='
cat .github/scripts/test_audit_workflow_trust.py
printf '%s\n' '== INVENTORY =='
cat specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '%s\n' '== WORKFLOW DIFFS =='
git diff --find-renames --find-copies "$base" HEAD -- \
  .github/workflows/cf06-oracle.yml \
  .github/workflows/ci.yml \
  .github/workflows/registry-download-smoke.yml
printf '%s\n' '== CURRENT WORKFLOWS =='
for f in .github/workflows/*.yml .github/workflows/*.yaml; do
  [ -f "$f" ] || continue
  printf '\n--- %s ---\n' "$f"
  cat "$f"
done

Length of output: 82811


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== AUDIT LINE NUMBERS: USES, CARGO, YAML ERROR HANDLING =='
nl -ba .github/scripts/audit_workflow_trust.py | sed -n '145,185p;230,335p'
printf '%s\n' '== TEST LINE NUMBERS =='
nl -ba .github/scripts/test_audit_workflow_trust.py | sed -n '100,245p'
printf '%s\n' '== ROOT ACTION METADATA =='
if [ -f action.yml ]; then
  nl -ba action.yml
fi
if [ -f action.yaml ]; then
  nl -ba action.yaml
fi
printf '%s\n' '== YAML SPECIAL-FORM SEARCH =='
rg -n --glob '*.yml' --glob '*.yaml' \
  '^\s*(["'\''](?:uses|runs-on|timeout-minutes|permissions|image)["'\'']|<<:|[^#]*&[A-Za-z0-9_-]+|[^#]*\*[A-Za-z0-9_-]+)' \
  .github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== CARGO LINE-CONTINUATION/GLOBAL-OPTION SEARCH =='
rg -n -U --glob '*.yml' --glob '*.yaml' \
  'cargo\s*(\\\s*\n|(?:--[A-Za-z0-9-]+(?:[ =][^[:space:]]+)?\s+|\+[A-Za-z0-9_.-]+\s+))' \
  .github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== DIFF WHITESPACE =='
git diff --check "$(git merge-base HEAD main)" HEAD
printf '%s\n' '== ALL CHANGES OUTSIDE AF-01 =='
git diff --name-only "$(git merge-base HEAD main)" HEAD -- . ':(exclude).github' ':(exclude)specs/015-af-01-trusted-development-baseline'

Length of output: 1634


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== AUDIT LINE NUMBERS: USES, CARGO, YAML ERROR HANDLING =='
cat -n .github/scripts/audit_workflow_trust.py | sed -n '145,185p;230,335p'
printf '%s\n' '== TEST LINE NUMBERS =='
cat -n .github/scripts/test_audit_workflow_trust.py | sed -n '100,245p'
printf '%s\n' '== ROOT ACTION METADATA =='
if [ -f action.yml ]; then
  cat -n action.yml
fi
if [ -f action.yaml ]; then
  cat -n action.yaml
fi
printf '%s\n' '== YAML SPECIAL-FORM SEARCH =='
rg -n --glob '*.yml' --glob '*.yaml' \
  '^\s*(["'\''](?:uses|runs-on|timeout-minutes|permissions|image)["'\'']|<<:|[^#]*&[A-Za-z0-9_-]+|[^#]*\*[A-Za-z0-9_-]+)' \
  .github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== CARGO LINE-CONTINUATION/GLOBAL-OPTION SEARCH =='
rg -n -U --glob '*.yml' --glob '*.yaml' \
  'cargo\s*(\\\s*\n|(?:--[A-Za-z0-9-]+(?:[ =][^[:space:]]+)?\s+|\+[A-Za-z0-9_.-]+\s+))' \
  .github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== DIFF WHITESPACE =='
git diff --check "$(git merge-base HEAD main)" HEAD
printf '%s\n' '== ALL CHANGES OUTSIDE AF-01 =='
git diff --name-only "$(git merge-base HEAD main)" HEAD -- . ':(exclude).github' ':(exclude)specs/015-af-01-trusted-development-baseline'

Length of output: 21294


@TheHalfMoon I reviewed exact commit e844f5b306cbf201cc31fa893539d2ed0875486c.

I found two substantive AF-01 Stack A findings.

  1. P1 — Quoted YAML keys bypass the uses: and checkout credential checks.
    .github/scripts/audit_workflow_trust.py Line 166 only matches an unquoted uses: key. YAML accepts quoted mapping keys. For example, this valid workflow step bypasses both rules:

    - "uses": actions/checkout@v5

    The audit does not report mutable_uses or checkout_credentials because _all_uses() ignores "uses":. The same gap applies to quoted uses keys in tracked action.yml and action.yaml files. Use a YAML parser and validate the parsed workflow or Action structure. At minimum, reject unsupported YAML syntax instead of treating it as safe. Add counterexamples for quoted uses keys in both workflow and Action metadata inputs.

  2. P1 — Valid Cargo invocation forms bypass --locked enforcement.
    .github/scripts/audit_workflow_trust.py Line 305 only detects cargo followed immediately by a tracked subcommand on one physical line. A workflow can run a lockfile-consuming command without detection by using valid shell and Cargo syntax such as:

    run: cargo +1.97.1 test --workspace

    A line continuation also bypasses the detector:

    run: |
      cargo \
        test --workspace

    Neither form matches CARGO_RE, so the audit returns success without --locked. Parse each run script as shell input, or fail closed when a Cargo invocation cannot be normalized. Add negative tests for toolchain selectors, Cargo global options, and line continuations.

The workflow diff preserves the existing CF-06 commands and registry smoke commands. The CI changes add the audit gates before Rust gates. The aggregation-only oracle-proof job now has permissions: {}. I found no product, oracle, or proof semantic changes outside the AF-01 scope.

I verified that the requested commit and canonical merge base match. I also verified that no changed paths exist outside .github and the AF-01 inventory specification. I did not execute repository scripts or tests because the review sandbox prohibits executing repository files.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 58 seconds.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1011374e-cdff-4905-ba07-907fd34352e0

📥 Commits

Reviewing files that changed from the base of the PR and between eeecb0b and 3c84f9a.

📒 Files selected for processing (8)
  • .github/scripts/audit_workflow_trust.py
  • .github/scripts/test_audit_workflow_trust.py
  • .github/workflow-trust-policy.json
  • .github/workflows/cf06-oracle.yml
  • .github/workflows/ci.yml
  • .github/workflows/registry-download-smoke.yml
  • specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
  • specs/015-af-01-trusted-development-baseline/tasks.md

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

review

Please review exact current head e844f5b306cbf201cc31fa893539d2ed0875486c. Focus on AF-01 Stack A only: deterministic workflow/action discovery, full-SHA external uses: enforcement, checkout credential checks, effective permission normalization/allowlist behavior, fixed runner and timeout policy, job/service container digest enforcement, Cargo --locked detection, exception fail-closed behavior, counterexample adequacy, and whether the workflow hardening preserves existing CF-06/CI/registry semantics. Do not infer approval from prior PRs.

I reviewed the supplied e844f5b306cbf201cc31fa893539d2ed0875486c diff as a fresh review. I found three issues in the AF-01 audit implementation.

Findings

  1. High — Cargo lock enforcement can be bypassed by unrelated text on the same line (.github/scripts/audit_workflow_trust.py:305-313)

    • The check searches from the cargo <subcommand> match to the end of the raw YAML line and only tests whether --locked appears anywhere in that suffix.
    • Consequently, an invocation such as cargo test --workspace && echo --locked passes, as does an unlocked first invocation followed by a locked second invocation on the same line. Comments and quoted text can also satisfy the check.
    • Parse shell commands sufficiently to associate --locked with each invocation, or conservatively reject ambiguous command lines. Add counterexamples covering chained commands, comments, quoted text, and multiple Cargo invocations.
  2. Medium — Timeout policy is bounded, not fixed to the declared value (.github/scripts/audit_workflow_trust.py:267-277)

    • A policy value of 30 accepts any timeout from 1 through 30 minutes. Thus a change from the declared CI timeout of 30 minutes to 1 minute passes the audit, despite the policy/inventory presenting timeout_minutes as the job’s configured timeout.
    • If AF-01 requires the declared timeout to be authoritative, compare for equality; otherwise rename/document this explicitly as a maximum and add tests proving the intended lower-bound behavior.
  3. Medium — Policy validation is not fail-closed for malformed job/rule schemas (.github/scripts/audit_workflow_trust.py:342-367)

    • The validator checks only schema and that rules is a dictionary. It does not validate that workflow entries and job entries are dictionaries, that expected permissions are valid allowlisted GitHub permission maps, or that timeout_minutes/runner have valid types.
    • A malformed policy can therefore raise KeyError/AttributeError during audit_repository; when invoked through main, those exceptions are not in the handled exception list, so the tool emits a traceback instead of its deterministic JSON failure result. An invalid permission policy can also become the comparison authority rather than being rejected.
    • Validate the complete policy schema before auditing, including an explicit permission-key/value allowlist, and convert all policy-shape failures into invalid_policy findings.

Areas that look correct

  • Discovery is deterministic over tracked files, covers both workflow extensions under .github/workflows/, and finds action.yml/action.yaml at arbitrary tracked paths.
  • Current external action references are full 40-character SHAs; local ./ actions are correctly treated as repository-owned rather than external.
  • Checkout steps in the changed workflows use persist-credentials: false; current job/service container images are digest-bound.
  • Current workflow permissions, runners, and timeouts match the policy, and the CF-06 oracle-proof job’s explicit {} permissions do not alter its validation logic.
  • The registry workflow’s restored done and the CF-06 proof assertion preserve the apparent existing semantics; the CI changes add pinned setup, the audit gates, and bounded execution without changing the Rust test commands.

Counterexample coverage

The tests cover the main happy path and several direct mutations, but they do not cover malformed policy shapes, unknown permission keys, multiple checkout steps, ambiguous/chained Cargo commands, quoted/commented Cargo text, or whether timeout equality versus maximum semantics is intended. These should be added before treating the audit as a complete fail-closed control.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Copy link
Copy Markdown
Owner Author

@qodo review

Fresh exact-head review required after self-review hardening. Review current head 0be4766d3f2631f1dba24f03706da8ffe81f3279; prior review activity on e844f5b306cbf201cc31fa893539d2ed0875486c is stale. Specifically verify the shared workflow/action-metadata checkout credential enforcement, invalid-policy fail-closed path, narrowed job/service container discovery, and all previously requested AF-01 Stack A trust invariants. No PASS is assumed.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Fresh exact-head review required after self-review hardening. Review current head 0be4766d3f2631f1dba24f03706da8ffe81f3279; prior review activity on e844f5b306cbf201cc31fa893539d2ed0875486c is stale. Focus on parser correctness/fail-closed behavior, action-metadata checkout credentials, container discovery precision, policy validation, test adequacy, workflow hardening, and any remaining false-negative authority escape. No PASS is assumed.

@qodo-code-review

qodo-code-review Bot commented Aug 26, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📜 Skill insights (0)

Grey Divider


Action required

1. Malformed rules disable enforcement ✓ Resolved 🐞 Bug ⛨ Security
Description
The validator only checks that rules is an object, while cargo_locked_subcommands is blindly
passed to set; setting it to the JSON string "test" produces a character set and silently
disables matching for every configured Cargo subcommand. Other malformed rule values can raise
uncaught TypeError or use string truthiness, so policy corruption is neither consistently rejected
nor fail-closed.
Code

.github/scripts/audit_workflow_trust.py[R344-345]

+    if policy.get("schema") != 1 or not isinstance(policy.get("rules"), dict):
+        findings.append(Finding("invalid_policy", ".github/workflow-trust-policy.json", "", "unsupported policy schema"))
Relevance

●●● Strong

Malformed policy handling directly protects the same fail-closed trust enforcement accepted in prior
hardening.

PR-#12
PR-#32

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Repository-level validation verifies only schema and that rules is a dictionary, but enforcement
assumes specific nested types. In particular, `set(policy["rules"].get("cargo_locked_subcommands",
[]))` accepts a string and changes it into individual characters, making all multi-character
subcommands fail membership checks; main does not catch the resulting type errors from other
malformed values.

.github/scripts/audit_workflow_trust.py[298-317]
.github/scripts/audit_workflow_trust.py[342-350]
.github/scripts/audit_workflow_trust.py[387-403]
.github/scripts/test_audit_workflow_trust.py[244-259]
specs/015-af-01-trusted-development-baseline/tasks.md[25-31]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Individual rule values are not type- or value-validated before enforcement. A malformed but parseable policy can silently disable checks or terminate the audit with an uncaught exception.

## Issue Context
The policy is a security boundary and malformed-input behavior is required to fail closed. Validate all nested workflow, job, rule, and exception structures before invoking any audit logic, then return deterministic `invalid_policy` findings without applying exceptions to schema errors.

## Fix Focus Areas
- .github/scripts/audit_workflow_trust.py[205-227]
- .github/scripts/audit_workflow_trust.py[241-305]
- .github/scripts/audit_workflow_trust.py[342-369]
- .github/scripts/audit_workflow_trust.py[387-403]
- .github/scripts/test_audit_workflow_trust.py[244-259]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Checkout setting is unscoped ✓ Resolved 🐞 Bug ⛨ Security
Description
_checkout_has_credentials_disabled accepts any later line named persist-credentials: false
before the next step, without requiring it to be under that checkout step's with mapping. Placing
the same key under env leaves checkout's credential persistence enabled but makes the audit pass.
Code

.github/scripts/audit_workflow_trust.py[R199-201]

+    for line in lines[uses_index + 1 : end]:
+        if re.match(r"^\s*persist-credentials:\s*false\s*(?:#.*)?$", line):
+            return True
Relevance

●●● Strong

Accepted checkout-security precedent supports enforcing credential settings on the actual checkout
step.

PR-#12

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The implementation scans every line until the next list item and checks only indentation-agnostic
text matching; it never verifies a with: parent. The policy enables this rule, and the plan
requires persist-credentials: false specifically on checkout steps.

.github/scripts/audit_workflow_trust.py[188-202]
.github/scripts/audit_workflow_trust.py[318-324]
.github/workflow-trust-policy.json[14-16]
specs/015-af-01-trusted-development-baseline/plan.md[76-78]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The checkout credential check searches text within a step rather than validating the checkout step's `with.persist-credentials` input. An unrelated nested key can satisfy the audit while checkout still uses its credential-persisting default.

## Issue Context
The policy and AF-01 plan require the checkout input itself to be set to false. The current test only covers complete omission and does not cover a same-named key in another mapping.

## Fix Focus Areas
- .github/scripts/audit_workflow_trust.py[188-202]
- .github/scripts/audit_workflow_trust.py[315-324]
- .github/scripts/test_audit_workflow_trust.py[185-190]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Comments satisfy Cargo locking ✓ Resolved 🐞 Bug ☼ Reliability
Description
The Cargo check treats any --locked text after the subcommand as proof that the invocation is
locked, including shell comments such as cargo test --workspace # --locked. It also misses
line-continued invocations where cargo and the subcommand are on separate lines, so
lockfile-consuming commands can run unlocked without a finding.
Code

.github/scripts/audit_workflow_trust.py[R308-310]

+            for match in CARGO_RE.finditer(command):
+                subcommand = match.group(1)
+                if subcommand in locked_subcommands and "--locked" not in command[match.start() :]:
Relevance

●●● Strong

The team explicitly accepted adding locked enforcement to all lockfile-consuming Cargo commands.

PR-#2

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The implementation scans one stripped physical line at a time and searches the raw remainder for
--locked, without removing comments or reconstructing continued shell commands. The inventory
explicitly defines these commands as lockfile-consuming and requires every invocation to be locked.

.github/scripts/audit_workflow_trust.py[305-313]
.github/scripts/test_audit_workflow_trust.py[237-242]
specs/015-af-01-trusted-development-baseline/stack-a-inventory.md[79-105]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The audit performs a substring check on each physical YAML line, so non-executed comment text can satisfy `--locked` and shell line continuation can hide the Cargo subcommand. This allows unlocked dependency resolution despite the configured policy.

## Issue Context
The inventory defines the relevant lockfile-consuming Cargo subcommands and says all configured invocations must use `--locked`. Unsupported shell syntax should fail closed rather than be silently accepted.

## Fix Focus Areas
- .github/scripts/audit_workflow_trust.py[21-21]
- .github/scripts/audit_workflow_trust.py[305-313]
- .github/scripts/test_audit_workflow_trust.py[237-242]
- specs/015-af-01-trusted-development-baseline/stack-a-inventory.md[79-105]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View high (1)
4. Inline uses escape scanning ✓ Resolved 🐞 Bug ⛨ Security
Description
_all_uses only recognizes lines whose first YAML key is uses, so a valid flow-style step such as
steps: [{uses: owner/action@v1}] is never audited. A workflow or composite action can therefore
execute a mutable external action while the trust audit reports no mutable_uses finding.
Code

.github/scripts/audit_workflow_trust.py[R170-173]

+    for index in range(start, end):
+        matched = USES_RE.match(lines[index])
+        if matched:
+            result.append((index, _scalar(matched.group(2))))
Relevance

●●● Strong

Accepted security-hardening precedents favor complete immutable-action coverage.

PR-#12
PR-#32

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The scanner only matches a whole line beginning with optional whitespace/dash followed immediately
by uses:, then both workflow and Action audits rely exclusively on that result. The checked-in
AF-01 requirements explicitly require every external reference in both file types to be inspected.

.github/scripts/audit_workflow_trust.py[17-20]
.github/scripts/audit_workflow_trust.py[166-174]
.github/scripts/audit_workflow_trust.py[315-339]
specs/015-af-01-trusted-development-baseline/plan.md[71-81]
PR-#34

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The line-oriented `USES_RE` scan misses external `uses` keys embedded in valid YAML flow mappings/sequences, allowing mutable actions or reusable workflows to evade the full-SHA policy.

## Issue Context
The AF-01 plan requires inspection of all external `uses:` references in both workflows and Action metadata. The same `_all_uses` implementation is used for both file classes, so the bypass affects both.

## Fix Focus Areas
- .github/scripts/audit_workflow_trust.py[166-185]
- .github/scripts/audit_workflow_trust.py[315-339]
- .github/scripts/test_audit_workflow_trust.py[151-183]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

5. Malformed exceptions crash audit ✓ Resolved 📘 Rule violation ☼ Reliability
Description
A non-object entry in externally supplied exceptions is recognized as invalid, but _excepted()
still calls .get() on it and raises an uncaught exception. The audit can therefore abort instead
of returning its documented fail-closed JSON result for malformed policy data.
Code

.github/scripts/audit_workflow_trust.py[220]

+        if exception.get("rule") != finding.code or exception.get("path") != finding.path:
Relevance

●●● Strong

Fail-closed audit behavior is consistent with the repository’s accepted security-hardening
direction.

PR-#12
PR-#32

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 2717388 requires externally supplied malformed data to produce an error result
rather than abort processing. _valid_exception() explicitly accepts object and rejects
non-dictionaries, but _excepted() later unconditionally invokes exception.get; the repository
audit calls this filtering path even after adding an invalid_policy finding.

Rule 2717388: Library functions must not panic on externally supplied data
.github/scripts/audit_workflow_trust.py[205-215]
.github/scripts/audit_workflow_trust.py[218-226]
.github/scripts/audit_workflow_trust.py[342-369]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Malformed external policy data can crash the workflow trust audit because `_excepted()` assumes every exception is a dictionary even after validation reports otherwise.

## Issue Context
Keep the existing `invalid_policy` finding, but ensure malformed exception entries such as `null`, strings, or lists cannot raise `AttributeError` while findings are filtered. Add a deterministic regression test proving the audit returns `ok: false` and `invalid_policy` rather than throwing.

## Fix Focus Areas
- .github/scripts/audit_workflow_trust.py[218-226]
- .github/scripts/audit_workflow_trust.py[342-369]
- .github/scripts/test_audit_workflow_trust.py[253-259]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Named checkout steps falsely fail ✓ Resolved 🐞 Bug ≡ Correctness
Description
For a standard named checkout step where - name: precedes an indented `uses:
actions/checkout@..., _all_uses finds the action but _checkout_has_credentials_disabled`
immediately returns false because it requires - uses: on that same line. Even with a correct
with.persist-credentials: false, the audit emits checkout_credentials, preventing valid workflow
refactors.
Code

.github/scripts/audit_workflow_trust.py[R188-191]

+def _checkout_has_credentials_disabled(lines: list[str], uses_index: int) -> bool:
+    matched = STEP_USES_RE.match(lines[uses_index])
+    if not matched:
+        return False
Relevance

●●● Strong

Checkout credential hardening was explicitly accepted; named-step compatibility is a deterministic
fix.

PR-#12

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
_all_uses accepts an indented uses: key, but the checkout-specific helper requires
STEP_USES_RE, whose pattern mandates a dash immediately before uses, and returns false
otherwise. Existing workflows demonstrate that named action steps with indented uses are an
established repository style.

.github/scripts/audit_workflow_trust.py[18-20]
.github/scripts/audit_workflow_trust.py[166-202]
.github/workflows/cf11-multi-version-proof.yml[177-183]
.github/workflows/cf13-quality-gate-proof.yml[111-117]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The checkout checker assumes `uses` is the first key in a step. Valid named checkout steps are therefore rejected even when credentials are correctly disabled.

## Issue Context
Other repository action steps already use the common `- name:` followed by indented `uses:` layout, so this is a realistic future edit pattern. Structural YAML traversal should locate the enclosing step independent of key order.

## Fix Focus Areas
- .github/scripts/audit_workflow_trust.py[18-20]
- .github/scripts/audit_workflow_trust.py[166-202]
- .github/scripts/test_audit_workflow_trust.py[185-190]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. Trust rules lack rationale 📘 Rule violation ▣ Testability
Description
The three public boolean trust rules are published only as configuration keys, without a rationale
field or corresponding documentation explaining why each rule exists. Their positive and
counterexample tests do not satisfy the checklist's separate rationale requirement.
Code

.github/workflow-trust-policy.json[R14-16]

+    "require_container_digest": true,
+    "require_checkout_credentials_disabled": true,
+    "require_external_uses_full_sha": true
Relevance

●● Moderate

Rationale documentation is subjective; no closely matching accepted or rejected precedent appeared.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 2717385 requires every public rule to include explicit rationale in addition to
positive, negative, and deterministic tests. The repository-owned policy exposes
require_container_digest, require_checkout_credentials_disabled, and
require_external_uses_full_sha as rules, but these entries contain only boolean values and no
rationale metadata.

Rule 2717385: Require rationale, positive and negative tests, and deterministic behavior for public rules
.github/workflow-trust-policy.json[14-16]
.github/scripts/test_audit_workflow_trust.py[122-126]
.github/scripts/test_audit_workflow_trust.py[151-190]
.github/scripts/test_audit_workflow_trust.py[223-242]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The public workflow trust policy defines three boolean rules without documenting the rationale for each rule.

## Issue Context
Preserve the current machine-readable values and audit behavior. Add an explicit rationale field supported by the policy format, or a clearly linked documentation section that explains why digest-bound containers, credentialless checkout, and full-SHA external references are required.

## Fix Focus Areas
- .github/workflow-trust-policy.json[14-16]
- specs/015-af-01-trusted-development-baseline/stack-a-inventory.md[96-106]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 13 rules
Review mode: ⚖️ Balanced: Downgraded extended -> standard: change is below the extended eligibility bar (hunks 12/18, lines 925/200; both must reach the floor). Router rationale: This introduces substantial security-sensitive audit logic with custom parsing, policy enforcement, broad workflow coverage, and many independent hardening changes where a redundant review could catch subtle defects.

Grey Divider

Tip of the day
💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread .github/scripts/audit_workflow_trust.py
Comment on lines +14 to +16
"require_container_digest": true,
"require_checkout_credentials_disabled": true,
"require_external_uses_full_sha": true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Trust rules lack rationale 📘 Rule violation ▣ Testability

The three public boolean trust rules are published only as configuration keys, without a rationale
field or corresponding documentation explaining why each rule exists. Their positive and
counterexample tests do not satisfy the checklist's separate rationale requirement.
Agent Prompt
## Issue description
The public workflow trust policy defines three boolean rules without documenting the rationale for each rule.

## Issue Context
Preserve the current machine-readable values and audit behavior. Add an explicit rationale field supported by the policy format, or a clearly linked documentation section that explains why digest-bound containers, credentialless checkout, and full-SHA external references are required.

## Fix Focus Areas
- .github/workflow-trust-policy.json[14-16]
- specs/015-af-01-trusted-development-baseline/stack-a-inventory.md[96-106]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread .github/scripts/audit_workflow_trust.py
Comment thread .github/scripts/audit_workflow_trust.py Outdated
Comment thread .github/scripts/audit_workflow_trust.py Outdated
Comment thread .github/scripts/audit_workflow_trust.py Outdated
Comment thread .github/scripts/audit_workflow_trust.py Outdated
@qodo-code-review

Copy link
Copy Markdown

review

Fresh exact-head review required after self-review hardening. Review current head 0be4766d3f2631f1dba24f03706da8ffe81f3279; prior review activity on e844f5b306cbf201cc31fa893539d2ed0875486c is stale. Specifically verify the shared workflow/action-metadata checkout credential enforcement, invalid-policy fail-closed path, narrowed job/service container discovery, and all previously requested AF-01 Stack A trust invariants. No PASS is assumed.

Review findings

The requested invariants are enforced for the currently tracked workflows: all runners and timeouts are explicit, checkout steps use persist-credentials: false, external uses references are SHA-pinned, permissions are checked, and job/service container images are restricted to digest references. The shared _uses_findings path also scans both workflows and discovered action.yml/action.yaml files.

However, I would not mark this fully passing yet:

  • Semantic policy validation is incomplete and can crash instead of returning the defined fail-closed result. _policy_errors() only validates that workflows is a dictionary; it does not validate that workflow keys are strings or that each workflow/job policy has the required shape (.github/scripts/audit_workflow_trust.py:450-471). A policy containing a non-string workflow key can reach root / path at lines 501-508 and raise TypeError, which is not caught by main()'s exception list (525-551). Mixed-type job keys can likewise fail during sorting. Add recursive schema/type validation before using policy paths and keys, and cover malformed semantic policies with tests.

  • Workflow/action YAML syntax is not actually validated. The audit only rejects tabs and checks for a top-level jobs:/runs: marker (297-304, 432-447). A syntactically invalid YAML document can therefore pass the trust checks if its relevant lines happen to match the scanner. This undermines the fail-closed invariant; parse YAML (or explicitly reject unsupported syntax) before applying the line scanner, with malformed nested mappings/sequences covered by tests.

The narrowed container discovery itself correctly limits checks to job container.image and service image fields, and the current workflow inventory appears reconciled. These two fail-closed gaps remain blocking review issues.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Copy link
Copy Markdown
Owner Author

@qodo review

Final fresh review request for current exact head 95a2b4829c7b1a832b1460a4a97aada107e23877. All earlier review activity is stale. Verify the new action-metadata checkout credential regressions, non-container image false-positive regression, invalid-policy fail-closed regression, and the complete AF-01 Stack A authority model. Please report material bugs/rule violations/requirement gaps only; no prior PASS is assumed.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

expected='0be4766d3f2631f1dba24f03706da8ffe81f3279'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nEXPECTED=%s\n' "$actual" "$expected"
if [ "$actual" != "$expected" ]; then
  printf 'ERROR: checkout is not the requested exact head\n' >&2
  exit 2
fi

base="$(git merge-base HEAD main)"
printf 'MERGE_BASE=%s\n' "$base"
printf '\n== changed files ==\n'
git diff --name-status "$base" HEAD
printf '\n== changed AF-01 stat ==\n'
git diff --stat "$base" HEAD -- .github specs/015-af-01-trusted-development-baseline
printf '\n== audit outline ==\n'
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== test outline ==\n'
ast-grep outline .github/scripts/test_audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== implementation ==\n'
cat -n .github/scripts/audit_workflow_trust.py
printf '\n== tests ==\n'
cat -n .github/scripts/test_audit_workflow_trust.py
printf '\n== policy ==\n'
cat -n .github/workflow-trust-policy.json
printf '\n== workflow diffs ==\n'
git diff --find-renames "$base" HEAD -- .github/workflows
printf '\n== current workflow/action metadata files ==\n'
fd -HI -t f -e yml -e yaml .github action.yml action.yaml 2>/dev/null | sort | while IFS= read -r f; do
  printf '\n--- %s ---\n' "$f"
  cat -n "$f"
done
printf '\n== static source checks ==\n'
python3 -m py_compile .github/scripts/audit_workflow_trust.py .github/scripts/test_audit_workflow_trust.py
python3 -m json.tool .github/workflow-trust-policy.json >/dev/null
git diff --check "$base" HEAD
printf 'STATIC_CHECKS=OK\n'

Length of output: 48470


🏁 Script executed:

#!/bin/bash
set -euo pipefail

previous='e844f5b306cbf201cc31fa893539d2ed0875486c'
current='0be4766d3f2631f1dba24f03706da8ffe81f3279'

printf '%s\n' '== incremental diff from prior reviewed head =='
git diff --find-renames "$previous" "$current" -- \
  .github/scripts/audit_workflow_trust.py \
  .github/scripts/test_audit_workflow_trust.py \
  .github/workflow-trust-policy.json \
  .github/workflows \
  specs/015-af-01-trusted-development-baseline/stack-a-inventory.md

printf '%s\n' '== tracked security files =='
git ls-files -z | python3 -c '
import sys
from pathlib import PurePosixPath
for path in sorted(p for p in sys.stdin.buffer.read().decode().split("\0") if p):
    name = PurePosixPath(path).name
    if (path.startswith(".github/workflows/") and name.endswith((".yml", ".yaml"))) or name in {"action.yml", "action.yaml"}:
        print(path)
'

printf '%s\n' '== full current workflow source =='
while IFS= read -r f; do
  printf '\n--- %s ---\n' "$f"
  cat -n "$f"
done < <(git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)')

printf '%s\n' '== parser-relevant implementation and test locations =='
rg -n -C 4 \
  'USES_RE|STEP_USES_RE|CARGO_RE|def _container_images|def _job_ranges|def _parse_permissions|def _policy_errors|quoted|continuation|toolchain|container|persist-credentials' \
  .github/scripts/audit_workflow_trust.py .github/scripts/test_audit_workflow_trust.py

printf '%s\n' '== static checks =='
python3 -m py_compile .github/scripts/audit_workflow_trust.py .github/scripts/test_audit_workflow_trust.py
python3 -m json.tool .github/workflow-trust-policy.json >/dev/null
git diff --check "$(git merge-base HEAD main)" HEAD
printf '%s\n' 'STATIC_CHECKS=OK'

Length of output: 20520


@TheHalfMoon I reviewed exact head 0be4766d3f2631f1dba24f03706da8ffe81f3279.

I found three substantive AF-01 findings.

  1. P1 — The policy can disable mandatory trust rules without an invalid_policy finding.
    .github/scripts/audit_workflow_trust.py Line 457 through Line 460 only validate that rules is an object. Later code uses permissive defaults. For example, if a pull request removes require_external_uses_full_sha or sets it to false, _uses_findings() does not check external references. If it removes require_container_digest, the container check does not run. If it removes cargo_locked_subcommands, the lockfile check uses an empty set. The repository audit can then pass with mutable uses:, mutable containers, or unlocked Cargo commands. Validate the complete required rule set and its types. Require the mandatory boolean rules to be true. Require the required Cargo subcommands. Add negative tests for missing, false, and incorrectly typed rule values.

  2. P1 — Quoted YAML keys still bypass uses: and checkout credential enforcement.
    .github/scripts/audit_workflow_trust.py Line 18 and Line 19 only match bare uses: keys. GitHub Actions YAML accepts quoted mapping keys. This valid step bypasses immutable-reference and checkout credential checks:

    - "uses": actions/checkout@v5

    The shared _uses_findings() helper extends the check to Action metadata, but it retains the same parser limitation. The bypass applies to workflows and composite Action metadata. Parse YAML and inspect the parsed mapping structure, or reject unsupported YAML syntax before treating the file as compliant. Add tests for quoted uses keys in both workflow and action.yml or action.yaml inputs.

  3. P1 — Cargo --locked enforcement still misses valid invocations.
    .github/scripts/audit_workflow_trust.py Line 21 requires the tracked subcommand immediately after cargo on one physical line. Line 414 through Line 426 applies that pattern per source line. These valid commands bypass the rule:

    run: cargo +1.97.1 test --workspace
    run: |
      cargo \
        test --workspace

    Neither form matches CARGO_RE, so the audit reports no cargo_unlocked finding. Normalize shell commands before inspection, or fail closed for unsupported Cargo invocation syntax. Add counterexamples for toolchain selectors, Cargo global options, and line continuations.

The incremental hardening correctly moves normal checkout credential checks into shared workflow and Action metadata handling. It also narrows the container scan to job and service containers. These changes do not resolve the parser bypasses above.

I verified the requested exact head and canonical merge base. I did not execute repository code or tests. The read-only inspection commands terminated before their final static-check section, so compilation and full file enumeration remain unverified.

You are interacting with an AI system.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Final fresh review request for current exact head 95a2b4829c7b1a832b1460a4a97aada107e23877. All earlier review activity is stale. Focus on false negatives/false positives in the constrained parser, action-metadata checkout credential enforcement, container discovery precision, invalid policy handling, test coverage, workflow hardening, and preservation of existing CF/proof semantics. No approval is assumed.

@qodo-code-review

Copy link
Copy Markdown

review

Final fresh review request for current exact head 95a2b4829c7b1a832b1460a4a97aada107e23877. All earlier review activity is stale. Verify the new action-metadata checkout credential regressions, non-container image false-positive regression, invalid-policy fail-closed regression, and the complete AF-01 Stack A authority model. Please report material bugs/rule violations/requirement gaps only; no prior PASS is assumed.

Fresh review of 95a2b4829c7b1a832b1460a4a97aada107e23877 found these material gaps:

  • finding 1 — action required: Policy validation only checks that rules is an object. Malformed nested values are then used as trusted types: for example, cargo_locked_subcommands: null raises TypeError, while a string silently becomes a character set and disables Cargo locking checks. This violates fail-closed policy handling (.github/scripts/audit_workflow_trust.py:450-471, 413-419).
  • finding 2 — action required: Checkout credential detection is not scoped to the checkout step’s with mapping. A persist-credentials: false under env or another nested mapping can satisfy the audit while checkout credentials remain enabled (.github/scripts/audit_workflow_trust.py:221-235).
  • finding 3 — action required: Cargo locking scans raw physical lines and treats comments as arguments, so cargo test --workspace # --locked passes. It also misses line-continued invocations (.github/scripts/audit_workflow_trust.py:413-425).
  • finding 4 — action required: uses discovery is line-based and misses valid flow-style YAML such as steps: [{uses: owner/action@v1}], leaving mutable external references unaudited (.github/scripts/audit_workflow_trust.py:197-207).
  • finding 6 — remediation recommended: Named checkout steps (- name: Checkout followed by an indented uses:) are discovered by _all_uses, but _checkout_has_credentials_disabled only accepts - uses: on the same line and falsely reports valid steps as insecure (.github/scripts/audit_workflow_trust.py:221-226).
  • finding 7 — remediation recommended: The three public boolean rules in .github/workflow-trust-policy.json:14-16 have no explicit rationale, leaving the AF-01 rule documentation requirement incomplete.

Verified requested regressions:

  • Action-metadata checkout checks are present, but the named-step scoping bug above still applies to action metadata.
  • The non-container image false-positive regression is covered by _container_images; it only treats job container.image and service image as authorities, so the current implementation does not report ordinary job env.image values.
  • Invalid top-level policy handling now fails closed before scanning (.github/scripts/audit_workflow_trust.py:474-486). However, nested rule-shape validation remains incomplete, as described in finding 1.
  • The checked-in Stack A policy enumerates all seven tracked workflows and the discovery code scans both action.yml and action.yaml at tracked paths. No additional authority-model omission was identified beyond the enforcement/parser gaps above.

Finding 5 is already implemented in the current control flow: _policy_errors() returns before malformed exceptions reach _excepted().

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

expected='95a2b4829c7b1a832b1460a4a97aada107e23877'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nEXPECTED=%s\n' "$actual" "$expected"
if [ "$actual" != "$expected" ]; then
  printf '%s\n' 'ERROR: checkout is not the requested exact head' >&2
  exit 2
fi

base="$(git merge-base HEAD main)"
printf 'MERGE_BASE=%s\n' "$base"

printf '\n== changed files ==\n'
git diff --name-status "$base" HEAD

printf '\n== incremental changes from prior head ==\n'
git diff --find-renames 0be4766d3f2631f1dba24f03706da8ffe81f3279 HEAD -- \
  .github/scripts/audit_workflow_trust.py \
  .github/scripts/test_audit_workflow_trust.py \
  .github/workflow-trust-policy.json \
  .github/workflows \
  specs/015-af-01-trusted-development-baseline/stack-a-inventory.md

printf '\n== audit outline ==\n'
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true

printf '\n== audit implementation ==\n'
cat -n .github/scripts/audit_workflow_trust.py

printf '\n== audit tests ==\n'
cat -n .github/scripts/test_audit_workflow_trust.py

printf '\n== policy ==\n'
cat -n .github/workflow-trust-policy.json

printf '\n== inventory ==\n'
cat -n specs/015-af-01-trusted-development-baseline/stack-a-inventory.md

printf '\n== workflow diffs ==\n'
git diff --find-renames "$base" HEAD -- .github/workflows

printf '\n== tracked workflow and Action metadata ==\n'
while IFS= read -r f; do
  printf '\n--- %s ---\n' "$f"
  cat -n "$f"
done < <(git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)')

printf '\n== parser and policy focus locations ==\n'
rg -n -C 5 \
  'USES|uses|CARGO|cargo|persist-credentials|container|service|image|policy|permissions|runs-on|timeout|exception|yaml|YAML' \
  .github/scripts/audit_workflow_trust.py \
  .github/scripts/test_audit_workflow_trust.py \
  .github/workflow-trust-policy.json

printf '\n== whitespace errors ==\n'
git diff --check "$base" HEAD

Length of output: 57473


🏁 Script executed:

#!/bin/bash
set -euo pipefail

expected='95a2b4829c7b1a832b1460a4a97aada107e23877'
test "$(git rev-parse HEAD)" = "$expected"
base="$(git merge-base HEAD main)"

printf '%s\n' '== tracked workflow and Action metadata paths =='
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort

printf '%s\n' '== current workflow and Action metadata source =='
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort | while IFS= read -r f; do
  printf '\n--- %s ---\n' "$f"
  cat -n "$f"
done

printf '%s\n' '== full changes outside AF-01 configuration and inventory =='
git diff --name-only "$base" HEAD -- . ':(exclude).github' ':(exclude)specs/015-af-01-trusted-development-baseline'

printf '%s\n' '== workflow changes with function-context =='
git diff --unified=12 "$base" HEAD -- .github/workflows

printf '%s\n' '== parser function regions =='
sed -n '15,240p;270,475p' .github/scripts/audit_workflow_trust.py | cat -n

printf '%s\n' '== test cases that cover parser exceptions and policies =='
sed -n '120,310p' .github/scripts/test_audit_workflow_trust.py | cat -n

printf '%s\n' '== changed file end-of-line checks =='
git diff --check "$base" HEAD

Length of output: 91075


@TheHalfMoon I reviewed exact head 95a2b4829c7b1a832b1460a4a97aada107e23877.

I found three material AF-01 Stack A findings.

  1. P1 — Quoted YAML keys bypass uses: and checkout credential enforcement.
    .github/scripts/audit_workflow_trust.py Line 18 and Line 19 only match bare uses: keys. GitHub Actions accepts quoted YAML mapping keys. This valid workflow step is not found by _all_uses():

    - "uses": actions/checkout@v5

    The audit reports neither mutable_uses nor checkout_credentials. The same bypass applies to composite Action metadata because audit_action_metadata() uses the shared _uses_findings() function. Parse the YAML mapping structure, or reject quoted and other unsupported key forms before the audit reports success. Add workflow and Action metadata regressions for quoted uses keys.

  2. P1 — Valid Cargo command forms bypass --locked enforcement.
    .github/scripts/audit_workflow_trust.py Line 21 only accepts cargo followed immediately by the tracked subcommand on one source line. The audit therefore misses valid commands such as:

    run: cargo +1.97.1 test --workspace

    It also misses shell continuations:

    run: |
      cargo \
        test --workspace

    Neither form matches CARGO_RE, so the audit can pass without --locked. Normalize the shell command before inspection, or reject unsupported Cargo invocation forms. Add negative tests for toolchain selectors, Cargo global options, and line continuations.

  3. P1 — The policy can disable mandatory rules without an invalid_policy finding.
    .github/scripts/audit_workflow_trust.py Line 450 through Line 471 only require rules to be an object. A policy change can remove require_external_uses_full_sha, set it to false, remove require_container_digest, or set cargo_locked_subcommands to an empty list. The audit then skips these mandatory checks and can return success.

    Require every Stack A rule. Validate each rule type. Require both boolean authority rules to be true. Require the complete lockfile-consuming Cargo subcommand set. Add regressions for missing, false, and incorrectly typed rule values.

The new Action metadata checkout tests and the non-container image regression test cover their intended cases. They do not cover the parser bypasses above. The invalid-policy-root test covers only an absent rules object. It does not cover rule removal or rule disablement.

I verified the workflow changes preserve the existing CF-06 oracle commands, CI product smoke commands, and registry smoke commands. The changes only add the AF-01 audit gates, pin runners and Actions, add timeouts, disable checkout credential persistence, and reduce the aggregation-only oracle-proof job permissions to {}.

I confirmed the requested exact head and canonical merge base. I did not execute repository code or tests because the review sandbox prohibits executing repository files. I do not approve this head while these fail-open paths remain.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Copy link
Copy Markdown
Owner Author

@qodo review

Fresh exact-head review for current final candidate 3c84f9a93674a5957fe731f70330ea94404fcf55. Prior reviews are stale. All seven previous findings were addressed with code/tests or machine-readable rationale: malformed exceptions fail closed; rule schema is type/value validated and cannot be disabled; flow-style uses fails closed; checkout requires exactly one scoped with.persist-credentials: false and supports named steps; Cargo parsing strips comments and handles continuations/chained commands; every trust rule has a checked policy rationale. Please re-evaluate the full AF-01 Stack A authority surface and report any remaining material bug, rule violation, or requirement gap. No PASS is assumed.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Fresh exact-head review for current final candidate 3c84f9a93674a5957fe731f70330ea94404fcf55. Prior review activity is stale. Focus on the complete constrained-parser security boundary after Qodo remediation: flow-style/unsupported uses, structurally scoped checkout inputs including named steps, immutable refs/containers, meta-policy anti-weakening validation, permission/runner/timeout limits, Cargo shell parsing with comments/continuations, exception handling, counterexample validity, and preservation of existing workflow/product/oracle semantics. No approval is assumed.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

expected='3c84f9a93674a5957fe731f70330ea94404fcf55'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nEXPECTED=%s\n' "$actual" "$expected"
test "$actual" = "$expected"

base="$(git merge-base HEAD main)"
printf 'MERGE_BASE=%s\n' "$base"

printf '\n== changed files vs merge base ==\n'
git diff --name-status "$base" HEAD

printf '\n== remediation diff from prior reviewed head ==\n'
git diff --find-renames 95a2b4829c7b1a832b1460a4a97aada107e23877 HEAD -- \
  .github/scripts/audit_workflow_trust.py \
  .github/scripts/test_audit_workflow_trust.py \
  .github/workflow-trust-policy.json \
  .github/workflows \
  specs/015-af-01-trusted-development-baseline/stack-a-inventory.md

printf '\n== audit source outline ==\n'
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true

printf '\n== test source outline ==\n'
ast-grep outline .github/scripts/test_audit_workflow_trust.py --items all 2>/dev/null || true

printf '\n== audit implementation ==\n'
cat -n .github/scripts/audit_workflow_trust.py

printf '\n== test suite ==\n'
cat -n .github/scripts/test_audit_workflow_trust.py

printf '\n== policy ==\n'
cat -n .github/workflow-trust-policy.json

printf '\n== inventory ==\n'
cat -n specs/015-af-01-trusted-development-baseline/stack-a-inventory.md

printf '\n== tracked workflow and Action metadata paths ==\n'
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort

printf '\n== tracked workflow and Action metadata content ==\n'
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort |
while IFS= read -r file; do
  printf '\n--- %s ---\n' "$file"
  cat -n "$file"
done

printf '\n== workflow diff vs merge base ==\n'
git diff --find-renames --unified=16 "$base" HEAD -- .github/workflows

printf '\n== changed paths outside AF-01 areas ==\n'
git diff --name-only "$base" HEAD -- . \
  ':(exclude).github' \
  ':(exclude)specs/015-af-01-trusted-development-baseline'

printf '\n== whitespace validation ==\n'
git diff --check "$base" HEAD

Length of output: 173621


@TheHalfMoon I reviewed exact head 3c84f9a93674a5957fe731f70330ea94404fcf55.

I found three material AF-01 Stack A findings.

  1. P1 — Quoted uses keys still bypass immutable-reference and checkout credential checks.
    .github/scripts/audit_workflow_trust.py Line 19 only matches bare uses:. The new flow-style check at Line 247 through Line 261 does not reject a quoted block mapping key. GitHub Actions accepts this valid step:

    - "uses": actions/checkout@v5

    _all_uses() does not return this reference. The audit therefore reports neither mutable_uses nor checkout_credentials. The same bypass applies to composite Action metadata. Reject quoted mapping keys and other unsupported block key syntax, or parse the YAML mapping structure. Add workflow and Action metadata regressions.

  2. P1 — Flow-style services bypasses mutable service-container enforcement.
    .github/scripts/audit_workflow_trust.py Line 178 through Line 215 only recognizes a service image: at eight-space block indentation. This valid workflow form is not inspected:

    services: { database: { image: postgres:18 } }

    _container_images() sets service_indent but never finds the nested flow-style image. The audit accepts the mutable service image. Reject flow-style container and services values, or support them structurally. Add a negative regression for a flow-style mutable service image.

  3. P1 — Indirect Cargo execution bypasses --locked enforcement.
    .github/scripts/audit_workflow_trust.py Line 410 only recognizes a token equal to cargo. A valid shell script can invoke Cargo through variable expansion:

    run: |
      tool=cargo
      "$tool" test --workspace

    The second command contains no literal cargo token. _cargo_findings() returns no finding. The workflow can therefore run a lockfile-consuming command without --locked. The constrained parser must reject Cargo-related variable assignments and indirect command forms unless it can prove they are safe. Add negative tests for variable expansion and command substitution.

The policy validation now prevents removal or disablement of the defined Stack A rules. The flow-style uses and named checkout-step tests cover their stated cases. The workflow diff preserves the existing CF-06 commands, CI smoke commands, and registry smoke commands.

I confirmed that no changed paths exist outside .github and the AF-01 specification area. I used static inspection only. I did not execute repository scripts or tests. I do not approve this head while these authority bypasses remain.

You are interacting with an AI system.

@qodo-code-review

Copy link
Copy Markdown

review

Fresh exact-head review for current final candidate 3c84f9a93674a5957fe731f70330ea94404fcf55. Prior reviews are stale. All seven previous findings were addressed with code/tests or machine-readable rationale: malformed exceptions fail closed; rule schema is type/value validated and cannot be disabled; flow-style uses fails closed; checkout requires exactly one scoped with.persist-credentials: false and supports named steps; Cargo parsing strips comments and handles continuations/chained commands; every trust rule has a checked policy rationale. Please re-evaluate the full AF-01 Stack A authority surface and report any remaining material bug, rule violation, or requirement gap. No PASS is assumed.

I found one remaining material fail-open gap: quoted job IDs can escape job coverage.

  • JOB_RE only recognizes unquoted job keys matching ^ ([A-Za-z0-9_.-]+): (.github/scripts/audit_workflow_trust.py:18).
  • _job_ranges() then relies on that regex to build the complete job set (.github/scripts/audit_workflow_trust.py:146-165).
  • Therefore a valid workflow such as:
    jobs:
      build:
        runs-on: ubuntu-24.04
      "unplanned-job":
        runs-on: ubuntu-24.04
    can cause unplanned-job to be omitted from actual_names; if the policy contains only build, the audit can miss the extra job and its permissions, steps, actions, containers, and Cargo commands.

This violates the stated requirement that every discovered job have exact policy coverage. The fix should either parse job keys with YAML semantics or fail closed whenever the jobs mapping contains unsupported/quoted job-key syntax, with a regression test for a quoted unplanned job.

The previous rationale finding, finding 7, appears addressed: all four supported rules now have substantive entries under rationales (.github/workflow-trust-policy.json:18-22), and the validator requires exact coverage and minimum rationale length (.github/scripts/audit_workflow_trust.py:736-750).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant