ci(af01): establish workflow trust baseline - #43
Conversation
|
ⓘ Your Qodo trial ends soon. Ask your workspace admin to set up billing to keep reviews running after the trial. Manage billing |
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
@qodo review Please review exact current head |
|
@coderabbitai review Please perform a fresh review of exact current head |
PR Summary by QodoEstablish machine-checkable GitHub Actions trust baseline
AI Description
Diagram
High-Level Assessment
Files changed (7)
|
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== HEAD =='
git rev-parse HEAD
printf '%s\n' '== STATUS =='
git status --short
printf '%s\n' '== MERGE BASE =='
git merge-base HEAD main || true
printf '%s\n' '== CHANGED FILES VS MAIN =='
git diff --name-status "$(git merge-base HEAD main)" HEAD
printf '%s\n' '== AF-01 CANDIDATE FILES =='
fd -HI -t f . .github | sort
printf '%s\n' '== PYTHON OUTLINE =='
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true
printf '%s\n' '== TEST OUTLINES =='
fd -HI -t f -e py .github | while IFS= read -r f; do
case "$f" in
*test*|*Test*) printf '%s\n' "--- $f"; ast-grep outline "$f" --items all 2>/dev/null || true ;;
esac
done
printf '%s\n' '== WORKFLOW DIFF STAT =='
git diff --stat "$(git merge-base HEAD main)" HEAD -- .githubLength of output: 4380 🏁 Script executed: #!/bin/bash
set -euo pipefail
base="$(git merge-base HEAD main)"
printf '%s\n' '== AUDIT IMPLEMENTATION: 1-230 =='
sed -n '1,230p' .github/scripts/audit_workflow_trust.py
printf '%s\n' '== AUDIT IMPLEMENTATION: 230-430 =='
sed -n '230,430p' .github/scripts/audit_workflow_trust.py
printf '%s\n' '== POLICY =='
cat .github/workflow-trust-policy.json
printf '%s\n' '== TEST FILE =='
cat .github/scripts/test_audit_workflow_trust.py
printf '%s\n' '== INVENTORY =='
cat specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '%s\n' '== WORKFLOW DIFFS =='
git diff --find-renames --find-copies "$base" HEAD -- \
.github/workflows/cf06-oracle.yml \
.github/workflows/ci.yml \
.github/workflows/registry-download-smoke.yml
printf '%s\n' '== CURRENT WORKFLOWS =='
for f in .github/workflows/*.yml .github/workflows/*.yaml; do
[ -f "$f" ] || continue
printf '\n--- %s ---\n' "$f"
cat "$f"
doneLength of output: 82811 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== AUDIT LINE NUMBERS: USES, CARGO, YAML ERROR HANDLING =='
nl -ba .github/scripts/audit_workflow_trust.py | sed -n '145,185p;230,335p'
printf '%s\n' '== TEST LINE NUMBERS =='
nl -ba .github/scripts/test_audit_workflow_trust.py | sed -n '100,245p'
printf '%s\n' '== ROOT ACTION METADATA =='
if [ -f action.yml ]; then
nl -ba action.yml
fi
if [ -f action.yaml ]; then
nl -ba action.yaml
fi
printf '%s\n' '== YAML SPECIAL-FORM SEARCH =='
rg -n --glob '*.yml' --glob '*.yaml' \
'^\s*(["'\''](?:uses|runs-on|timeout-minutes|permissions|image)["'\'']|<<:|[^#]*&[A-Za-z0-9_-]+|[^#]*\*[A-Za-z0-9_-]+)' \
.github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== CARGO LINE-CONTINUATION/GLOBAL-OPTION SEARCH =='
rg -n -U --glob '*.yml' --glob '*.yaml' \
'cargo\s*(\\\s*\n|(?:--[A-Za-z0-9-]+(?:[ =][^[:space:]]+)?\s+|\+[A-Za-z0-9_.-]+\s+))' \
.github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== DIFF WHITESPACE =='
git diff --check "$(git merge-base HEAD main)" HEAD
printf '%s\n' '== ALL CHANGES OUTSIDE AF-01 =='
git diff --name-only "$(git merge-base HEAD main)" HEAD -- . ':(exclude).github' ':(exclude)specs/015-af-01-trusted-development-baseline'Length of output: 1634 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== AUDIT LINE NUMBERS: USES, CARGO, YAML ERROR HANDLING =='
cat -n .github/scripts/audit_workflow_trust.py | sed -n '145,185p;230,335p'
printf '%s\n' '== TEST LINE NUMBERS =='
cat -n .github/scripts/test_audit_workflow_trust.py | sed -n '100,245p'
printf '%s\n' '== ROOT ACTION METADATA =='
if [ -f action.yml ]; then
cat -n action.yml
fi
if [ -f action.yaml ]; then
cat -n action.yaml
fi
printf '%s\n' '== YAML SPECIAL-FORM SEARCH =='
rg -n --glob '*.yml' --glob '*.yaml' \
'^\s*(["'\''](?:uses|runs-on|timeout-minutes|permissions|image)["'\'']|<<:|[^#]*&[A-Za-z0-9_-]+|[^#]*\*[A-Za-z0-9_-]+)' \
.github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== CARGO LINE-CONTINUATION/GLOBAL-OPTION SEARCH =='
rg -n -U --glob '*.yml' --glob '*.yaml' \
'cargo\s*(\\\s*\n|(?:--[A-Za-z0-9-]+(?:[ =][^[:space:]]+)?\s+|\+[A-Za-z0-9_.-]+\s+))' \
.github action.yml action.yaml 2>/dev/null || true
printf '%s\n' '== DIFF WHITESPACE =='
git diff --check "$(git merge-base HEAD main)" HEAD
printf '%s\n' '== ALL CHANGES OUTSIDE AF-01 =='
git diff --name-only "$(git merge-base HEAD main)" HEAD -- . ':(exclude).github' ':(exclude)specs/015-af-01-trusted-development-baseline'Length of output: 21294
I found two substantive AF-01 Stack A findings.
The workflow diff preserves the existing CF-06 commands and registry smoke commands. The CI changes add the audit gates before Rust gates. The aggregation-only I verified that the requested commit and canonical merge base match. I also verified that no changed paths exist outside
|
|
Warning Review limit reachedNext included review available in 58 seconds. View limit detailsLimit details: You’ve used all 10 included reviews currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
Comment |
I reviewed the supplied Findings
Areas that look correct
Counterexample coverageThe tests cover the main happy path and several direct mutations, but they do not cover malformed policy shapes, unknown permission keys, multiple checkout steps, ambiguous/chained Cargo commands, quoted/commented Cargo text, or whether timeout equality versus maximum semantics is intended. These should be added before treating the audit as a complete fail-closed control. |
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
@qodo review Fresh exact-head review required after self-review hardening. Review current head |
|
@coderabbitai review Fresh exact-head review required after self-review hardening. Review current head |
Code Review by Qodo
1.
|
| "require_container_digest": true, | ||
| "require_checkout_credentials_disabled": true, | ||
| "require_external_uses_full_sha": true |
There was a problem hiding this comment.
2. Trust rules lack rationale 📘 Rule violation ▣ Testability
The three public boolean trust rules are published only as configuration keys, without a rationale field or corresponding documentation explaining why each rule exists. Their positive and counterexample tests do not satisfy the checklist's separate rationale requirement.
Agent Prompt
## Issue description
The public workflow trust policy defines three boolean rules without documenting the rationale for each rule.
## Issue Context
Preserve the current machine-readable values and audit behavior. Add an explicit rationale field supported by the policy format, or a clearly linked documentation section that explains why digest-bound containers, credentialless checkout, and full-SHA external references are required.
## Fix Focus Areas
- .github/workflow-trust-policy.json[14-16]
- specs/015-af-01-trusted-development-baseline/stack-a-inventory.md[96-106]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Review findingsThe requested invariants are enforced for the currently tracked workflows: all runners and timeouts are explicit, checkout steps use However, I would not mark this fully passing yet:
The narrowed container discovery itself correctly limits checks to job |
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
@qodo review Final fresh review request for current exact head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
expected='0be4766d3f2631f1dba24f03706da8ffe81f3279'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nEXPECTED=%s\n' "$actual" "$expected"
if [ "$actual" != "$expected" ]; then
printf 'ERROR: checkout is not the requested exact head\n' >&2
exit 2
fi
base="$(git merge-base HEAD main)"
printf 'MERGE_BASE=%s\n' "$base"
printf '\n== changed files ==\n'
git diff --name-status "$base" HEAD
printf '\n== changed AF-01 stat ==\n'
git diff --stat "$base" HEAD -- .github specs/015-af-01-trusted-development-baseline
printf '\n== audit outline ==\n'
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== test outline ==\n'
ast-grep outline .github/scripts/test_audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== implementation ==\n'
cat -n .github/scripts/audit_workflow_trust.py
printf '\n== tests ==\n'
cat -n .github/scripts/test_audit_workflow_trust.py
printf '\n== policy ==\n'
cat -n .github/workflow-trust-policy.json
printf '\n== workflow diffs ==\n'
git diff --find-renames "$base" HEAD -- .github/workflows
printf '\n== current workflow/action metadata files ==\n'
fd -HI -t f -e yml -e yaml .github action.yml action.yaml 2>/dev/null | sort | while IFS= read -r f; do
printf '\n--- %s ---\n' "$f"
cat -n "$f"
done
printf '\n== static source checks ==\n'
python3 -m py_compile .github/scripts/audit_workflow_trust.py .github/scripts/test_audit_workflow_trust.py
python3 -m json.tool .github/workflow-trust-policy.json >/dev/null
git diff --check "$base" HEAD
printf 'STATIC_CHECKS=OK\n'Length of output: 48470 🏁 Script executed: #!/bin/bash
set -euo pipefail
previous='e844f5b306cbf201cc31fa893539d2ed0875486c'
current='0be4766d3f2631f1dba24f03706da8ffe81f3279'
printf '%s\n' '== incremental diff from prior reviewed head =='
git diff --find-renames "$previous" "$current" -- \
.github/scripts/audit_workflow_trust.py \
.github/scripts/test_audit_workflow_trust.py \
.github/workflow-trust-policy.json \
.github/workflows \
specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '%s\n' '== tracked security files =='
git ls-files -z | python3 -c '
import sys
from pathlib import PurePosixPath
for path in sorted(p for p in sys.stdin.buffer.read().decode().split("\0") if p):
name = PurePosixPath(path).name
if (path.startswith(".github/workflows/") and name.endswith((".yml", ".yaml"))) or name in {"action.yml", "action.yaml"}:
print(path)
'
printf '%s\n' '== full current workflow source =='
while IFS= read -r f; do
printf '\n--- %s ---\n' "$f"
cat -n "$f"
done < <(git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)')
printf '%s\n' '== parser-relevant implementation and test locations =='
rg -n -C 4 \
'USES_RE|STEP_USES_RE|CARGO_RE|def _container_images|def _job_ranges|def _parse_permissions|def _policy_errors|quoted|continuation|toolchain|container|persist-credentials' \
.github/scripts/audit_workflow_trust.py .github/scripts/test_audit_workflow_trust.py
printf '%s\n' '== static checks =='
python3 -m py_compile .github/scripts/audit_workflow_trust.py .github/scripts/test_audit_workflow_trust.py
python3 -m json.tool .github/workflow-trust-policy.json >/dev/null
git diff --check "$(git merge-base HEAD main)" HEAD
printf '%s\n' 'STATIC_CHECKS=OK'Length of output: 20520
I found three substantive AF-01 findings.
The incremental hardening correctly moves normal checkout credential checks into shared workflow and Action metadata handling. It also narrows the container scan to job and service containers. These changes do not resolve the parser bypasses above. I verified the requested exact head and canonical merge base. I did not execute repository code or tests. The read-only inspection commands terminated before their final static-check section, so compilation and full file enumeration remain unverified. You are interacting with an AI system. |
|
@coderabbitai review Final fresh review request for current exact head |
Fresh review of
Verified requested regressions:
Finding 5 is already implemented in the current control flow: |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
expected='95a2b4829c7b1a832b1460a4a97aada107e23877'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nEXPECTED=%s\n' "$actual" "$expected"
if [ "$actual" != "$expected" ]; then
printf '%s\n' 'ERROR: checkout is not the requested exact head' >&2
exit 2
fi
base="$(git merge-base HEAD main)"
printf 'MERGE_BASE=%s\n' "$base"
printf '\n== changed files ==\n'
git diff --name-status "$base" HEAD
printf '\n== incremental changes from prior head ==\n'
git diff --find-renames 0be4766d3f2631f1dba24f03706da8ffe81f3279 HEAD -- \
.github/scripts/audit_workflow_trust.py \
.github/scripts/test_audit_workflow_trust.py \
.github/workflow-trust-policy.json \
.github/workflows \
specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '\n== audit outline ==\n'
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== audit implementation ==\n'
cat -n .github/scripts/audit_workflow_trust.py
printf '\n== audit tests ==\n'
cat -n .github/scripts/test_audit_workflow_trust.py
printf '\n== policy ==\n'
cat -n .github/workflow-trust-policy.json
printf '\n== inventory ==\n'
cat -n specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '\n== workflow diffs ==\n'
git diff --find-renames "$base" HEAD -- .github/workflows
printf '\n== tracked workflow and Action metadata ==\n'
while IFS= read -r f; do
printf '\n--- %s ---\n' "$f"
cat -n "$f"
done < <(git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)')
printf '\n== parser and policy focus locations ==\n'
rg -n -C 5 \
'USES|uses|CARGO|cargo|persist-credentials|container|service|image|policy|permissions|runs-on|timeout|exception|yaml|YAML' \
.github/scripts/audit_workflow_trust.py \
.github/scripts/test_audit_workflow_trust.py \
.github/workflow-trust-policy.json
printf '\n== whitespace errors ==\n'
git diff --check "$base" HEADLength of output: 57473 🏁 Script executed: #!/bin/bash
set -euo pipefail
expected='95a2b4829c7b1a832b1460a4a97aada107e23877'
test "$(git rev-parse HEAD)" = "$expected"
base="$(git merge-base HEAD main)"
printf '%s\n' '== tracked workflow and Action metadata paths =='
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort
printf '%s\n' '== current workflow and Action metadata source =='
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort | while IFS= read -r f; do
printf '\n--- %s ---\n' "$f"
cat -n "$f"
done
printf '%s\n' '== full changes outside AF-01 configuration and inventory =='
git diff --name-only "$base" HEAD -- . ':(exclude).github' ':(exclude)specs/015-af-01-trusted-development-baseline'
printf '%s\n' '== workflow changes with function-context =='
git diff --unified=12 "$base" HEAD -- .github/workflows
printf '%s\n' '== parser function regions =='
sed -n '15,240p;270,475p' .github/scripts/audit_workflow_trust.py | cat -n
printf '%s\n' '== test cases that cover parser exceptions and policies =='
sed -n '120,310p' .github/scripts/test_audit_workflow_trust.py | cat -n
printf '%s\n' '== changed file end-of-line checks =='
git diff --check "$base" HEADLength of output: 91075
I found three material AF-01 Stack A findings.
The new Action metadata checkout tests and the non-container I verified the workflow changes preserve the existing CF-06 oracle commands, CI product smoke commands, and registry smoke commands. The changes only add the AF-01 audit gates, pin runners and Actions, add timeouts, disable checkout credential persistence, and reduce the aggregation-only I confirmed the requested exact head and canonical merge base. I did not execute repository code or tests because the review sandbox prohibits executing repository files. I do not approve this head while these fail-open paths remain.
|
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
@qodo review Fresh exact-head review for current final candidate |
|
@coderabbitai review Fresh exact-head review for current final candidate |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
expected='3c84f9a93674a5957fe731f70330ea94404fcf55'
actual="$(git rev-parse HEAD)"
printf 'HEAD=%s\nEXPECTED=%s\n' "$actual" "$expected"
test "$actual" = "$expected"
base="$(git merge-base HEAD main)"
printf 'MERGE_BASE=%s\n' "$base"
printf '\n== changed files vs merge base ==\n'
git diff --name-status "$base" HEAD
printf '\n== remediation diff from prior reviewed head ==\n'
git diff --find-renames 95a2b4829c7b1a832b1460a4a97aada107e23877 HEAD -- \
.github/scripts/audit_workflow_trust.py \
.github/scripts/test_audit_workflow_trust.py \
.github/workflow-trust-policy.json \
.github/workflows \
specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '\n== audit source outline ==\n'
ast-grep outline .github/scripts/audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== test source outline ==\n'
ast-grep outline .github/scripts/test_audit_workflow_trust.py --items all 2>/dev/null || true
printf '\n== audit implementation ==\n'
cat -n .github/scripts/audit_workflow_trust.py
printf '\n== test suite ==\n'
cat -n .github/scripts/test_audit_workflow_trust.py
printf '\n== policy ==\n'
cat -n .github/workflow-trust-policy.json
printf '\n== inventory ==\n'
cat -n specs/015-af-01-trusted-development-baseline/stack-a-inventory.md
printf '\n== tracked workflow and Action metadata paths ==\n'
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort
printf '\n== tracked workflow and Action metadata content ==\n'
git ls-files | awk '($0 ~ /^\.github\/workflows\/.*\.(yml|yaml)$/) || ($0 ~ /(^|\/)action\.ya?ml$/)' | sort |
while IFS= read -r file; do
printf '\n--- %s ---\n' "$file"
cat -n "$file"
done
printf '\n== workflow diff vs merge base ==\n'
git diff --find-renames --unified=16 "$base" HEAD -- .github/workflows
printf '\n== changed paths outside AF-01 areas ==\n'
git diff --name-only "$base" HEAD -- . \
':(exclude).github' \
':(exclude)specs/015-af-01-trusted-development-baseline'
printf '\n== whitespace validation ==\n'
git diff --check "$base" HEADLength of output: 173621
I found three material AF-01 Stack A findings.
The policy validation now prevents removal or disablement of the defined Stack A rules. The flow-style I confirmed that no changed paths exist outside You are interacting with an AI system. |
I found one remaining material fail-open gap: quoted job IDs can escape job coverage.
This violates the stated requirement that every discovered job have exact policy coverage. The fix should either parse job keys with YAML semantics or fail closed whenever the The previous rationale finding, finding 7, appears addressed: all four supported rules now have substantive entries under |
AF-01 Stack A — workflow trust baseline
Canonical base at PR creation:
This PR implements AF-01 Phase 1 / Stack A only. It does not change commandF product semantics, CF-06 oracle identity, CF-10 corpus authority, package/diff/terminology/impact rules, or dependency policy.
T010 — inventory
Adds
stack-a-inventory.mdcovering every tracked workflow and Action metadata file, externaluses:authority, job runners, effective permissions, timeouts, proof containers, checkout credentials, and lockfile-consuming Cargo commands at the canonical planning base.T011/T012 — machine-checkable policy + deterministic audit
Adds:
.github/workflow-trust-policy.json.github/scripts/audit_workflow_trust.pyThe audit discovers all tracked
.github/workflows/*.yml|*.yamland every trackedaction.yml|action.yamlat any depth, rejects unplanned workflows/jobs, mutable externaluses:references, mutable latest runner labels, missing/excessive timeouts, unresolved/default or overbroad token permissions, checkout credentials left enabled, mutable job/service container identities, and configured lockfile-consuming Cargo commands without--locked. Local./actions remain allowed.Output is deterministic JSON and exceptions require explicit rule/path/reason/revisit evidence.
T013/T016 — counterexamples and future-file coverage
Adds a repository-owned test suite covering:
action.yamlmutable uses;persist-credentials: false;*-latestrunner labels;T014/T015 — current workflow hardening
ci.ymlnow uses full-SHA checkout/rust-toolchain references, credentialless checkout,ubuntu-24.04, explicit 30-minute timeout, and executes the AF-01 audit tests plus live repository audit before Rust gates.cf06-oracle.ymlpreserves all existing oracle commands/evidence while fixing runners toubuntu-24.04, bounding all three jobs, and reducing the aggregation-onlyoracle-proofjob topermissions: {}.registry-download-smoke.ymlmoves its runner fromubuntu-latesttoubuntu-24.04; existing timeout, pinned actions, credentials, and semantic smoke behavior remain unchanged.The CF-11/CF-11G/CF-12/CF-13 proof workflows already satisfied the selected Stack A baseline and are not semantically rewritten.
Qualification required before merge
ci;No PASS is assumed from queue state or review launch.
Summary by cubic
Establishes a deterministic trust baseline for all tracked GitHub workflows and Action metadata. Previously CI allowed mutable runners and action references; now every workflow and job must match the policy and pass the audit before Rust gates run.
Changes
action.yml/action.yamlmetadata, checkout credentials, runners, permissions, timeouts, container digests, and lockfile-consuming Cargo commands.uses:keys fail closed.oracle-proofpermissions, and disables checkout credentials where needed.Written for commit 3c84f9a. Summary will update on new commits.