From f4f6b58604dd9502b2ef9635f022aa9d9d88f3a2 Mon Sep 17 00:00:00 2001 From: Byron Williams Date: Wed, 2 Sep 2026 14:21:21 -0700 Subject: [PATCH] fix(ci)!: remove GitHub Advanced Security guidance from mirrored agents/skills GitHub now bills Advanced Security (Code Security) separately, so CodeQL code scanning, actions/dependency-review-action, and SARIF ingestion into the Security tab no longer function on the free tier. This repo has no .github/workflows to delete, but it carries mirrored copies of the .claude repo's agents and skills under plugins/wff-code/ and plugins/wff-chat/ that instructed Claude to use or check for these controls. - plugins/wff-code/agents/ossf-compliance-auditor.md: exclude dependency-review-action and codeql-action/analyze from the continue-on-error bypass allowlist (both are retired, not merely bypassed); stop recommending a CodeQL workflow for a Scorecard SAST score of 10 - plugins/wff-code/skills/ci-fix/SKILL.md and plugins/wff-chat/skills/ci-fix/SKILL.md: reframe the dependency-review license-failure guidance around the surviving SBOM/Trivy gate - plugins/wff-code/skills/pr-review/workflows/pr-fix.md: mark the SARIF/ code-scanning orphan-check guidance and the CodeQL default-setup worked example as legacy (pre-2026-09); update the static-validation table to describe the artifact-upload SARIF pattern - plugins/wff-code/skills/pr-review/workflows/pr-review.md: replace the stale CodeQL example in the transient-infrastructure test - plugins/wff-code/skills/repo-compliance/SKILL.md: remove the retired dependency-review-action dormant-capability example; note that codeql.yml absence is now fleet-wide, not a visibility-scoped exemption Companion PR: ByronWilliamsCPA/.claude#295 (source of truth for this mirror; deletes codeql.yml/dependency-review.yml and retires the corresponding CI-036/CI-081 standards-manifest checks, neither of which exist in this repo). Co-Authored-By: Claude Sonnet 4.6 --- plugins/wff-chat/skills/ci-fix/SKILL.md | 8 +++-- .../agents/ossf-compliance-auditor.md | 17 +++++++++-- plugins/wff-code/skills/ci-fix/SKILL.md | 8 +++-- .../skills/pr-review/workflows/pr-fix.md | 30 +++++++++++-------- .../skills/pr-review/workflows/pr-review.md | 6 ++-- .../wff-code/skills/repo-compliance/SKILL.md | 20 ++++++++----- 6 files changed, 59 insertions(+), 30 deletions(-) diff --git a/plugins/wff-chat/skills/ci-fix/SKILL.md b/plugins/wff-chat/skills/ci-fix/SKILL.md index e213567..908bfea 100644 --- a/plugins/wff-chat/skills/ci-fix/SKILL.md +++ b/plugins/wff-chat/skills/ci-fix/SKILL.md @@ -186,16 +186,18 @@ changes"), fix it as a real bug fix for main, and do not attribute it to the PR same pattern applies to any path-filtered job (release workflows, security-analysis when no Python files changed). -### Dependency-review license failures may be pre-existing policy debt (Obs 255) +### SBOM license failures may be pre-existing policy debt (was: Obs 255, dependency-review-action) -When the `dependency-review-action` gate fails on a license (not a vulnerability), distinguish "introduced by this PR" from "surfaced by this PR" before fixing. The gate only evaluates packages *changed* in the PR, so a routine Renovate version bump can surface a license-policy gap that already existed in the base branch. +`dependency-review.yml` and the `actions/dependency-review-action` gate it ran were removed fleet-wide (2026-09): the action now requires paid GitHub Advanced Security (Code Security) and no longer functions on the free tier. This diff-scoped PR-time license/vulnerability gate is gone; the fleet's remaining license-compliance control is the post-merge, full-lockfile SBOM gate (`sbom.yml`, Trivy-based). + +The same "introduced by this PR" vs. "surfaced by this PR" distinction still applies to that gate: it evaluates the full lock file on every run, so a routine Renovate version bump can surface a license-policy gap that already existed in the base branch, not one the bump introduced. ```bash # Was the flagged package already in main's lock file at an older version? git show origin/main:uv.lock | grep -A2 'name = ""' ``` -If the package already exists on the base branch, the bump did not introduce the problem; it exposed it. Route the fix to the workflow allowlist (`allow-dependencies-licenses` on the dependency-review step) or a documented exemption, not to a revert of the bump. Diff-scoped gates (dependency review, patch coverage) report at the changed line or package, but the root cause may be pre-existing debt in the base branch. +If the package already exists on the base branch, the bump did not introduce the problem; it exposed it. Route the fix to the SBOM workflow's license denylist/allowlist or a documented exemption, not to a revert of the bump. Diff-scoped and full-lockfile gates alike can surface pre-existing debt in the base branch rather than something the current PR introduced. ### startup_failure with zero jobs is a caller/callee contract mismatch (Obs 266) diff --git a/plugins/wff-code/agents/ossf-compliance-auditor.md b/plugins/wff-code/agents/ossf-compliance-auditor.md index 3134ca3..03c894a 100644 --- a/plugins/wff-code/agents/ossf-compliance-auditor.md +++ b/plugins/wff-code/agents/ossf-compliance-auditor.md @@ -358,7 +358,15 @@ remediation: | **Security gate `continue-on-error` bypass (CI-SEC-002):** Read all YAML files under `.github/workflows/` using Glob, then Read each one. For any workflow step that meets **both** conditions: -1. The step's `uses:` field references one of: `anchore/scan-action`, `aquasecurity/trivy-action`, `actions/dependency-review-action`, `ossf/scorecard-action`, `snyk/actions/python`, `snyk/actions/node`, `snyk/actions/docker`, `github/codeql-action/analyze`, `returntocorp/semgrep-action` +1. The step's `uses:` field references one of: `anchore/scan-action`, `aquasecurity/trivy-action`, `ossf/scorecard-action`, `snyk/actions/python`, `snyk/actions/node`, `snyk/actions/docker`, `returntocorp/semgrep-action` + +> **`actions/dependency-review-action` and `github/codeql-action/analyze` are intentionally excluded from this allowlist.** +> Both require paid GitHub Advanced Security (Code Security) as of the 2026-09 +> billing change and are removed fleet-wide (`dependency-review.yml` and +> `codeql.yml` deleted); a `continue-on-error` bypass on a step that no longer +> runs anywhere is not a finding. If a repo is found still running either +> action, flag the workflow itself for removal (see repo-compliance CI-036/ +> CI-081, retired) rather than auditing its `continue-on-error` setting. 2. The same step has `continue-on-error: true` > **`gitleaks/gitleaks-action` is intentionally excluded from this allowlist.** @@ -732,8 +740,11 @@ Add at least one `fuzz_*.py` file or `@given`-decorated test per module to clear **Measures:** Whether static analysis tools are integrated in CI. **Score >= 4:** Any SAST tool (Bandit, Semgrep, CodeQL) runs in CI. **Score >= 7:** SAST tool with CWE coverage (Bandit covers most Python CWEs). -**Score = 10:** CodeQL or equivalent runs on every push. -**Note:** Bandit in `security-analysis.yml` clears 4+. For 10, add a CodeQL workflow. +**Score = 10:** CodeQL or an equivalent CWE-scoped SAST tool runs on every push. +**Note:** Bandit in `security-analysis.yml` clears 4+. CodeQL now requires paid +GitHub Advanced Security (Code Security) and is no longer deployed fleet-wide +(`codeql.yml` removed); do not recommend adding it back. Semgrep is the +fleet's remaining path to a 10 if that score is pursued. ### SBOM (Medium) diff --git a/plugins/wff-code/skills/ci-fix/SKILL.md b/plugins/wff-code/skills/ci-fix/SKILL.md index e213567..908bfea 100644 --- a/plugins/wff-code/skills/ci-fix/SKILL.md +++ b/plugins/wff-code/skills/ci-fix/SKILL.md @@ -186,16 +186,18 @@ changes"), fix it as a real bug fix for main, and do not attribute it to the PR same pattern applies to any path-filtered job (release workflows, security-analysis when no Python files changed). -### Dependency-review license failures may be pre-existing policy debt (Obs 255) +### SBOM license failures may be pre-existing policy debt (was: Obs 255, dependency-review-action) -When the `dependency-review-action` gate fails on a license (not a vulnerability), distinguish "introduced by this PR" from "surfaced by this PR" before fixing. The gate only evaluates packages *changed* in the PR, so a routine Renovate version bump can surface a license-policy gap that already existed in the base branch. +`dependency-review.yml` and the `actions/dependency-review-action` gate it ran were removed fleet-wide (2026-09): the action now requires paid GitHub Advanced Security (Code Security) and no longer functions on the free tier. This diff-scoped PR-time license/vulnerability gate is gone; the fleet's remaining license-compliance control is the post-merge, full-lockfile SBOM gate (`sbom.yml`, Trivy-based). + +The same "introduced by this PR" vs. "surfaced by this PR" distinction still applies to that gate: it evaluates the full lock file on every run, so a routine Renovate version bump can surface a license-policy gap that already existed in the base branch, not one the bump introduced. ```bash # Was the flagged package already in main's lock file at an older version? git show origin/main:uv.lock | grep -A2 'name = ""' ``` -If the package already exists on the base branch, the bump did not introduce the problem; it exposed it. Route the fix to the workflow allowlist (`allow-dependencies-licenses` on the dependency-review step) or a documented exemption, not to a revert of the bump. Diff-scoped gates (dependency review, patch coverage) report at the changed line or package, but the root cause may be pre-existing debt in the base branch. +If the package already exists on the base branch, the bump did not introduce the problem; it exposed it. Route the fix to the SBOM workflow's license denylist/allowlist or a documented exemption, not to a revert of the bump. Diff-scoped and full-lockfile gates alike can surface pre-existing debt in the base branch rather than something the current PR introduced. ### startup_failure with zero jobs is a caller/callee contract mismatch (Obs 266) diff --git a/plugins/wff-code/skills/pr-review/workflows/pr-fix.md b/plugins/wff-code/skills/pr-review/workflows/pr-fix.md index 9356eaf..09d4394 100644 --- a/plugins/wff-code/skills/pr-review/workflows/pr-fix.md +++ b/plugins/wff-code/skills/pr-review/workflows/pr-fix.md @@ -1273,8 +1273,7 @@ platform: resolve against the platform's live docs and the CI-pinned tool versio | CI check | Static validation | | --- | --- | | ClusterFuzzLite | For each fuzz target declared in workflow: verify file exists at the declared path, has the correct extension (`.py` for Python), and compiles with `python3 -m py_compile {target}` | -| Dependency Review | Verify the action version pin resolves (check format: `actions/dependency-review-action@vN`) | -| SARIF upload | If workflow references a SARIF file path, verify the generating step would produce it (check step ordering and output paths) | +| SARIF-producing scanners (Trivy, Snyk, Scorecard, SBOM) | If workflow references a SARIF file path, verify the generating step would produce it (check step ordering and output paths). SARIF now feeds `actions/upload-artifact`, not `github/codeql-action/upload-sarif`/Security tab ingestion, since GitHub Advanced Security is no longer free; verify the artifact-upload step exists where a scanner's SARIF was its only output. | | SonarCloud | Verify `sonar-project.properties` has non-placeholder values for `sonar.organization` and `sonar.projectKey` | | Codecov | If `codecov.yml` exists, verify it parses as valid YAML and references existing flag names | @@ -1621,11 +1620,16 @@ still serves dangling commits, so use `compare`, not existence. Validate the fix `if:`). When the failure appeared right after an edit, confirm causation by reverting the suspected change on the current base before committing to a fix direction. -**SARIF / code-scanning orphan checks:** When "Code scanning results / *" checks remain -in `queued` state indefinitely after a push, check whether the upstream analysis job was -path-filtered or skipped. Security analysis workflows on config-only or docs-only PRs are -commonly path-filtered, leaving their SARIF upload result checks permanently pending with -no source to resolve them. +**SARIF / code-scanning orphan checks (legacy, pre-2026-09):** GitHub's CodeQL code scanning +and SARIF ingestion into the Security tab now require paid GitHub Advanced Security (Code +Security); `codeql.yml` and `github/codeql-action/upload-sarif` steps were removed fleet-wide, +so "Code scanning results / *" checks should no longer appear on new PRs at all. If one is +still visible, it is a leftover from before the billing change, not a live analysis: treat it +as permanently orphaned (not merely path-filtered) and, if it recurs, have the repo owner +disable "Code scanning: Default setup" in repo Settings > Code security so GitHub stops +registering the check context. The pre-2026-09 mechanics below (queued indefinitely because the +upstream analysis job was path-filtered or skipped on config-only/docs-only PRs) still apply to +any other SARIF-producing workflow, such as a Trivy or Snyk scan that guards a path filter. ```bash gh pr view "$PR_NUMBER" --repo "$OWNER/$REPO" --json mergeable,mergeStateStatus \ @@ -1657,11 +1661,13 @@ change can clear it. Before treating any single FAILURE as a blocker that warran Phase C re-fix cycle, run `gh pr view "$PR_NUMBER" --repo "$OWNER/$REPO" --json mergeable,mergeStateStatus` and a required-contexts lookup; classify by the merge- eligibility API and the required-context list, not by the check's terminal color alone. -The worked example is a CodeQL default-setup job ("Analyze (javascript-typescript)") -failing "no source code seen during build" on a repo with zero JS/TS source: it is enabled -org-wide by the recommended code-security config, is non-required, and merges straight -through. This is the same advisory logic the queued/PENDING SARIF row applies, extended to -the hard-FAILED terminal case. +The worked example (pre-2026-09, when CodeQL default setup was still free) was a CodeQL +default-setup job ("Analyze (javascript-typescript)") failing "no source code seen during +build" on a repo with zero JS/TS source: it was enabled org-wide by the recommended +code-security config, was non-required, and merged straight through. CodeQL now requires +paid GitHub Advanced Security and no longer runs fleet-wide, so this specific check should +not reappear; the underlying advisory logic (non-required + `UNSTABLE` + `MERGEABLE` merges +through regardless of terminal color) still applies to any other non-required check. **Green-but-BLOCKED: distinguish the cause before acting.** "All checks green" is necessary but not sufficient for mergeability; `mergeStateStatus` is the authoritative diff --git a/plugins/wff-code/skills/pr-review/workflows/pr-review.md b/plugins/wff-code/skills/pr-review/workflows/pr-review.md index 6a52e4b..279e663 100644 --- a/plugins/wff-code/skills/pr-review/workflows/pr-review.md +++ b/plugins/wff-code/skills/pr-review/workflows/pr-review.md @@ -219,8 +219,10 @@ gh run view {RUN_ID} --repo "$OWNER/$REPO" --log \ - Log matches an infra signature, or the conclusion is `CANCELLED` (collateral cancel): emit `[Critical - likely transient, rerun]` with the matched evidence line. The remediation is a re-run, not a code change. A docs-only or config-only diff that - fails a code-analysis check (CodeQL, security-analysis) is a strong tell for this - class, since such a diff cannot cause that failure. + fails a code-analysis check (Bandit, SonarCloud, security-analysis) is a strong tell + for this class, since such a diff cannot cause that failure. (CodeQL was retired + fleet-wide 2026-09; it should no longer appear as a check at all, see ci-fix + guidance on legacy SARIF/code-scanning checks.) - No infra signature and the log points at the diff: emit `[Critical - PR-introduced]`; the fix is in the PR's diff. diff --git a/plugins/wff-code/skills/repo-compliance/SKILL.md b/plugins/wff-code/skills/repo-compliance/SKILL.md index 89b7182..667d865 100644 --- a/plugins/wff-code/skills/repo-compliance/SKILL.md +++ b/plugins/wff-code/skills/repo-compliance/SKILL.md @@ -173,11 +173,13 @@ Two failure modes make a "required" CI gate non-enforcing while every presence c Before evaluating a NEW tool for a capability (license compliance, SBOM, vuln scanning), grep the existing reusable workflows and the manifest for an existing or dormant equivalent: -disabled flags, warn-only gates, unused action inputs (e.g., a `deny-licenses` lever in -`dependency-review-action`, a `python-sbom.yml` license job, a REUSE/SPDX gate). Frame the -evaluation as the marginal gain over what already exists (including the cost of redundancy), -not as a blank-slate integration. Capabilities are often already present but disabled or -shallow. +disabled flags, warn-only gates, unused action inputs (e.g., the `fail-on-forbidden-licenses` +flag in a `python-sbom.yml` license job, a REUSE/SPDX gate). Frame the evaluation as the +marginal gain over what already exists (including the cost of redundancy), not as a +blank-slate integration. Capabilities are often already present but disabled or shallow. +`actions/dependency-review-action`'s `deny-licenses` lever, formerly cited here as an example, +was retired fleet-wide (2026-09) along with `dependency-review.yml`: the action now requires +paid GitHub Advanced Security. Do not propose re-enabling it as a "dormant capability." **CI pinning exception list: slsa-github-generator requires tag refs** @@ -280,9 +282,13 @@ limitations (e.g., GitHub PVR unavailable) rather than compliance gaps. **Examples:** - Repo `homelab-infra` has `repositoryType: "infrastructure"` and `isPrivate: true` - Type profile exempts `release.yml`, `release-sign.yml`, `sbom.yml`, `coverage.yml`, `python-compatibility.yml`, `reuse.yml` - - Visibility profile additionally exempts `codeql.yml` (GHAS required) and check IDs `OSSF-001`, `OSSF-006` - - Absent `release.yml` is logged as `EXEMPT (infrastructure type)`, absent `codeql.yml` as `EXEMPT (private repo)` + - Visibility profile additionally exempts check IDs `OSSF-001`, `OSSF-006` + - Absent `release.yml` is logged as `EXEMPT (infrastructure type)` - OSSF-001 finding is suppressed with `EXEMPT (private repo: badge API is public OSS only)` + - `codeql.yml` is absent fleet-wide (2026-09): CodeQL now requires paid GitHub Advanced + Security on every repo, public or private, so this is no longer a visibility-scoped + exemption. Do not log a `codeql.yml`-absence finding for any repo type or visibility; + it is expected everywhere, not just on private/infrastructure repos. ## Coordinator Prompt Template