ci(security): security-gates workflow (split 2/4 of #1317)

[SharedQA]

Split out of #1317 (too large to review as one). This is the security slice.

What

• .github/workflows/security-gates.yml — mandatory gates, none using continue-on-error:
  • secrets-scan — TruffleHog, verified secrets are a hard fail.
  • sast — Semgrep p/default (report-only during ratchet-in).
  • deps-audit — cargo audit (RUSTSEC) + pip-audit (CVEs). Unfixable transitive CVEs are waived with tracking (nltk==3.9.1 / cryptography<47 pinned by airbyte-cdk).
  • trivy-config — Trivy IaC + filesystem (CRITICAL/HIGH).
  • All third-party actions + the SAST image pinned to commit SHAs/digests.
• .trivyignore.

Notes

• Self-standing: scanners run directly (no Makefile), no docs/helm/release files.
• Part of splitting ci: release train + coverage ratchet (release slice; split from the omnibus) #1317; ci: release train + coverage ratchet (release slice; split from the omnibus) #1317 keeps the release-train + coverage slice.

Summary by CodeRabbit

• New Features
  • Added a blocking “Security Gates” workflow for pull requests with manual and nightly scheduled runs, combining secrets scanning, SAST, dependency audits, and Trivy checks for misconfigurations and vulnerabilities.
• Bug Fixes
  • Enforced stricter gating behavior so only documented exceptions can pass; other high-risk findings still prevent merges.
• Documentation
  • Updated Trivy waiver guidance to explicitly track only the approved, documented misconfiguration exceptions.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 85ef135c-3bf9-4c48-b8df-bc0df9ddde8d

📥 Commits

Reviewing files that changed from the base of the PR and between cc4d9f2 and 2af3061.

📒 Files selected for processing (2)

• .github/workflows/security-gates.yml
• .trivyignore

✅ Files skipped from review due to trivial changes (1)

• .trivyignore

🚧 Files skipped from review as they are similar to previous changes (1)

• .github/workflows/security-gates.yml

---

📝 Walkthrough

Walkthrough

Adds a new GitHub Actions security-gates workflow with blocking scans for secrets, SAST, dependency audits, and Trivy, plus documented Trivy waivers.

Changes

Security Gates Workflow

Layer / File(s)	Summary
Workflow header, triggers, permissions, and concurrency .github/workflows/security-gates.yml	Defines the workflow name, pull request/manual/nightly triggers, read-only contents permissions, ref-scoped concurrency with cancellation, and comments describing the blocking-gate and pinning rules.
Secrets scan and Semgrep jobs .github/workflows/security-gates.yml	Adds secrets-scan with TruffleHog base/head selection for PRs and sast with Semgrep in a pinned container image, one excluded rule, and report-only behavior.
Rust and Python dependency audit job .github/workflows/security-gates.yml	Adds deps-audit with cargo audit for src/backend, Python manifest scanning in src/ingestion, fixed ignore sets, accumulated exit codes, and a failure path when no manifests are found.
Trivy scans and waivers .github/workflows/security-gates.yml, .trivyignore	Adds trivy-config with config and filesystem scans constrained to CRITICAL,HIGH, hard-fail exit codes, and an updated .trivyignore with four annotated waiver entries.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hopped through gates of steel and light,
With TruffleHog and Semgrep bright.
Cargo, pip, and Trivy sing,
While waivers hum on guarded wing.
The warren’s scans այժմ stand tall and neat,
With secrets tucked and checks complete.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the security-gates workflow and matches the PR’s main change, including its split-from-#1317 context.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/security-gates.yml:
- Around line 113-122: The manifest audit step in the security-gates workflow
can exit successfully without auditing anything when the find/filter pipeline
produces no files. Update the shell logic around the while/read loop and the rc
handling so it tracks whether at least one Python manifest was scanned, and
force a failure if none were found or audited; keep the existing pip-audit
execution paths for requirements files and pyproject.toml entries, but gate the
final exit on that new “found any manifest” state.
- Around line 42-44: The fetch-depth logic in the security-gates workflow is too
shallow for pull_request runs and can miss the full base.sha..head.sha range
used by TruffleHog. Update the fetch-depth expression so that schedule runs
still use full history and pull_request runs also fetch full history instead of
50, keeping the existing workflow job and the fetch-depth setting in sync with
the secrets scan requirements.
- Around line 4-7: The Semgrep gate is not currently enforcing the blocking
contract because the security-gates workflow runs the semgrep scan without a
failing exit on findings. Update the job in security-gates.yml around the
semgrep scan step to use the failing behavior (for example, the scan’s
error-on-findings mode) so the job exits nonzero when issues are found, or if
this is intentional temporary ratcheting, add a clearly documented exception
with owner and exit condition in the top-level contract.
- Line 139: The Trivy ignore entries are including inline rationale text, which
prevents the ignore IDs from matching correctly. Update the handling around the
`.trivyignore` entries so only the ignore IDs remain on each entry, and move any
explanatory text to a separate comment line before the entry. Use the existing
`trivyignores` configuration and the `.trivyignore` file format as the reference
points when making the change.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6fd729dc-24fd-4f59-9294-0fb94a83a52c

📥 Commits

Reviewing files that changed from the base of the PR and between 18549e3 and cc4d9f2.

📒 Files selected for processing (2)

• .github/workflows/security-gates.yml
• .trivyignore

[ktursunov]

Closing without merging. The single bundled security-gates.yml is being decomposed into per-tool tasks under the security-scanner umbrella #1478, so each scanner can be triaged and ratcheted to blocking independently (per #1478's acceptance: report-only first, agree gate thresholds with the team before flipping to blocking).

Follow-ups:

• Trivy → ci(security): Trivy misconfig + fs scan gate (split from #1463) #1796
• Semgrep (SAST) → ci(security): Semgrep SAST gate (split from #1463) #1797

TruffleHog (secrets-scan) and cargo-audit/pip-audit (deps-audit) remain tracked under #1478 and will be split out as needed. The workflow YAML here stays available on branch claude/split-security-gates to lift from.