Skip to content

016 US6: CodeQL — the deepest corpus, and the one thing it may not ask for - #17

Merged
jlgore merged 1 commit into
mainfrom
019-codeql-us6
Jul 30, 2026
Merged

016 US6: CodeQL — the deepest corpus, and the one thing it may not ask for#17
jlgore merged 1 commit into
mainfrom
019-codeql-us6

Conversation

@jlgore

@jlgore jlgore commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Closes T058–T061. 016's last user story; the phase was deferred until the external tier (US3) was proven, and it now is.

What lands

A CodeQL adapter that consumes an operator-provisioned, version-pinned bundle and analyses only build-mode: none languages.

[exec]
allow = ["!/opt/codeql-bundle/codeql/codeql"]   # the grant names the bundle's own CLI

[security.scanners.codeql]
bundle_version = "codeql-bundle-v2.26.1"        # or "2.26.1" — both spell the same pin

Analysable: actions, csharp, java, javascript, python, ruby, plus CodeQL's own aliases so typescript reaches the javascript extractor. Everything else is declined by name, carrying what is analysable so the refusal is actionable.

Why the language rule is absolute

Extracting a compiled language works by intercepting the build's process spawns (--begin-tracing, --trace-process-name). Admitting that means admitting every compiler and linker the build happens to invoke — unbounded, un-attenuable, and the exact widening SC-006 exists to prevent. Half-support would be worse than none: a traced language extracted without tracing yields a thin database, and a thin database yields few findings, which reads exactly like clean code.

The supported set is a static list, deliberately. CodeQL decides tracedness by stat'ing <extractor>/tools/tracing-config.lua in the provisioned distribution (codeql-action/src/codeql.ts:535) — a file bee cannot read from the harness without reading around its own sandbox. The list is backed by what codeql-action's own handling shows, and it fails closed: unlisted ⇒ declined, so staleness costs coverage and never correctness. rust is absent on purpose — very likely buildless, but "likely" is not evidence.

Two contract shapes had to give

Before After Why
argv() -> Vec<String> steps() -> Vec<Vec<String>> an analysis is create-then-analyse. Intermediate steps are judged by exit status (they have no report to be judged by); the last is still judged by the report, because an exit status conflates "findings exist" with "run failed" (R6).
probe(grant) probe(grant, req) "can this run" and "can this answer what was asked" are different questions, and only the second knows about languages.
preflight() / verify_preflight() asking a binary what it is means running it, and running it from the harness would run it outside the scope every other child is held to (Constitution III).

Opengrep comes along as a one-step adapter with no preflight; its behaviour is unchanged.

Testing

tests/codeql_adapter.rs — 10 cases, driven through ScanTool against a stub CLI rather than against the adapter directly, because the wiring is where a refusal would leak. Every refusal case asserts what did not happen: the stub logs each invocation, and the log must be empty — or, for a version mismatch, hold exactly one entry with nothing after it.

Covers both named scenarios plus two the stub made cheap: a CLI whose version output is unreadable is a mismatch rather than a pass, and an unpinned bundle produces no preflight and no command at all.

  • Full suite 696 passed / 0 failed (--features sec)
  • cargo clippy --workspace --all-targets --features sec,astgrep-rust,astgrep-python -- -D warnings clean
  • cargo fmt --all -- --check clean
  • Default build and --features scanners standalone both green (SC-009 unmoved)

What this does not prove

A stub cannot show that CodeQL, handed these arguments, produces useful findings. Proven here is bee's half — the argv it authors, the order it runs, every path on which it refuses, and the report reaching the ledger. The real walkthrough against a provisioned bundle is filed as T073, and it is also where rust gets settled: if the bundle's Rust extractor ships no tools/tracing-config.lua, it joins the list and bee can analyse itself.

🤖 Generated with Claude Code

…US6)

CodeQL is the last adapter and the only one whose *availability* is a real
question. Opengrep either runs or is absent. A CodeQL bundle can be present,
correctly pinned, and still be unable to answer — because the language asked
for can only be extracted by watching a build.

That is the line this holds. Extraction for a compiled language works by
intercepting the build's process spawns (`--begin-tracing`,
`--trace-process-name`), which would mean admitting every compiler and linker
the build happens to invoke: unbounded, un-attenuable, and the exact widening
SC-006 exists to prevent. So databases are built one way, `--build-mode=none`,
and everything else is declined by name. Half-support would be worse than none
— a traced language extracted without tracing yields a thin database, and a
thin database yields few findings, which reads exactly like clean code.

The supported set is a static list, and that is a decision rather than a
shortcut. CodeQL answers tracedness by stat'ing
`<extractor>/tools/tracing-config.lua` inside the provisioned distribution
(`codeql-action/src/codeql.ts:535`), a file bee cannot read from the harness
without reading around its own sandbox. So the list lives in source, backed by
what codeql-action's own handling shows, and it fails closed: unlisted means
declined, so staleness costs coverage and never correctness. `rust` is
deliberately absent — very likely buildless, but "likely" is not evidence, and
the cost of being wrong is a scan that quietly under-reports.

Two shapes had to give. `argv` became `steps`, because an analysis is
create-then-analyse; an intermediate step is judged by its exit status, since
it has no report to be judged by, while the last is still judged by the report.
And `probe` now takes the request, because "can this run" and "can this answer
what was asked" are different questions and only the second one knows about
languages. The version pin became `preflight`/`verify_preflight` rather than
part of `probe`: asking a binary what it is means running it, and running it
from the harness would run it outside the scope every other child is held to.

Every refusal is proven by what did not happen. The stub CLI logs each
invocation, and each refusal case asserts the log is empty — or, for a version
mismatch, holds exactly one entry and nothing after it.

Not walked against a real bundle; that is T073, filed, and it is also where
`rust` gets settled.

Closes T058–T061.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jlgore
jlgore merged commit fcfba32 into main Jul 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant