016 US6: CodeQL — the deepest corpus, and the one thing it may not ask for - #17
Merged
Conversation
…US6) CodeQL is the last adapter and the only one whose *availability* is a real question. Opengrep either runs or is absent. A CodeQL bundle can be present, correctly pinned, and still be unable to answer — because the language asked for can only be extracted by watching a build. That is the line this holds. Extraction for a compiled language works by intercepting the build's process spawns (`--begin-tracing`, `--trace-process-name`), which would mean admitting every compiler and linker the build happens to invoke: unbounded, un-attenuable, and the exact widening SC-006 exists to prevent. So databases are built one way, `--build-mode=none`, and everything else is declined by name. Half-support would be worse than none — a traced language extracted without tracing yields a thin database, and a thin database yields few findings, which reads exactly like clean code. The supported set is a static list, and that is a decision rather than a shortcut. CodeQL answers tracedness by stat'ing `<extractor>/tools/tracing-config.lua` inside the provisioned distribution (`codeql-action/src/codeql.ts:535`), a file bee cannot read from the harness without reading around its own sandbox. So the list lives in source, backed by what codeql-action's own handling shows, and it fails closed: unlisted means declined, so staleness costs coverage and never correctness. `rust` is deliberately absent — very likely buildless, but "likely" is not evidence, and the cost of being wrong is a scan that quietly under-reports. Two shapes had to give. `argv` became `steps`, because an analysis is create-then-analyse; an intermediate step is judged by its exit status, since it has no report to be judged by, while the last is still judged by the report. And `probe` now takes the request, because "can this run" and "can this answer what was asked" are different questions and only the second one knows about languages. The version pin became `preflight`/`verify_preflight` rather than part of `probe`: asking a binary what it is means running it, and running it from the harness would run it outside the scope every other child is held to. Every refusal is proven by what did not happen. The stub CLI logs each invocation, and each refusal case asserts the log is empty — or, for a version mismatch, holds exactly one entry and nothing after it. Not walked against a real bundle; that is T073, filed, and it is also where `rust` gets settled. Closes T058–T061. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes T058–T061. 016's last user story; the phase was deferred until the external tier (US3) was proven, and it now is.
What lands
A CodeQL adapter that consumes an operator-provisioned, version-pinned bundle and analyses only
build-mode: nonelanguages.Analysable:
actions,csharp,java,javascript,python,ruby, plus CodeQL's own aliases sotypescriptreaches thejavascriptextractor. Everything else is declined by name, carrying what is analysable so the refusal is actionable.Why the language rule is absolute
Extracting a compiled language works by intercepting the build's process spawns (
--begin-tracing,--trace-process-name). Admitting that means admitting every compiler and linker the build happens to invoke — unbounded, un-attenuable, and the exact widening SC-006 exists to prevent. Half-support would be worse than none: a traced language extracted without tracing yields a thin database, and a thin database yields few findings, which reads exactly like clean code.The supported set is a static list, deliberately. CodeQL decides tracedness by stat'ing
<extractor>/tools/tracing-config.luain the provisioned distribution (codeql-action/src/codeql.ts:535) — a file bee cannot read from the harness without reading around its own sandbox. The list is backed by whatcodeql-action's own handling shows, and it fails closed: unlisted ⇒ declined, so staleness costs coverage and never correctness.rustis absent on purpose — very likely buildless, but "likely" is not evidence.Two contract shapes had to give
argv() -> Vec<String>steps() -> Vec<Vec<String>>probe(grant)probe(grant, req)preflight()/verify_preflight()Opengrep comes along as a one-step adapter with no preflight; its behaviour is unchanged.
Testing
tests/codeql_adapter.rs— 10 cases, driven throughScanToolagainst a stub CLI rather than against the adapter directly, because the wiring is where a refusal would leak. Every refusal case asserts what did not happen: the stub logs each invocation, and the log must be empty — or, for a version mismatch, hold exactly one entry with nothing after it.Covers both named scenarios plus two the stub made cheap: a CLI whose
versionoutput is unreadable is a mismatch rather than a pass, and an unpinned bundle produces no preflight and no command at all.--features sec)cargo clippy --workspace --all-targets --features sec,astgrep-rust,astgrep-python -- -D warningscleancargo fmt --all -- --checkclean--features scannersstandalone both green (SC-009 unmoved)What this does not prove
A stub cannot show that CodeQL, handed these arguments, produces useful findings. Proven here is bee's half — the argv it authors, the order it runs, every path on which it refuses, and the report reaching the ledger. The real walkthrough against a provisioned bundle is filed as T073, and it is also where
rustgets settled: if the bundle's Rust extractor ships notools/tracing-config.lua, it joins the list and bee can analyse itself.🤖 Generated with Claude Code