016: harness-native security tooling - #13
Merged
Merged
Conversation
Opengrep reports a rule's severity on the rule, not the result, so `Finding.advisory_level` could never be populated by the one shipped adapter — a field in the data model that nothing could ever fill. Reaching it did not require materialising `tool.driver.rules`, which research R4 measured at 99.96% of the document. `retain_rule_levels` is a streaming seq visitor that collapses the array to `id → level` as it passes: two short strings per rule, capped at MAX_RULE_LEVELS (4096), with the descriptions, help text, and tags parsed and dropped rather than allocated. The visitor drains the sequence past the cap instead of stopping. An early return would abort the deserializer mid-document and turn a large catalogue into "unparseable report" — a scan that ran fine reading as a failure, which is the one outcome this whole feature exists to prevent. Overflow costs advisory levels, never a finding and never the scan. A result's own `level` still wins over the rule default (SARIF §3.27.10). FR-005 is unmoved: the level is carried and rendered as advisory, and `severity` stays None. Closes T070. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a security-tooling surface to the harness: structural search, a finding ledger that outlives the episode, an external scanner tier, and computed severity — all on a fail-closed tool seam where no error path can render as a clean scan.
What lands
The seam (Phase 2).
ToolOutcome<T>—Completed/Unavailable { reason }/Failed { reason }— with a rendering in which no spelling of "did not run" can be mistaken for "found nothing". EveryUnavailable/Failedemits a structured audit event first. A tool whose feature is compiled out stays known and returnsNotCompiledInat the call, so a scenario referencing it fails loudly rather than silently at validation.US1 — structural search.
ast_greptool + worker overast-grep-core, oneastgrep-<lang>feature per grammar. Read-only: no replace path is exposed.$X.unwrap()over the decoy fixture returns the one live-code match where regex returns all three.US2 — the finding ledger. Append-only
.bee/findings/ledger.jsonl, one line per event, capped at 4000 bytes so a payload plus its newline stays insidePIPE_BUF— the atomicity the lock-free design rests on, with a concurrent-writer test that substantiates it. Folding toview.jsonis pure and total. Identity excludes the line number, so a finding that moved merges rather than duplicating.bee findings adjudicateis the only writer of a verdict and is deliberately CLI-only: adjudication is the human's act, never a model-facing tool.US3 — external scanners. SARIF subset parser,
sarif-worker,ScannerAdapter+ an Opengrep adapter. Grants are!-pinned exec entries only; a binary merely present onPATHyieldsNotGranted, and the inode pin is re-checked immediately before spawn so a substituted binary is never executed. Success is read from the report (executionSuccessful), never the exit code.--config autois refused at argv construction.US4 — computed severity.
cvsstool over v3.1/v4.0 vectors. A caller-supplied score is refused rather than silently recomputed (FR-005), including nested under another name —deny_unknown_fieldsmeans there is no spelling of "here is my score" bee accepts quietly.Why the pipeline has the shape it does
An Opengrep SARIF over a three-line file measures 1,912,546 bytes, of which 99.96% is the embedded rule catalogue, to deliver 839 bytes of results — 19x the output cap. Capturing it on stdout would truncate it into unparseable JSON and surface as "the scanner found nothing", the exact failure this feature exists to prevent. So the scanner writes a file and a second scope-joined child normalises it in-scope, emitting JSONL so a transport cut costs the last record rather than the whole parse.
The final commit closes the last gap in that reasoning: the catalogue is where Opengrep keeps a rule's severity, so it is now streamed and collapsed to
id → level— bounded, with the 99.96% still never materialised.Cost to the default build
None.
SC-009is verified: the default dependency graph and binary size are unchanged frommain, every feature builds standalone, andtests/core_deps_guard.rsrefuses to let any of the new crates reachbee-core/bee-common. The application package moves to MSRV 1.88; the embeddable core crates keep their 1.85 floor.Not in this slice
git_log) and US6 (CodeQL) — deferred by design; phases are marked DEFERRED intasks.md. CodeQL's traced languages would require admitting every compiler and linker a build invokes, which surrenders SC-006.scanner-escape-deniedVM matrix case. The matrix is 35/35 today; this adds a 36th.repl_command::no_provider_at_all_reports_what_is_missingfails on non-enforcebuilds. Verified pre-existing onmainat default features, not a regression from this branch, and the only failure in the suite.Verification
cargo fmt --all -- --checkandcargo clippy --workspace --all-targets --features sec,astgrep-rust,astgrep-python -- -D warningsare clean. Full suite green but for T072 above.quickstart.mdwas walked end to end on a fresh clone with no security tooling installed; the four documentation defects that walk found are fixed in this branch.🤖 Generated with Claude Code