test(detection): add benign corpus coverage

README.md

@@ -643,7 +643,7 @@ It replays a corpus of recorded sessions through the real rule engine and prints
 what fired:
 
 ```
-  cases            54 (26 attack, 28 benign)
+  cases            57 (26 attack, 31 benign)
   rules covered    13
   expected firings 26
   detected         26

ROADMAP.md

@@ -37,7 +37,7 @@ Two rules, so it does not happen again.
 | Version | 0.7.0 on PyPI, released 2026-08-27 |
 | Canonical event schema | 1.2.0, additive |
 | Detection rules | 14 published sequence rules plus 1 experimental, ATT&CK on every event, ATLAS on the AI-specific subset. All 14 exported as Sigma, generated from the engine |
-| Benchmark | 54 recorded sessions, 26 attack and 28 benign, 0 missed and 0 false positives. The four known FPs are now *in* the corpus |
+| Benchmark | 57 recorded sessions, 26 attack and 31 benign, 0 missed and 0 false positives. The four known FPs are now *in* the corpus |
 | Tests | 1148 passing, 81% coverage, floor enforced in CI |
 | Dogfood gate | **0 of 4.** Clock restarted 2026-08-30, after 0.7.0, deliberately, to fix four shipping false positives |
 | Own trail | 27,504 hash-chained lines, 24 external anchors |

SECURITY.md

@@ -73,7 +73,7 @@ and on master; none of it is a periodic manual pass.
 | Data flow a linter cannot follow | CodeQL `security-extended`, Python and TypeScript | `codeql.yml` |
 | Vulnerable dependencies | `pip-audit --strict` on the full resolved tree | `ci.yml` |
 | Dependency currency | Dependabot, weekly, including GitHub Actions | `dependabot.yml` |
-| Detection accuracy | 50-case benchmark, fails on any miss or false positive | `ci.yml`, `release.yml` |
+| Detection accuracy | 57-case benchmark, fails on any miss or false positive | `ci.yml`, `release.yml` |
 | Packaging | wheel installed into a clean environment; `doctor` must report no FAIL | `release.yml` |
 | Publication | PyPI Trusted Publishing (OIDC). No API token exists to leak | `release.yml` |
 

apps/orchestrator/agentmetry/core/audit/detection/corpus/benign_container_registry_publish.jsonl

@@ -0,0 +1,3 @@
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-registry-publish","event_id":"7a7001f1-6c5b-51ab-92e7-8a72c8f6c001","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-registry-publish","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T10:00:00+00:00","tool":{"arguments":{"command":"docker login registry.example.com --username ci-bot --password-stdin"},"command":"docker login registry.example.com --username ci-bot --password-stdin","input_hash":"1f3b6a8a28dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d01","input_redaction":"hash+command","mitre":{"tactic":"Command and Control","tactic_id":"TA0011","technique":"Web Protocols","technique_id":"T1071.001"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude","traits":["net_egress"]}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-registry-publish","event_id":"7a7001f1-6c5b-51ab-92e7-8a72c8f6c002","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-registry-publish","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T10:00:30+00:00","tool":{"arguments":{"command":"docker build -t registry.example.com/team/service:build-42 ."},"command":"docker build -t registry.example.com/team/service:build-42 .","input_hash":"2a4c6d8e28dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d02","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-registry-publish","event_id":"7a7001f1-6c5b-51ab-92e7-8a72c8f6c003","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-registry-publish","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T10:01:20+00:00","tool":{"arguments":{"command":"docker push registry.example.com/team/service:build-42"},"command":"docker push registry.example.com/team/service:build-42","input_hash":"3b5d7e9f28dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d03","input_redaction":"hash+command","mitre":{"tactic":"Command and Control","tactic_id":"TA0011","technique":"Web Protocols","technique_id":"T1071.001"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude","traits":["net_egress"]}}

apps/orchestrator/agentmetry/core/audit/detection/corpus/benign_dependency_audit_network_burst.jsonl

@@ -0,0 +1,4 @@
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-dependency-audit","event_id":"9c7001f1-6c5b-51ab-92e7-8a72c8f6e001","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-dependency-audit","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T12:00:00+00:00","tool":{"arguments":{"command":"npm audit --json > reports/npm-audit.json"},"command":"npm audit --json > reports/npm-audit.json","input_hash":"8091234428dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d08","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-dependency-audit","event_id":"9c7001f1-6c5b-51ab-92e7-8a72c8f6e002","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-dependency-audit","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T12:00:20+00:00","tool":{"arguments":{"command":"pip-audit -r requirements.txt -f json -o reports/pip-audit.json"},"command":"pip-audit -r requirements.txt -f json -o reports/pip-audit.json","input_hash":"91a2345528dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d09","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-dependency-audit","event_id":"9c7001f1-6c5b-51ab-92e7-8a72c8f6e003","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-dependency-audit","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T12:00:40+00:00","tool":{"arguments":{"command":"curl -s https://registry.npmjs.org/-/npm/v1/security/advisories?limit=20 > reports/npm-advisories.json"},"command":"curl -s https://registry.npmjs.org/-/npm/v1/security/advisories?limit=20 > reports/npm-advisories.json","input_hash":"a2b3456628dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d10","input_redaction":"hash+command","mitre":{"tactic":"Command and Control","tactic_id":"TA0011","technique":"Web Protocols","technique_id":"T1071.001"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude","traits":["net_egress"]}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-dependency-audit","event_id":"9c7001f1-6c5b-51ab-92e7-8a72c8f6e004","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-dependency-audit","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T12:01:20+00:00","tool":{"arguments":{"command":"python scripts/summarise_audit.py reports/npm-audit.json reports/pip-audit.json reports/npm-advisories.json"},"command":"python scripts/summarise_audit.py reports/npm-audit.json reports/pip-audit.json reports/npm-advisories.json","input_hash":"b3c4567728dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d11","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}

apps/orchestrator/agentmetry/core/audit/detection/corpus/benign_terraform_apply_with_profile.jsonl

@@ -0,0 +1,4 @@
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-terraform-apply","event_id":"8b7001f1-6c5b-51ab-92e7-8a72c8f6d001","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-terraform-apply","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T11:00:00+00:00","tool":{"arguments":{"command":"terraform init -backend-config=env/staging.backend"},"command":"terraform init -backend-config=env/staging.backend","input_hash":"4c6e8f1028dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d04","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-terraform-apply","event_id":"8b7001f1-6c5b-51ab-92e7-8a72c8f6d002","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-terraform-apply","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T11:02:00+00:00","tool":{"arguments":{"command":"terraform plan -var-file env/staging.tfvars -out tfplan"},"command":"terraform plan -var-file env/staging.tfvars -out tfplan","input_hash":"5d7f901128dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d05","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-terraform-apply","event_id":"8b7001f1-6c5b-51ab-92e7-8a72c8f6d003","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-terraform-apply","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T11:04:00+00:00","tool":{"arguments":{"command":"AWS_PROFILE=staging terraform apply -auto-approve tfplan"},"command":"AWS_PROFILE=staging terraform apply -auto-approve tfplan","input_hash":"6e80912228dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d06","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude"}}
+{"action":{"outcome":"success","reason":"decision:allow;hook:PreToolUse","type":"tool_called"},"actor":{"id":"operator","role":"operator","type":"agent"},"agent":{"name":"claude","skill_id":""},"correlation_id":"b-terraform-apply","event_id":"8b7001f1-6c5b-51ab-92e7-8a72c8f6d004","fleet_id":"example-fleet","host_id":"BUILD-01","initiator":{"actor_type":"agent","operator_id":"operator","trigger":"manual"},"model":{"id":"claude","provider":"claude"},"schema_version":"1.1.0","session_id":"b-terraform-apply","source":{"adapter":"claude_hook","app":"claude","tier":"external"},"source_topic":"external/claude/tool_called","timestamp_utc":"2026-08-12T11:06:00+00:00","tool":{"arguments":{"command":"aws sts get-caller-identity --profile staging"},"command":"aws sts get-caller-identity --profile staging","input_hash":"7f90123328dd96c5d2e9b8a6a722f78c87b80ac3e4d2cc9d7f5d3a44a9148d07","input_redaction":"hash+command","mitre":{"tactic":"Execution","tactic_id":"TA0002","technique":"Unix Shell","technique_id":"T1059.004"},"name":"Bash","parameters_redacted":false,"qualified":"Bash","server":"claude","traits":["cloud_api"]}}

apps/orchestrator/agentmetry/core/audit/detection/corpus/corpus.yaml

@@ -419,6 +419,34 @@ cases:
       Thirty read-only calls spread over half an hour. Burst rules measure a
       time window; a long session must not trip them by length alone.
 
+  - name: container registry publish
+    session: benign_container_registry_publish.jsonl
+    expect: []
+    benign: true
+    note: >
+      `docker login`, `docker build` and `docker push` against a registry. This
+      looks like authenticated egress, but no credential is read from disk or
+      the environment; the password arrives through stdin and never belongs in
+      the trail.
+
+  - name: terraform apply with a named profile
+    session: benign_terraform_apply_with_profile.jsonl
+    expect: []
+    benign: true
+    note: >
+      Infrastructure automation touches cloud state and later calls `aws sts`,
+      but it uses a named profile rather than reading credentials. Cloud API use
+      by itself is not lateral movement after credential harvest.
+
+  - name: dependency audit touches remote advisories
+    session: benign_dependency_audit_network_burst.jsonl
+    expect: []
+    benign: true
+    note: >
+      Package audits query registries and advisory feeds in quick succession,
+      then run a local summariser. A short network-heavy audit is not a session
+      tool burst, a download cradle or credential exfiltration.
+
   - name: human-driven deletes currently fire
     session: benign_human_driven_deletes.jsonl
     expect: [destructive-delete-burst]

docs/whitepaper.md

@@ -263,8 +263,8 @@ the firings to the wrong queue.
 This is a security tool, so the interesting question is not what it detects but
 why you should believe the numbers.
 
-**A benchmark anyone can run in about ten seconds.** 54 recorded sessions, 26
-attack and 28 benign. Current result: 26 of 26 expected detections fire, 0
+**A benchmark anyone can run in about ten seconds.** 57 recorded sessions, 26
+attack and 31 benign. Current result: 26 of 26 expected detections fire, 0
 missed, 0 false positives. **13 of the 14 published rules have corpus
 coverage**, and the one that does not is named in the generator with a reason
 rather than rounded up.