docs(security): posture write-up + presentation script + (optional) blog outline - #107
Merged
Conversation
Documents the security work shipped in #97 (ML-DSA verify gate + /plan/verify) and #100 (model supply-chain manifest + AST scanner) without overselling it. Honest gap list called out separately. - docs/security/posture.md: threat model, defenses-shipped table with one row per real defense, out-of-scope list, verification recipe, and a §5 honest-gaps section that flags the /plan/verify trust-list asymmetry vs verify_cot, the unsigned-manifest gap, the AST scanner bug on aliased imports, and the dev-only Ed25519 fallback. - docs/presentation/security-beat.md: 90s presenter script for the PRD §12 3:00-3:30 slot. Two cuts (signed-route tamper-reject; model integrity CI fail), three sentences each, one framing line. P2 suggested as presenter per PRD lane mapping. - docs/security/blog-outline.md: 5-section industry-blog outline with explicit "what this is NOT" disclaimer; venues ranked house-blog first, no peer-reviewed crypto venues. Conditional on items A and E scoring NOVEL — INDUSTRY-BLOG-PUBLISHABLE in the assessment. - docs/demo-recording-plan.md: adds S6 model-integrity clip slot, inserted right after S1 (the existing PQC tamper-reject clip), to match the security-beat ordering. - docs/issue-scrub-2026-05-03.md: post-scrub deltas appendix noting #57 closed by #100, #81 closed by #97 (with trust-list caveat cross-linking posture.md §5.1), #79 closed by #102. Refs #57, #81, #97, #100. Scope: docs only; no code changes.
jdev-02
requested review from
aleens-labs,
benschwierking,
khicks1724 and
kylemhicks
as code owners
May 3, 2026 08:45
PR #105 (merged after the original posture pass) wires the trust list into the /plan/verify HTTP path and adds a FastAPI lifespan that auto-bootstraps the device's own key. This commit refreshes the docs so they describe what shipped, not what was on main pre-#105: - posture.md §2: replace "Trust list (CoT path)" row with a "Trust list — both render paths (post-#105)" row that cites the new wiring (agent/orchestrator.py:420-459) and the lifespan (agent/app.py:42-60) and the three new tests. - posture.md §5.1: remove the "verify_plan_response does not consult a trust list" finding (no longer true). Replace with an honest framing of the (intentional) self-attestation trust model and what it does and does not defend against. Renumber the remaining gap subsections 5.2-5.6 (manifest unsigned, AST scanner, prompt injection, operator_approved stub, Ed25519 fallback all unchanged). - posture.md §3: reframe CRL/rotation as "intentionally out of scope for hackathon" rather than "broken". - posture.md §4: bump verification recipe from 6 to 7 commands; pytest invocation now includes the three post-#105 tests; add a spoofed-key_id construct as the third recipe entry. Re-validated locally: 6 verify-gate tests pass in 0.76s. - presentation/security-beat.md: keep Cut 1 (tamper-reject) and Cut 2 (model integrity); add an optional Cut 1b (~10s) showing the spoofed-key_id reject — punchier than byte-tampering for non-crypto judges. Update line citations to post-#105 numbers. - blog-outline.md §2: walk the verify gate end-to-end through the trust-list lookup and bootstrap; cover the bootstrap as a deployment-tooling story, not a crypto novelty. Updated cite list. - blog-outline.md §5: trust-list bullet now reads "extend the wired flat-file trust list with enrollment + CRL + rotation" instead of "implement key_id lookup at /plan/verify". - issue-scrub-2026-05-03.md: add #105 to the post-scrub deltas. Item B in the novelty assessment moves from OVERSOLD to COMPETENT — STANDARD PRACTICE: a flat-file trust list with auto-bootstrap is correct industrial practice for a single-device demo, but it is not novel research and the doc does not promote it. All other findings (unsigned manifest, placeholder hashes, AST scanner unreachable branch, hard-coded operator_approved=True, substring blocklist weakness) are unchanged — they all still hold post-#105. Refs #57, #81, #97, #100, #105.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documentation pass on the security work shipped in #97 (ML-DSA-65 verify gate
/plan/verifyendpoint) and feat(security): model supply-chain integrity — issue #57 #100 (model supply-chain manifest + ASTscanner). Companion to the broader
security/+crypto/stack already onmain. Jon asked for the wins logged into demo / presentation / docs and —where genuinely novel — drafted up as a blog outline. This PR is the
documentation deliverable; no code changes.
The novelty pass scored everything honestly; nothing scored
NOVEL — RESEARCH-PUBLISHABLE, so there is no paper outline. Two items
(ML-DSA verify gate before ATAK render and the combined edge-LLM threat
model) scored NOVEL — INDUSTRY-BLOG-PUBLISHABLE, so a blog outline is
included. Standard-practice items (SHA-256 manifest pinning, AST
torch.loadscanner, trust list) are framed as competent engineering, not research.
The §5 "Honest gaps" section in
posture.mdis intentionally longer than thewins list. It surfaces, among other things, that
verify_plan_responseonmaindoes not consult a trust list — the trust-list code lives incrypto.cot_signer.verify_cot(CoT XML path) but is not wired into the/plan/verifyHTTP endpoint. That asymmetry is documented so judges andreviewers don't think we're claiming more than we built.
Files
docs/security/posture.md— public-facing security posture: threat model(cited from PRD §6/§8/§9), defenses-shipped table with file:line citations
for each row, out-of-scope list, 6-command verification recipe, honest gaps.
docs/presentation/security-beat.md— 90s presenter script for PRD §123:00–3:30 slot. Two cuts (signed-route tamper-reject; model integrity CI
fail), three sentences each, one framing line. P2 (Satriyo) suggested as
presenter per PRD lane mapping.
docs/security/blog-outline.md— 5-section industry-blog outline withexplicit "what this is NOT" disclaimer. Venues ranked house-blog first; no
peer-reviewed crypto venues.
docs/demo-recording-plan.md— adds the S6 model-integrity clip slot,inserted right after S1 (the existing PQC tamper-reject clip).
docs/issue-scrub-2026-05-03.md— post-scrub deltas appendix noting [security] Model supply-chain integrity -- checksum + signed bundles for every external model #57closed by feat(security): model supply-chain integrity — issue #57 #100, [atak] Verify ML-DSA signature on /plan response before rendering #81 closed by feat(crypto): verify plan response before ATAK render #97 (with the trust-list caveat
cross-linked to
posture.md§5.1), [atak] Wire format alignment: TeraPlanClient → /plan PlanRequest schema #79 closed by fix(atak): wire TeraPlanClient to /plan with PlanRequest schema — clo… #102.Refs #57, #81, #97, #100.
Test plan
make model-integrityruns green on a fresh checkout (verifies theS6 demo recording happy path).
pytest tests/test_orchestrator.py::test_verify_plan_response_accepts_signed_response tests/test_orchestrator.py::test_verify_plan_response_rejects_tampered_route tests/test_orchestrator.py::test_verify_plan_response_rejects_missing_signature -vpasses (verifies the citations in
posture.mdtable row 1 are real).make cistill passes (we did not touch any code; this is adocs-only change, so CI should be green by construction).
posture.md§ honest gaps reads as more credible thanthe wins. If it doesn't, the doc has failed its purpose.
Revision (post-#105): PR #105 merged into
mainafter this PR was opened, wiringcrypto.cot_signer.load_trust_listintoagent.orchestrator.verify_plan_responseand adding a FastAPI lifespan that auto-bootstraps the device's own public key intocrypto/keys/trust_list.json. The docs in this PR have been refreshed (commit79d6d4d) to describe the post-#105 wiring, including the auto-bootstrap as an intentional self-attestation trust model rather than a gap. Item B in the novelty assessment moves from OVERSOLD to COMPETENT — STANDARD PRACTICE; all other honest-gap findings (unsigned manifest, placeholder hashes, AST scanner unreachable branch, hard-codedoperator_approved=True, substring blocklist weakness) are unchanged. Verification recipe re-run locally: 6 verify-gate tests pass in 0.76s.