docs(FAFF-269): infosec threat-prior design-settle spike — ADR-0046 + refutation log + eval fixtures#282
Merged
Merged
Conversation
…g + eval fixtures Design-settle spike. SQ1: producer is a human-curated committed threat-model doc (autonomous mining refuted — semantic content, and a persisted .faff/ prior is build-lane-forgeable). SQ2: version-control storage + FAFF-267 --context consumption, no aggregation change. SQ3: value is empirical, GO/NO-GO gated on a measured primed-vs-generic run against committed fixtures 007-010 (the human-supervised step; frontier eval can't nest in-session). Re-casts the ticket's misspecified code-blindness question to the forge/integrity concern. No shippable miner/prompt/schema/grounding artifact.
…or fixtures Adding refutation-spec-007..010 (the FAFF-269 primed-vs-generic threat-prior corpus) requires the lockstep golden-count update in the 'all eval/cases load and validate' test. No behaviour change; the fixtures reuse the existing refutation-spec KIND + seam-registry row.
7806441 to
bdb177d
Compare
#283 (FAFF-346) landed its own ADR-0046 while this branch sat — the concurrent-graft allocation collision (faff adr new allocates at branch-base time). Renumbered to the next free number; faff adr validate passes (contiguous 0001..0047).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design-settle spike for FAFF-269 (design register E; the FAFF-9 §A stopped branch). Deliverable is a refutation log + ADR + reproducible eval fixtures — no shippable code (per the spike anti-pattern rules).
What this settles
faff profile mine), and a persisted LLM-mined.faff/prior is build-lane-forgeable (ADR-0039) → a new attack surface on the lens meant to catch attacks. Autonomy relocates to consumption (the lens reads the doc every review, no human in the loop).--contextfile via the FAFF-267-reserved seam — no aggregation or verdict-contract change. Forge concern dodged by construction.refutation-spec-007..010(the human-supervised step — the frontier eval driver can't nest in a session). GO-narrow on a real lift; NULL closes design-E's infosec sub-question if ≈0.Re-casts the ticket's misspecified code-blindness open question to the real forge/integrity concern (the spec reviewer is not the code-blind lane; the evaluator is).
Deliverables
docs/adr/0046-…md— the decision (Nygard;faff adr validatepasses).docs/spikes/2026-07-05-FAFF-269-…-refutation-log.md— SQ1/SQ2/SQ3, ≥2 refutation attempts each, candidates A/B/C refuted.eval/cases/refutation-spec-007..010.json— 3 faff-specific threat catches + 1 clean near-miss; reuse the existingrefutation-specKIND + seam-registry row (no new KIND).Acceptance Criteria (the spike's DONE)
docs/adr/0046-…md,faff adr validateexit 0eval/cases/as reproduciblerefutation-speccases — 007..010Verified: comments posted (f13e2587 / 029e6c6e)
Verified:
git diff main...HEAD --stat→ onlydocs/+eval/cases/(7 files, +418)faff adr validatepassesVerified: exit 0, "46 ADR(s) valid"
Review (senior-engineer pass): pass, zero findings — fixtures verified as genuine generic-misses + a real near-miss, ADR coherent, measurement deferral a legitimate narrow, revert-clean.