FE-1431: Build the plugin authoring surface - #9337
Open
lunelson wants to merge 23 commits into
Open
Conversation
This was referenced Aug 26, 2026
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
This was referenced Aug 26, 2026
3 tasks
Contributor
Author
lunelson
changed the base branch from
ln/fe-1406-harness-teaching-adr
to
graphite-base/9337
August 26, 2026 08:49
lunelson
force-pushed
the
ln/fe-1431-plugin-authoring-cycle
branch
from
August 26, 2026 10:42
57631f3 to
b3d5343
Compare
lunelson
force-pushed
the
graphite-base/9337
branch
from
August 26, 2026 10:42
000d346 to
44fb37e
Compare
lunelson
changed the base branch from
graphite-base/9337
to
ln/fe-1406-harness-teaching-adr
August 26, 2026 10:42
lunelson
changed the base branch from
ln/fe-1406-harness-teaching-adr
to
graphite-base/9337
August 26, 2026 13:44
4 tasks
Cycle one of ADR-0007's co-authoring convergence (S-009): the harness-owned key catalogue exists as code, both test-case plugins are written against the same schema, and the harness's own teaching ships as a package that the binding renders interleaved with each plugin's cells. Harness (`packages/core`): - `keys.ts` is the catalogue — jobs, the eight guidance keys, the two movements, the three runbook keys — with the one-paragraph definition the interviewer reads above every rendered key. - `plugin-definition.ts` reads `plugin.yaml` under a strict valibot schema (an unknown key anywhere fails to load) and cross-checks what a schema cannot say: rows name declared kinds, every kind has a row, the anchor is a counted `must_know` row, runbooks belong to declared jobs. The anchor is declared (`schema.anchor`), replacing `objective`-by-convention; precision words become harness vocabulary (`PRECISION_LADDER`). - `repertoire.ts` types the repertoire and requires every key filled and every entry sourced. - `instructions.ts` renders harness preamble → contract → each guidance key (definition, default, cell) → each declared job's runbook. The completion protocol sentences move from `cue.ts` into `HARNESS_PREAMBLE`. - `schema/plugin.schema.json` is derived from the valibot schema (`yarn schema:emit`) and a structural drift test keeps it honest; `schema/CHANGELOG.md` opens the cycle log with cycle one's shape and its open questions. - `plugin-file.ts` (the Markdown parser) and its test are removed; completion, fold, slot-assertion, cue, and `Plugin.definition` consume the definition. Fixture is now YAML. Repertoire (`packages/repertoire`, `@hashintel/brunch-agent-repertoire`): every guidance and runbook cell filled from the v0 prompt, the failure catalogue (FM-02/03/04/06/07/08/12/13/14/15), the condition-3 hints, GEN-Q02, the literature cards, and ADR-0007 decision 4; each entry names its source. Depends on core only; the boundaries gate now checks that only bindings depend on it, and the role regex and kernel §12.2 admit it. Plugins: `plugin-sdcpn/plugin.yaml` (objective anchor, ten kinds, 24 rows, eight patterns, formalism-specific cells only) replaces `plugin.md`; `plugin-gherkin/plugin.yaml` (feature anchor, four kinds, feature → rule → example → step) is written in the same cycle so the schema is pulled by two formalisms at once. Gherkin keeps its `statement-noted` proposal for now; the binding folds and cues only for definitions that declare `slot-asserted`. Binding: `renderInstructions(repertoire, definition)` replaces the plugin-file instructions and the separate completion fragments. Docs: `plugin-contract.md` amended to the key contract; INDEX rows, kernel §12.2, README, CONTEXT, and every `plugin.md` link repointed; STEERING's authoring lane names the simulated-interview re-run as each cycle's run stage. Gates: `test:unit`, `lint:tsc`, `lint:eslint`, `build` green across core, repertoire, binding-flue, plugin-gherkin, plugin-sdcpn, and apps/brunch-agent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first "run" stage of an ADR-0007 co-authoring cycle (S-009; STEERING's authoring lane): re-do the simulated interview with the interviewer taught by the rendered repertoire and SDCPN plugin definition, and nothing else. `run.ts 4` assembles the system prompt from `condition-4-prompt.md` — a framing that stands in for the harness preamble, telling a harness-less interviewer to keep the Must-know tally and the assumption ledger itself — plus `renderContract`, `renderGuidance`, and `renderRunbook(construct)` from the harness's built output over the shipped `repertoire.yaml` and `plugin-sdcpn/plugin.yaml`. The assembled prompt is written beside the transcript as `condition-4-system.md` so the run is reproducible against the exact text it saw. Mechanics are conditions 1–2's (legacy impatience probe, turn budget, delivery classifier), so the 2→4 delta isolates what the fixed keys and the repertoire buy over the seven-category prompt; a later harness-in-the-loop run isolates what the machinery buys over this text. Verified hermetically through the stub client (delivers on turn 1; the system prompt renders every key in catalogue order, ~5.7k words). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first co-authoring cycle's validate step: a read-only desk review of the ADR-0007 key catalogue against 100 situations from the CPS process-modelling material, the literature, and the condition-2 run, with a discrete-event and a formal-verification plugin sketched against the keys. Placed as evidence under docs/evidence/proofs/design with a provenance header and indexed. Its verdict is folded into the schema changelog as the cycle-two input: no key changes; two shape changes proposed (an optional slot predicate on pattern rows; any-of precision on must-know rows), one applicability facet for repertoire entries, the content findings (motifs restating patterns, the posture nothing consumes, repertoire under-fill against the ADR's own rows, seven source contradictions resolved silently), and eight changes considered and left. One defect it found is fixed here: a pattern indexed on no kind (`on: []`, documented as "any node") never fired, because the sweep list tested `kinds.includes(node.kind)` on an empty list. sdcpn P08 (source-regime divergence) therefore never reached the interviewer as a harness fact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One run of the rendered ADR-0007 layer (condition-4 framing + repertoire + sdcpn definition, prompt only, no machinery) against the simulated master scheduler: 22 interviewer turns, delivered-after-forced-wrap. Transcripts, raw record, assembled system prompt, and delivered model added; the read-out appended to readout.md under its own heading with headline findings, the seven-category and pack-tier tables extended with a condition-4 column, the silent-assumption audit, stopping behaviour, the 2->4 delta, and a "Strains for cycle two" list keyed to plugin.yaml and repertoire.yaml entries. Headline: the best-disciplined questioning of any condition (objectives first, one to three questions a turn, quantiles on every changeover type, per-slot precision and an attributed assumption ledger in the deliverable) and the thinnest model (no run time, no arrival pattern, no QA spread; one tacit fact of nine fully surfaced). Stopping failed in a third form: the interviewer delivered on the expert's stop at turn 10 and then had no concept of ending. Must-know rows were graded rather than asked; the anchor's dependency slot was authored by the interviewer and drifted across three deliverables. Instrument: the haiku delivery classifier answered NO on the turn-10 deliverable (replayed 3/3) because it names its own gaps, so the stop reason overstates the failure; the raw record keeps classifier usage but not verdicts. Both noted for the rerun. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The baseline protocol's fourth condition measured the ADR-0007 teaching layer as prompt text; nothing yet measured the machinery around that text. Condition 5 drives the shipped SDCPN elicitor in the Flue runtime — the binding's ask suspension, settlement nudge, private sweep extraction into the capture store, fold, and computed completion — against the same simulated expert, probes, and turn budget as conditions 1, 2, and 4. The runner reads every harness fact from durable history and the store, takes interviewer tokens from Flue's observe() turn events, and needs no delivery classifier: the deliverable is the folded store, and the interviewer ends its own turn-taking by replying without a question. The runner lives with the protocol but is launched from the application package (`baseline:harness`), which owns the agent composition; the elicitor's model id gains a `BRUNCH_SDCPN_MODEL` override so the runner can use the same interviewer model as the other conditions. A hermetic test swaps both models for stand-ins — the existing Anthropic stub for the expert, a faux pi provider for the interviewer — and asserts the loop end to end: ask bound, sweep applied with completion, stall detected, artifacts written. Condition 3 is retired before its first model call (superseded by the harness it would have projected by hand; lock left sealed, amendment atop the preregistration) and conditions 1 and 2 are frozen as reference evidence, per the checkpoint review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The shipped SDCPN elicitor (claude-opus-5) in the Flue runtime against the simulated expert, single-shot: twelve interviewer turns, 267 captures, 69 nodes, completion not reached (46 unsatisfied; 53 nodes outside every objective's slice). The runner's `stalled` rule fired on what the transcript shows to be a deliberate close: at turn 10, immediately after the impatience probe, the interviewer read the record back in the expert's words and declined to call it finished; the two turns after it are notes-to-self. Evidence only — the read-out and the cycle-two edits wait for review. A first attempt that reached turn 10 died in a network outage and is kept outside the tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…essment Arc-close for the first harness-in-the-loop run. Records S-010 (retire condition 3, freeze 1–2, conditions 4 and 5 as the live arms; harness facts replace the classifier), the Proof 1 first-run result and narrowed claim in STEERING, four belief rows, the candidate successor streams, and the per-turn latency as an immediate concern with its own evidence document: 29 min for 12 turns, ~97% of output tokens extraction on the critical path, actions R0–R5 and a deferred spike. Ledger rows §8.1, §8.2, §9.1, §9.5, §11.1 reflect what the run exercised. Notes the FE-1404 numbering gap (its "condition 3" is the protocol's condition 5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t stub Salvage assessed as none: the instrument's projection schema and semantic validators encoded the domain-keyed DemandTable that S-007 ruled the wrong level; the kind-level fold and evaluateCompletion in core now do that job on the production path. Removes condition-3-instrument.ts (1,317 lines), its lock, the operator, scoring, legibility, and pre-run-review documents, its unit test, and the condition-3 paths in run.ts (1,720 → 623 lines; conditions 1, 2, and 4 keep their reviewed resume/continue semantics). The preregistration and prompt stay as the record, with a dated amendment. Also: the condition-5 test passed replies inline while the shared expert stub has always read them from BASELINE_STUB_REPLIES_PATH, so the committed test could not pass; it now writes the replies file. S-010's Supersedes line is bare "none" so the control-surfaces gate accepts it. Protocol, STEERING, and S-010 record the deletion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Latency targets adopted provisionally; the isolating spike as first task of the next arc marked provisional pending Lu's further next-steps input; the condition-5 read-out deferred until after the spike; and the black-triangle directive recorded as the frame the streams are worked inside: the full end-to-end flow through the Petrinaut assistant interface first, so the surface stabilises for the team and the voice-mode work while the implementation behind it keeps changing. S-011 is appended when Lu's input lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reduce the local steering protocol to its Brunch-specific supplement and route strategic refreshes through ds-steer. Align Wayfinder commission, landing, issue-close, and concern vocabulary with the canonical skills while preserving local Linear approval and proof-execution rules. Refresh STEERING as current concern-first control without diary or mirrored tracker history.
Two jobs, one order: construct through the Petrinaut panel is Proof 0, review-and-revise sits on top. G0 (FE-1503) is the selected frontier with its cut, parallel streams, gap assessment, and tool inventory pinned as current truth against intent; G1 and G2 follow as a sequence. Records the plugin-API-in-flux and package-topology concerns, the voice-edge attach surface and provider gate, and defines legibility on the human in the legibility protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The work inside G0 shares one proof and is a set of moves joined at Proof 0, not streams; "stream" is reserved for separately proven parallel work, and G0 -> G1 -> G2 is a sequence. Defines the three terms in CONTEXT.md and corrects STEERING, S-011 (addendum), and the legibility protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ds-steer selects one frontier and leaves other available work unplanned; Brunch adds a partition step: efforts with their own proof, projection, disjoint write set, and named join points, run in separate worktrees with control documents written only from the driver. Records today's partition (W1-W5 and the voice work) in STEERING and defines effort and join point in CONTEXT. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
S-011's link to the H-6763 plan pointed at a file that exists only on the voice branch; three older strategy entries linked STEERING headings that no longer exist; the E1 and E2 epicentre rows described gaps that landed on the stack. Adds the unmerged stack as an active gate, since G0's baseline is not on main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The steering control retired the `Governing strategic decisions:` line in 6903277; the architecture test still matched it and failed on every checkout. Match the `## Governing concerns` section instead, keeping the invariant that steering cites at least one strategy entry and every cited entry resolves and is unsuperseded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The app's condition-5 test executes the context-owned runner, so package-isolated CI must copy that non-workspace path. Co-authored-by: Cursor <cursoragent@cursor.com>
The contract text and slice reader both assumed objective-shaped demand: a hardcoded "objective-relative" floor line, and a dependency reader that dropped a single kind:node string. Steering also linked ADR-0008 and cycle-one transcripts this branch does not yet carry. Co-authored-by: Cursor <cursoragent@cursor.com>
lunelson
force-pushed
the
ln/fe-1406-harness-teaching-adr
branch
from
August 27, 2026 17:42
a3724ed to
bed9c08
Compare
lunelson
force-pushed
the
ln/fe-1431-plugin-authoring-cycle
branch
from
August 27, 2026 17:42
25e0280 to
3db1b81
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

🌟 What is the purpose of this PR?
Brunch interviews a domain expert and builds a formal model from what they say. ADR-0007 decided that the interviewer's craft — how to open, when to walk one case and when to sweep, how to probe, where not to dig, how to close — belongs to the harness, and that a target formalism contributes only data under a fixed set of harness-owned keys. This pull request makes that decision executable. It materialises the key catalogue as a schema in core, writes the harness's default teaching (the repertoire) against it, and co-authors both test-case plugins (SDCPN process models and Gherkin scenarios) against the same shape, so that a plugin is one validated YAML document rather than prose the harness interprets. It then puts the rendered layer under evaluation: baseline condition 4 tests the teaching as a prompt alone, and condition 5 runs the harness itself inside the baseline loop against the same expert simulator, recording the first turn-latency figures.
The branch also carries the steering pass that closed the condition-5 arc and selected the next frontier (S-011, the first human elicitation from the Petrinaut panel), together with the control-vocabulary and worktree protocols that let that work run as a parallel partition. This pull request was opened as a draft while cycle one was under review; cycle two (the freeze) and ADR-0008 (repertoire ownership) have since been built on branches stacked above it, so its scope is the cycle-one materialisation and the evaluation and control work that followed it.
🔗 Related links
libs/@hashintel/brunch-agent/docs/adr/0007-harness-teaching-meets-plugin-content-at-fixed-keys.mdlibs/@hashintel/brunch-agent/docs/specs/plugin-contract.md; schema changelog:libs/@hashintel/brunch-agent/packages/core/schema/CHANGELOG.mdlibs/@hashintel/brunch-agent/docs/evidence/evaluations/process-model-elicitation/baseline/readout.md(conditions 4 and 5);docs/evidence/proofs/design/plugin-keys-pressure-review-cycle-1.mdlibs/@hashintel/brunch-agent/docs/control/STRATEGY-LOG.mdS-009 to S-011; frontier and partition:docs/control/STEERING.md🚫 Blocked by
Nothing in code. Stacks on #9327 (ADR-0007). The branches above it — the panel run, timing, the cycle-two freeze, and the ADR-0008 move — depend on this one landing.
🔍 What does this change?
The plugin contract as code.
PluginDefinitionSchemainpackages/core/src/plugin-definition.tsdefines the sixteen keys in four groups (contract data, guidance typed by mechanism, per-job runbooks, machinery).packages/core/schema/plugin.schema.jsonis derived from it and a test fails when they drift. Readers reject an unknown key, require the repertoire to fill every guidance and runbook key, and let a plugin leave cells blank but never add a key. The instruction renderer interleaves key → harness default → plugin cell, so cells add and never override.The repertoire and both plugins.
packages/repertoire/repertoire.yamlholds the harness's default filling of every key, admitted from the evidence ADR-0007 names (the catalogued elicitor failures, the v0 prompt's headings, the lifted rows).plugin-sdcpn/plugin.yamlandplugin-gherkin/plugin.yamlare written against the same schema; the SDCPN Markdown plugin file retires, its conversational triggers become techniques and sweep movements, and the precision ladder becomes harness vocabulary. The cycle-one pressure review (100 situations, two further plugins sketched) found every situation landing on an existing key and fixed one matching defect (patterns.items[*].on: []never fired).Evaluation. Condition 4 puts the rendered teaching into the baseline as a prompt only; condition 5 runs the harness inside the baseline loop, which required the Flue binding to accept the rendered instruction and the runner to observe per-turn timing. The retired condition-3 instrument is deleted. The condition-5 readout records the arc's headline: about 145 seconds per interviewer turn, which S-011 makes the first thing the next frontier measures.
Control and protocols. The strategy log gains S-009 to S-011;
STEERING.mdis rewritten to the dogsled shape (objective, frontier, governing concerns, beliefs, cuts, stop triggers) and records the parallel partition of the next arc into efforts.CONTEXT.mddefines move, stream, sequence, effort, and partition.docs/agents/gains the partition-worktrees protocol and the steering supplement's partition step;legibility.md,arc-close.md,issue-writing.md, andgit-workflow.mdare tightened from what the arc taught.🏗️ Agent notes
Commits
63eeb6aff1Materialise the plugin keys, the repertoire, and both plugin definitions259255f5a9/1e58aa1677Add and score baseline condition 4: the teaching layer as prompt onlyf447cc2ba8Record the cycle-1 pressure review and fix any-kind patterns67d4ff4b15/0159d7ac84Put the harness in the baseline loop as condition 5; record the first run7501dcfa26Close the condition-5 arc: steering, ledger, and the turn-latency assessment312ddf095fDelete the retired condition-3 instrument and fix the condition-5 test stubf2e52e298f…a0bf9c8ffcReconcile steering with dogsled; select the black triangle as the frontier (S-011)eaef0fdca9…dcf5b37ebfControl vocabulary (moves, streams, sequences, partition) and the partition-worktrees protocol323cdd09f8,19f7e9247aRepair stale control docs; anchor the control-surface test on the governing-concerns sectionf4ced883eb…9543278418Driver-side steering reconciliation during the partitioned arcVerification
turbo run test:unit lint:eslint lint:tscacross@hashintel/brunch-agent, the repertoire, both plugins,binding-flue,transport-aisdk, and@apps/brunch-agent; the architecture suite (package direction, Flue entrypoint, hermetic runtime, control surfaces) passes. The schema-drift test tiesplugin.schema.jsontoPluginDefinitionSchema. Condition-4 and condition-5 runs are recorded as immutable snapshots underdocs/evidence/evaluations/process-model-elicitation/baseline/runs/.Stack
main → #9322 (FE-1404) → #9325 (FE-1497) → #9327 (FE-1406) → this PR → #9345 (FE-1504) → FE-1506 → FE-1505 → FE-1431 cycle two → FE-1507. Thepackages/repertoireworkspace this PR introduces is removed again by the FE-1507 branch under ADR-0008; it is kept here because the cycle-two freeze evidence was produced against it.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR: