Skip to content

FE-1431: Build the plugin authoring surface - #9337

Open
lunelson wants to merge 23 commits into
ln/fe-1406-harness-teaching-adrfrom
ln/fe-1431-plugin-authoring-cycle
Open

FE-1431: Build the plugin authoring surface#9337
lunelson wants to merge 23 commits into
ln/fe-1406-harness-teaching-adrfrom
ln/fe-1431-plugin-authoring-cycle

Conversation

@lunelson

@lunelson lunelson commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

🌟 What is the purpose of this PR?

Brunch interviews a domain expert and builds a formal model from what they say. ADR-0007 decided that the interviewer's craft — how to open, when to walk one case and when to sweep, how to probe, where not to dig, how to close — belongs to the harness, and that a target formalism contributes only data under a fixed set of harness-owned keys. This pull request makes that decision executable. It materialises the key catalogue as a schema in core, writes the harness's default teaching (the repertoire) against it, and co-authors both test-case plugins (SDCPN process models and Gherkin scenarios) against the same shape, so that a plugin is one validated YAML document rather than prose the harness interprets. It then puts the rendered layer under evaluation: baseline condition 4 tests the teaching as a prompt alone, and condition 5 runs the harness itself inside the baseline loop against the same expert simulator, recording the first turn-latency figures.

The branch also carries the steering pass that closed the condition-5 arc and selected the next frontier (S-011, the first human elicitation from the Petrinaut panel), together with the control-vocabulary and worktree protocols that let that work run as a parallel partition. This pull request was opened as a draft while cycle one was under review; cycle two (the freeze) and ADR-0008 (repertoire ownership) have since been built on branches stacked above it, so its scope is the cycle-one materialisation and the evaluation and control work that followed it.

🔗 Related links

  • FE-1431: Build the plugin authoring surface (internal; owning issue, Done)
  • FE-1406 (internal; ADR-0007, the decision this implements)
  • Decision: libs/@hashintel/brunch-agent/docs/adr/0007-harness-teaching-meets-plugin-content-at-fixed-keys.md
  • Contract: libs/@hashintel/brunch-agent/docs/specs/plugin-contract.md; schema changelog: libs/@hashintel/brunch-agent/packages/core/schema/CHANGELOG.md
  • Evidence: libs/@hashintel/brunch-agent/docs/evidence/evaluations/process-model-elicitation/baseline/readout.md (conditions 4 and 5); docs/evidence/proofs/design/plugin-keys-pressure-review-cycle-1.md
  • Strategy: libs/@hashintel/brunch-agent/docs/control/STRATEGY-LOG.md S-009 to S-011; frontier and partition: docs/control/STEERING.md

🚫 Blocked by

Nothing in code. Stacks on #9327 (ADR-0007). The branches above it — the panel run, timing, the cycle-two freeze, and the ADR-0008 move — depend on this one landing.

🔍 What does this change?

The plugin contract as code. PluginDefinitionSchema in packages/core/src/plugin-definition.ts defines the sixteen keys in four groups (contract data, guidance typed by mechanism, per-job runbooks, machinery). packages/core/schema/plugin.schema.json is derived from it and a test fails when they drift. Readers reject an unknown key, require the repertoire to fill every guidance and runbook key, and let a plugin leave cells blank but never add a key. The instruction renderer interleaves key → harness default → plugin cell, so cells add and never override.

The repertoire and both plugins. packages/repertoire/repertoire.yaml holds the harness's default filling of every key, admitted from the evidence ADR-0007 names (the catalogued elicitor failures, the v0 prompt's headings, the lifted rows). plugin-sdcpn/plugin.yaml and plugin-gherkin/plugin.yaml are written against the same schema; the SDCPN Markdown plugin file retires, its conversational triggers become techniques and sweep movements, and the precision ladder becomes harness vocabulary. The cycle-one pressure review (100 situations, two further plugins sketched) found every situation landing on an existing key and fixed one matching defect (patterns.items[*].on: [] never fired).

Evaluation. Condition 4 puts the rendered teaching into the baseline as a prompt only; condition 5 runs the harness inside the baseline loop, which required the Flue binding to accept the rendered instruction and the runner to observe per-turn timing. The retired condition-3 instrument is deleted. The condition-5 readout records the arc's headline: about 145 seconds per interviewer turn, which S-011 makes the first thing the next frontier measures.

Control and protocols. The strategy log gains S-009 to S-011; STEERING.md is rewritten to the dogsled shape (objective, frontier, governing concerns, beliefs, cuts, stop triggers) and records the parallel partition of the next arc into efforts. CONTEXT.md defines move, stream, sequence, effort, and partition. docs/agents/ gains the partition-worktrees protocol and the steering supplement's partition step; legibility.md, arc-close.md, issue-writing.md, and git-workflow.md are tightened from what the arc taught.

🏗️ Agent notes

Commits

  • 63eeb6aff1 Materialise the plugin keys, the repertoire, and both plugin definitions
  • 259255f5a9 / 1e58aa1677 Add and score baseline condition 4: the teaching layer as prompt only
  • f447cc2ba8 Record the cycle-1 pressure review and fix any-kind patterns
  • 67d4ff4b15 / 0159d7ac84 Put the harness in the baseline loop as condition 5; record the first run
  • 7501dcfa26 Close the condition-5 arc: steering, ledger, and the turn-latency assessment
  • 312ddf095f Delete the retired condition-3 instrument and fix the condition-5 test stub
  • f2e52e298fa0bf9c8ffc Reconcile steering with dogsled; select the black triangle as the frontier (S-011)
  • eaef0fdca9dcf5b37ebf Control vocabulary (moves, streams, sequences, partition) and the partition-worktrees protocol
  • 323cdd09f8, 19f7e9247a Repair stale control docs; anchor the control-surface test on the governing-concerns section
  • f4ced883eb9543278418 Driver-side steering reconciliation during the partitioned arc

Verification

turbo run test:unit lint:eslint lint:tsc across @hashintel/brunch-agent, the repertoire, both plugins, binding-flue, transport-aisdk, and @apps/brunch-agent; the architecture suite (package direction, Flue entrypoint, hermetic runtime, control surfaces) passes. The schema-drift test ties plugin.schema.json to PluginDefinitionSchema. Condition-4 and condition-5 runs are recorded as immutable snapshots under docs/evidence/evaluations/process-model-elicitation/baseline/runs/.

Stack

main → #9322 (FE-1404) → #9325 (FE-1497) → #9327 (FE-1406) → this PR → #9345 (FE-1504) → FE-1506 → FE-1505 → FE-1431 cycle two → FE-1507. The packages/repertoire workspace this PR introduces is removed again by the FE-1507 branch under ADR-0008; it is kept here because the cycle-two freeze evidence was produced against it.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

@vercel

vercel Bot commented Aug 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview Aug 27, 2026 5:56pm
petrinaut Ready Ready Preview Aug 27, 2026 5:56pm
petrinaut-docs Ready Ready Preview Aug 27, 2026 5:56pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Aug 27, 2026 5:56pm

Request Review

lunelson commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

lunelson and others added 23 commits August 27, 2026 19:41
Cycle one of ADR-0007's co-authoring convergence (S-009): the harness-owned
key catalogue exists as code, both test-case plugins are written against the
same schema, and the harness's own teaching ships as a package that the
binding renders interleaved with each plugin's cells.

Harness (`packages/core`):
- `keys.ts` is the catalogue — jobs, the eight guidance keys, the two
  movements, the three runbook keys — with the one-paragraph definition the
  interviewer reads above every rendered key.
- `plugin-definition.ts` reads `plugin.yaml` under a strict valibot schema
  (an unknown key anywhere fails to load) and cross-checks what a schema
  cannot say: rows name declared kinds, every kind has a row, the anchor is a
  counted `must_know` row, runbooks belong to declared jobs. The anchor is
  declared (`schema.anchor`), replacing `objective`-by-convention; precision
  words become harness vocabulary (`PRECISION_LADDER`).
- `repertoire.ts` types the repertoire and requires every key filled and every
  entry sourced.
- `instructions.ts` renders harness preamble → contract → each guidance key
  (definition, default, cell) → each declared job's runbook. The completion
  protocol sentences move from `cue.ts` into `HARNESS_PREAMBLE`.
- `schema/plugin.schema.json` is derived from the valibot schema
  (`yarn schema:emit`) and a structural drift test keeps it honest;
  `schema/CHANGELOG.md` opens the cycle log with cycle one's shape and its
  open questions.
- `plugin-file.ts` (the Markdown parser) and its test are removed; completion,
  fold, slot-assertion, cue, and `Plugin.definition` consume the definition.
  Fixture is now YAML.

Repertoire (`packages/repertoire`, `@hashintel/brunch-agent-repertoire`):
every guidance and runbook cell filled from the v0 prompt, the failure
catalogue (FM-02/03/04/06/07/08/12/13/14/15), the condition-3 hints, GEN-Q02,
the literature cards, and ADR-0007 decision 4; each entry names its source.
Depends on core only; the boundaries gate now checks that only bindings depend
on it, and the role regex and kernel §12.2 admit it.

Plugins: `plugin-sdcpn/plugin.yaml` (objective anchor, ten kinds, 24 rows,
eight patterns, formalism-specific cells only) replaces `plugin.md`;
`plugin-gherkin/plugin.yaml` (feature anchor, four kinds, feature → rule →
example → step) is written in the same cycle so the schema is pulled by two
formalisms at once. Gherkin keeps its `statement-noted` proposal for now; the
binding folds and cues only for definitions that declare `slot-asserted`.

Binding: `renderInstructions(repertoire, definition)` replaces the plugin-file
instructions and the separate completion fragments.

Docs: `plugin-contract.md` amended to the key contract; INDEX rows, kernel
§12.2, README, CONTEXT, and every `plugin.md` link repointed; STEERING's
authoring lane names the simulated-interview re-run as each cycle's run stage.

Gates: `test:unit`, `lint:tsc`, `lint:eslint`, `build` green across core,
repertoire, binding-flue, plugin-gherkin, plugin-sdcpn, and apps/brunch-agent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first "run" stage of an ADR-0007 co-authoring cycle (S-009; STEERING's
authoring lane): re-do the simulated interview with the interviewer taught by
the rendered repertoire and SDCPN plugin definition, and nothing else.

`run.ts 4` assembles the system prompt from `condition-4-prompt.md` — a
framing that stands in for the harness preamble, telling a harness-less
interviewer to keep the Must-know tally and the assumption ledger itself — plus
`renderContract`, `renderGuidance`, and `renderRunbook(construct)` from the
harness's built output over the shipped `repertoire.yaml` and
`plugin-sdcpn/plugin.yaml`. The assembled prompt is written beside the
transcript as `condition-4-system.md` so the run is reproducible against the
exact text it saw. Mechanics are conditions 1–2's (legacy impatience probe,
turn budget, delivery classifier), so the 2→4 delta isolates what the fixed
keys and the repertoire buy over the seven-category prompt; a later
harness-in-the-loop run isolates what the machinery buys over this text.

Verified hermetically through the stub client (delivers on turn 1; the
system prompt renders every key in catalogue order, ~5.7k words).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first co-authoring cycle's validate step: a read-only desk review of the
ADR-0007 key catalogue against 100 situations from the CPS process-modelling
material, the literature, and the condition-2 run, with a discrete-event and a
formal-verification plugin sketched against the keys. Placed as evidence under
docs/evidence/proofs/design with a provenance header and indexed.

Its verdict is folded into the schema changelog as the cycle-two input: no key
changes; two shape changes proposed (an optional slot predicate on pattern
rows; any-of precision on must-know rows), one applicability facet for
repertoire entries, the content findings (motifs restating patterns, the
posture nothing consumes, repertoire under-fill against the ADR's own rows,
seven source contradictions resolved silently), and eight changes considered
and left.

One defect it found is fixed here: a pattern indexed on no kind (`on: []`,
documented as "any node") never fired, because the sweep list tested
`kinds.includes(node.kind)` on an empty list. sdcpn P08 (source-regime
divergence) therefore never reached the interviewer as a harness fact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One run of the rendered ADR-0007 layer (condition-4 framing + repertoire +
sdcpn definition, prompt only, no machinery) against the simulated master
scheduler: 22 interviewer turns, delivered-after-forced-wrap. Transcripts,
raw record, assembled system prompt, and delivered model added; the read-out
appended to readout.md under its own heading with headline findings, the
seven-category and pack-tier tables extended with a condition-4 column, the
silent-assumption audit, stopping behaviour, the 2->4 delta, and a
"Strains for cycle two" list keyed to plugin.yaml and repertoire.yaml entries.

Headline: the best-disciplined questioning of any condition (objectives first,
one to three questions a turn, quantiles on every changeover type, per-slot
precision and an attributed assumption ledger in the deliverable) and the
thinnest model (no run time, no arrival pattern, no QA spread; one tacit fact
of nine fully surfaced). Stopping failed in a third form: the interviewer
delivered on the expert's stop at turn 10 and then had no concept of ending.
Must-know rows were graded rather than asked; the anchor's dependency slot was
authored by the interviewer and drifted across three deliverables.

Instrument: the haiku delivery classifier answered NO on the turn-10
deliverable (replayed 3/3) because it names its own gaps, so the stop reason
overstates the failure; the raw record keeps classifier usage but not
verdicts. Both noted for the rerun.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The baseline protocol's fourth condition measured the ADR-0007 teaching
layer as prompt text; nothing yet measured the machinery around that text.
Condition 5 drives the shipped SDCPN elicitor in the Flue runtime — the
binding's ask suspension, settlement nudge, private sweep extraction into
the capture store, fold, and computed completion — against the same
simulated expert, probes, and turn budget as conditions 1, 2, and 4. The
runner reads every harness fact from durable history and the store, takes
interviewer tokens from Flue's observe() turn events, and needs no delivery
classifier: the deliverable is the folded store, and the interviewer ends
its own turn-taking by replying without a question.

The runner lives with the protocol but is launched from the application
package (`baseline:harness`), which owns the agent composition; the
elicitor's model id gains a `BRUNCH_SDCPN_MODEL` override so the runner can
use the same interviewer model as the other conditions. A hermetic test
swaps both models for stand-ins — the existing Anthropic stub for the
expert, a faux pi provider for the interviewer — and asserts the loop end
to end: ask bound, sweep applied with completion, stall detected, artifacts
written.

Condition 3 is retired before its first model call (superseded by the
harness it would have projected by hand; lock left sealed, amendment atop
the preregistration) and conditions 1 and 2 are frozen as reference
evidence, per the checkpoint review.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The shipped SDCPN elicitor (claude-opus-5) in the Flue runtime against the
simulated expert, single-shot: twelve interviewer turns, 267 captures,
69 nodes, completion not reached (46 unsatisfied; 53 nodes outside every
objective's slice). The runner's `stalled` rule fired on what the
transcript shows to be a deliberate close: at turn 10, immediately after
the impatience probe, the interviewer read the record back in the expert's
words and declined to call it finished; the two turns after it are
notes-to-self. Evidence only — the read-out and the cycle-two edits wait
for review. A first attempt that reached turn 10 died in a network outage
and is kept outside the tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…essment

Arc-close for the first harness-in-the-loop run. Records S-010 (retire
condition 3, freeze 1–2, conditions 4 and 5 as the live arms; harness
facts replace the classifier), the Proof 1 first-run result and narrowed
claim in STEERING, four belief rows, the candidate successor streams, and
the per-turn latency as an immediate concern with its own evidence
document: 29 min for 12 turns, ~97% of output tokens extraction on the
critical path, actions R0–R5 and a deferred spike. Ledger rows §8.1,
§8.2, §9.1, §9.5, §11.1 reflect what the run exercised. Notes the
FE-1404 numbering gap (its "condition 3" is the protocol's condition 5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t stub

Salvage assessed as none: the instrument's projection schema and
semantic validators encoded the domain-keyed DemandTable that S-007
ruled the wrong level; the kind-level fold and evaluateCompletion in
core now do that job on the production path. Removes
condition-3-instrument.ts (1,317 lines), its lock, the operator,
scoring, legibility, and pre-run-review documents, its unit test, and
the condition-3 paths in run.ts (1,720 → 623 lines; conditions 1, 2,
and 4 keep their reviewed resume/continue semantics). The
preregistration and prompt stay as the record, with a dated amendment.

Also: the condition-5 test passed replies inline while the shared
expert stub has always read them from BASELINE_STUB_REPLIES_PATH, so
the committed test could not pass; it now writes the replies file.
S-010's Supersedes line is bare "none" so the control-surfaces gate
accepts it. Protocol, STEERING, and S-010 record the deletion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Latency targets adopted provisionally; the isolating spike as first task
of the next arc marked provisional pending Lu's further next-steps input;
the condition-5 read-out deferred until after the spike; and the
black-triangle directive recorded as the frame the streams are worked
inside: the full end-to-end flow through the Petrinaut assistant
interface first, so the surface stabilises for the team and the voice-mode
work while the implementation behind it keeps changing. S-011 is appended
when Lu's input lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reduce the local steering protocol to its Brunch-specific supplement and route strategic refreshes through ds-steer. Align Wayfinder commission, landing, issue-close, and concern vocabulary with the canonical skills while preserving local Linear approval and proof-execution rules. Refresh STEERING as current concern-first control without diary or mirrored tracker history.
Two jobs, one order: construct through the Petrinaut panel is Proof 0,
review-and-revise sits on top. G0 (FE-1503) is the selected frontier with
its cut, parallel streams, gap assessment, and tool inventory pinned as
current truth against intent; G1 and G2 follow as a sequence. Records the
plugin-API-in-flux and package-topology concerns, the voice-edge attach
surface and provider gate, and defines legibility on the human in the
legibility protocol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The work inside G0 shares one proof and is a set of moves joined at Proof 0,
not streams; "stream" is reserved for separately proven parallel work, and
G0 -> G1 -> G2 is a sequence. Defines the three terms in CONTEXT.md and
corrects STEERING, S-011 (addendum), and the legibility protocol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ds-steer selects one frontier and leaves other available work unplanned;
Brunch adds a partition step: efforts with their own proof, projection,
disjoint write set, and named join points, run in separate worktrees with
control documents written only from the driver. Records today's partition
(W1-W5 and the voice work) in STEERING and defines effort and join point
in CONTEXT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
S-011's link to the H-6763 plan pointed at a file that exists only on the
voice branch; three older strategy entries linked STEERING headings that no
longer exist; the E1 and E2 epicentre rows described gaps that landed on
the stack. Adds the unmerged stack as an active gate, since G0's baseline
is not on main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The steering control retired the `Governing strategic decisions:` line in
6903277; the architecture test still matched it and failed on every
checkout. Match the `## Governing concerns` section instead, keeping the
invariant that steering cites at least one strategy entry and every cited
entry resolves and is unsuperseded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The app's condition-5 test executes the context-owned runner, so
package-isolated CI must copy that non-workspace path.

Co-authored-by: Cursor <cursoragent@cursor.com>
The contract text and slice reader both assumed objective-shaped
demand: a hardcoded "objective-relative" floor line, and a dependency
reader that dropped a single kind:node string. Steering also linked
ADR-0008 and cycle-one transcripts this branch does not yet carry.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > frontend Owned by the @frontend team type/legal Owned by the @legal team

Development

Successfully merging this pull request may close these issues.

1 participant