Skip to content

epic: data-driven harness descriptors (instant magic, phase 2) #122

Description

@slowdini

Part of #113 ("Instant magic"). Phase 2 / future tracker — NOT scheduled. Revisit only after Epic 1 (#A–#D) lands and proves the value.

Why

Epic 1 (#A–#D) reframes harness compatibility as a documented minimal interface + progressive enhancements, but the per-harness specifics still live as hardcoded Rust (each adapter impl). The ambitious end state #113 gestures at is "any harness compatible as long as it supports certain features natively" — i.e. add a baseline harness via configuration + docs, with no Rust.

Scope sketch

  • Move the hardcoded, declarative per-harness specifics into a data-driven descriptor: exec command template, events filename, model flag, skills dir, skill-block format.
  • Decide whether descriptors are compile-time embedded data or user-providable config files (the latter is what makes "bring your own harness" real).
  • Out of scope (stays code): transcript parsers and write-guard hooks — these are genuinely format/hook-specific and cannot be reduced to a descriptor.

Open questions

  • Embedded vs user-config descriptors (or both)?
  • How does a user-supplied descriptor coexist with the built-in ones (override/extend)?
  • What's the minimum descriptor that still produces a valid llm_judge-graded run?

Dependencies

Epic 1 (#A, #B, #C, #D) complete.


Filed from the #113 feasibility/breakdown plan.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions