Sub 5 (#23): Provider transport via Claude CLI + user auth (deferred)

[usurobor]

Parent: #23
Labels: enhancement, P3, runtime
Priority: P3 — explicitly deferred. Not on the v3.2.0 critical path. The current ezcurl + LLM_API_KEY transport works.
Status: Not started. Deferred until master #23 critical path (Sub 1, 2, 3, 4, 6) is closed.

Problem

What exists: engine/ocaml/bin/provider.ml calls a provider HTTP endpoint via ezcurl, authenticated by LLM_API_KEY from environment or .tsc/.env. Provider name and model are configured via LLM_PROVIDER and LLM_MODEL.

What is expected: The engine can invoke Claude via the local claude CLI using the operator's existing CLI auth, eliminating the need for a separately-managed LLM_API_KEY.

Where they diverge: No CLI-based provider exists. All LLM calls are direct API.

Impact

• Operator must maintain a separate API key alongside their claude CLI auth.
• API key handling in .tsc/.env is one more secret surface.
• This is operator convenience and secret-surface reduction, not a correctness issue.

Status truth

Surface	Status	Notes
engine/ocaml/bin/provider.ml	Shipped	ezcurl + JSON.
engine/ocaml/bin/dotenv.ml	Shipped	Loads LLM_API_KEY.
Claude CLI (claude)	Available externally	Not invoked from the engine.
Operator's CLI auth	Available externally	Not consumed.

Source of truth

Claim / surface	Canonical source	Status
Provider transport interface	engine/ocaml/bin/provider.ml	Shipped
LLM scoring instruction	runtime/SELF-MEASURE.md	Shipped (will be rewritten in Sub 1)
Bundle / prompt assembly	engine/ocaml/lib/prompt.ml	Shipped
Claude CLI documentation	External (Anthropic docs for claude CLI)	External

Scope

In scope (when un-deferred):

• New module engine/ocaml/bin/provider_claude_cli.ml (or pluggable provider behind a common signature) that invokes claude via process spawn, sends the prompt on stdin, parses the response on stdout.
• Provider selection: LLM_PROVIDER=claude_cli resolves to the new module; existing values (anthropic, openai, etc.) keep their direct-API behavior.
• Stream/large-prompt handling consistent with current prompt.ml size budget.
• Operator manual (and QUICKSTART) update describing both transports.

Out of scope:

• Replacing direct-API transport entirely. Both should coexist.
• Multi-provider parallel scoring.
• Claude-specific features beyond what runtime/SELF-MEASURE.md exercises.

Deferred behind this sub-issue:

• Anything else relying on out-of-process tool invocation.

Decision needed before implementing

• Process boundary: how does the engine call claude? Likely Lwt_process or Unix.create_process with a JSON prompt assembly that the CLI consumes via --print or stdin.
• Response shape: does Claude CLI emit raw text the engine then parses against response_schema.ml, or does the CLI's structured-output mode produce a JSON shape directly?
• Error handling: rate limits, auth refresh, partial responses. The direct-API path uses HTTP status; the CLI path uses exit code + stderr.
• Cost visibility: the CLI charges to the operator's plan; the engine should record provider: claude_cli and any usage metadata the CLI exposes.

Acceptance criteria (when un-deferred)

AC1: Claude CLI provider works end-to-end

Invariant: LLM_PROVIDER=claude_cli coh --target spec produces a valid report using the operator's CLI auth, no LLM_API_KEY required.
Oracle: Integration test (manual when un-deferred — full automation requires a claude mock).
Positive: Report produced; provenance records provider: claude_cli.
Negative: Engine fails or silently falls back to direct API.
Surface: engine/ocaml/bin/provider_claude_cli.ml, integration test.

AC2: Direct-API transport unchanged

Invariant: Existing direct-API providers (anthropic, etc.) work exactly as before.
Oracle: Existing integration tests.
Positive: No regression.
Negative: Direct-API behavior changed.
Surface: engine/ocaml/bin/provider.ml.

AC3: Documentation reflects both transports

Invariant: Operator manual describes both LLM_PROVIDER=claude_cli and LLM_PROVIDER=anthropic (or whatever direct-API name is canonical) paths.
Oracle: Manual review.
Positive: Both transports described, with caveats (CLI: requires claude on PATH; direct: requires API key).
Negative: One omitted.
Surface: docs/beta/guides/OPERATOR-MANUAL.md.

AC4: Provenance records transport

Invariant: Reports record which provider transport was used.
Oracle: Schema check.
Positive: provider_transport ∈ {direct_api, claude_cli} present.
Negative: Field absent or implicit.
Surface: engine/ocaml/lib/report.ml.

Proof plan (when un-deferred)

Invariant: Engine has a working Claude CLI transport in addition to the direct-API path; transport choice is operator-visible.
Surface: engine/ocaml/bin/, operator manual.
Oracle: Manual integration runs (one per transport); schema check on reports.
Positive case: Both transports pass on a representative target.
Negative case: Either transport regresses.
Operator-visible projection: Operator chooses transport via LLM_PROVIDER; provenance records the choice.
Known gap: Automating the CLI-transport integration test requires a mock for claude (out of scope).

Skills to load (when un-deferred)

Tier 3:

• cdd/design,
• cdd/plan,
• cdd/review,
• cdd/release.

Why:

• Process-boundary work has subtle failure modes; design pass before implementation.

Active design constraints

• Both transports MUST coexist. This is additive, not a replacement.
• runtime/SELF-MEASURE.md remains the canonical instruction regardless of transport.
• Provider selection via existing LLM_PROVIDER env var.
• Operator's CLI auth is consumed transparently; the engine must not store or proxy credentials.

Related artifacts

• engine/ocaml/bin/provider.ml
• engine/ocaml/bin/dotenv.ml
• engine/ocaml/lib/prompt.ml
• engine/ocaml/lib/response_schema.ml
• runtime/SELF-MEASURE.md
• docs/beta/guides/OPERATOR-MANUAL.md
• External: Claude CLI documentation

Non-goals

• Replacing direct-API transport.
• Multi-provider parallel comparison.
• Streaming the CLI's incremental output to the operator.

Success / closure condition

This issue is closeable when:

• AC1–AC4 all pass (after un-deferral).
• Both transports coexist.
• Operator manual describes both.
• Reports record transport choice.
• Master Bring TSC to coherent state — engine implements v3.2.0, Python retired, tests in OCaml #23 acknowledges closure (or closes as deferred-debt with this sub-issue tracked separately).

Defer rationale

The user's explicit framing — "We'll use Claude CLI for LLM calls using my auth (can prob wait)" — is recorded here as the deferral basis. Master #23 may close before this sub-issue resolves, provided this is named as named debt in the closing self-coherence artifact.