Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions docs/dag-system-review-2026-08-09.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# DAG system review — 2026-08-09

## Outcome

The review found one reproducible correctness defect, fixed it before any
architecture work, and found no second defect that could be reproduced through
a supported runtime path. The scheduling core is modular and well covered. The
remaining work is change-locality and recovery hardening, not a rewrite.

## Scope and evidence

Reviewed surfaces:

- graph validation, admission, scheduling, transitions, projection, and store
code under `packages/core/src/dag`;
- workflow commands, runtime scheduling, recovery, spawning, wake delivery, and
summary publication under `packages/opencode/src/dag` and
`packages/opencode/src/tool/workflow.ts`;
- DAG inspector state, reducers, layout helpers, and rendering under
`packages/tui/src`;
- durable events and generated boundary types used by those packages.

Verification baseline:

- `packages/core`: 90 targeted DAG tests passed;
- `packages/opencode`: 389 targeted DAG tests passed before the fix, 392 after
adding three deterministic regressions;
- `packages/tui`: 50 targeted DAG tests passed;
- `packages/opencode`: `bun typecheck` passed;
- the `dev` push CI for the fix passed Linux E2E; Linux unit and Windows E2E
were still running when this report was written.

Coverage is strongest at the core state-machine seams: graph validation,
scheduling, transitions, admission, evaluation, and projection are effectively
fully covered. Runtime execution is also high but less complete:
`src/dag/runtime/loop.ts` was 93.73% line / 92% function, spawn 94.58% line,
recovery 99.30% line, and the workflow tool 89.81% line / 79.49% function.
Coverage alone did not expose the confirmed timing defect.

## Confirmed bug — fixed

### Summary updates could be lost during an in-flight read

`packages/opencode/src/dag/runtime/summary-publisher.ts` used a `Set` as an
early-return coalescer. An event arriving while a workflow or Session summary
read was already running observed the key in the set and returned. The active
read could have captured the old state, and no dirty rerun was scheduled, so
the TUI could remain stale until an unrelated later event.

Three `Deferred`-gated regression tests reproduced the lost update for:

1. two events for one workflow;
2. two workflows sharing one parent Session;
3. a newer event arriving while the first read fails.

The fix replaces the boolean in-flight set with keyed dirty state. Events
during debounce are absorbed; events during an active read mark the key dirty;
completion or failure reruns once with the latest durable state. Interruptions
still propagate. The fix is merged to `dev` in PR #202.

## Architecture findings

### A1 — Runtime loop has poor change locality (high, no behavior change yet)

`packages/opencode/src/dag/runtime/loop.ts` constructs most runtime behavior
inside one roughly 1,200-line `layer` closure. It owns adoption, subscriptions,
child-session ownership, spawn planning, wake batching, delivery, recovery,
and terminal decisions. The public module is deep, but the internal
collaborators are invisible to code navigation and can only be tested through
the whole layer.

Recommended boundary: keep one public runtime layer, but extract cohesive
private constructors for child ownership, wake delivery, and recovery. Each
constructor should receive the minimum services it uses and expose only the
operation needed by the coordinator. Do this in behavior-preserving commits
after adding tests for the uncovered failure branches.

### A2 — Workflow command dispatch mixes transport and domain preparation (medium, scheduled)

`packages/opencode/src/tool/workflow.ts` has one 200+ line `execute` switch
(cyclomatic complexity 21, cognitive complexity 64). It combines parameter
validation, file/YAML transport, admission normalization, model readiness,
domain commands, and user-facing receipts. This coupling is also why a one-off
graph must be written to YAML before it can start.

The next product change will introduce one spec-source boundary: inline
structured specs are the default for one-off start/extend/replan operations;
`spec_path` remains for saved workflows. Action handlers may be extracted only
where this names a real boundary and lowers the main dispatch complexity.

### A3 — Core and TUI seams are appropriately deep (retain)

The core splits graph rules, scheduling, transitions, projection, and storage
into independently testable modules. The TUI consumes server summaries and
keeps non-trivial layout logic in pure utilities. Recombining these modules or
moving aggregation into the TUI would make the system harder to verify.

## Unconfirmed robustness risks

These are review observations, not bugs. No supported-path red test was found.

- `readWakeBatch` catches typed failures, while store database failures are
defects (`orDie`). A defect can postpone delivery until another event or
restart. Add an explicit retry policy only after a fault-injection test
proves the desired semantics.
- Startup recovery intentionally admits that a store defect can defer
redelivery until the next restart. This is operationally weak, but changing
it requires a retry/backoff and shutdown contract.
- The final review-acceptance guard does not re-check an output fingerprint.
Normal spawn and recovery paths validate fingerprints before settlement, so
no supported writer currently reaches the stale state. Keep this as
defense-in-depth backlog unless a reachable sequence is demonstrated.

## Batch C decision

The `spawnReady` O(ready × nodes) candidate remains closed without code. Current
workflow limits and observed test/runtime scale do not show material cost, and
the scheduling implementation is easy to reason about. Reopen only with a
profile showing scheduling latency or CPU cost at realistic node counts.

## Ordered follow-up

1. Ship the inline workflow-spec and parent-orchestrator policy change.
2. Refactor `runtime/loop.ts` behind regression tests, without changing the
public layer contract.
3. Add store fault injection, then decide retry/backoff semantics from evidence.

6 changes: 3 additions & 3 deletions packages/core/src/plugin/command/dag-flow.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ For a non-empty task:
- deep review of an already-built module, subsystem, or codebase → saved workflow `deep-review-dag-module`
- a small bounded working-tree change review → saved workflow `change-review`
- none of the above names resolves (bare dev checkout without the config repo) → compose the smallest fresh graph; do not force an unrelated reference
2. Treat the selected YAML as a reviewed topology reference, not as a script to replay blindly. Start a saved workflow by name only when its embedded target and inputs already match the request. Otherwise read the reference, derive a one-off YAML, inject the complete `/dag-flow` task into its root planning/exploration prompt, retarget its lanes, and pass that file to `workflow(action=start)`.
2. Treat the selected saved spec as a reviewed topology reference, not as a script to replay blindly. Start a saved workflow by name only when its embedded target and inputs already match the request. Otherwise read the reference, derive one inline `spec`, inject the complete `/dag-flow` task into its root planning/exploration prompt, retarget its lanes, and pass it directly to `workflow(action=start)`. Do not create a transient YAML file.
3. The derived graph may expand or prune non-protected lanes. Record the selected `reference_template`, every added node, and every prune as `{node, prune_reason, replacement_coverage}` in the first planning/exploration artifact; require the next fresh review gate to audit that manifest. Missing prune evidence is fail-closed.
4. Preserve the selected reference's protected spine: fresh-context local review, deterministic/evidence verification where applicable, one final arbiter, and PASS-only finalization. Gates return `PASS | LOOP | BLOCKED` with reason, evidence, minimal `loop_scope`, and `stop_reason`. `LOOP` means pause → replan new local correction/review nodes → resume; never create a cycle or restart terminal nodes.
5. During compilation, preserve every user constraint in the graph, including named `@agent` roles, exact model selections, read-only or "Do not modify files" scope, required checks, forbidden actions, and requested deliverables.
6. Resolve capability slots against the eligible configured worker types shown in the `workflow` tool description. Do not invent a missing role or model; if a required capability cannot be resolved, do not start and report the gap.
7. Scale the graph to the task's blast radius. A small, well-bounded target gets the smallest useful dependency graph. A large or system-level target (an entire module, subsystem, or codebase) is never satisfied by a single wave of parallel opinions: stage exploration, independent analysis, evidence verification, and synthesis as separate dependent waves. Keep independent viewpoints or work packages parallel and use real fan-in nodes for synthesis, arbitration, integration, and final reporting.
7. Scale one consolidated graph to the task's blast radius. Related flows for this user objective become nodes and edges under the same workflow ID. A small, well-bounded target gets the smallest useful dependency graph. A large or system-level target (an entire module, subsystem, or codebase) is never satisfied by a single wave of parallel opinions: stage exploration, independent analysis, evidence verification, and synthesis as separate dependent waves. Keep independent viewpoints or work packages parallel and use real fan-in nodes for synthesis, arbitration, integration, and final reporting.
8. For a large-target review or audit, require every reviewer to cite file:line evidence and to mark claims it could not verify. Insert a verification wave between the reviewers and the arbiter that checks disputed, unverified, and uncovered scope against the actual code, so the arbiter rules on verified findings only.
9. Call the `workflow` tool with `action=start` in this response. Merely printing a plan, graph, JSON, or YAML does not mean a workflow was started.
9. Call the `workflow` tool with `action=start` and inline `spec` in this response. Use `spec_path` only when the selected saved workflow already matches or persistence was explicitly requested. Merely printing a plan, graph, JSON, or YAML does not mean a workflow was started.
10. Do not claim the workflow is running unless the tool call succeeds. On success, report the exact Workflow ID and initial state returned by the tool, then tell the user to run `/dag` for live inspection.
11. The workflow runs asynchronously and wakes this parent session when attention or a terminal result is ready. Do not poll it with `action=status`, sleep, retry, or loop merely to wait. End the current response after the brief success report.
12. On failure, state that the workflow was not started and report the actual error. Never invent a Workflow ID or start a replacement workflow unless the user explicitly asked for automatic retries.
Expand Down
7 changes: 3 additions & 4 deletions packages/core/src/plugin/command/orchestration-domains.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,8 @@ against the code, with fix waves through the audit loop).

## Choosing and Combining

Playbooks compose: Large Engineering embeds Deep Review at its gate; Deep
Speculation can front-load any of them. Selection still obeys Execution Mode
Selection and the Depth Ladder — a playbook is justified only when the task
shows both a scenario and a structural signal, its wave count meets the
Playbooks compose inside one live DAG: Large Engineering embeds Deep Review at
its gate; Deep Speculation can front-load any of them. Selection still obeys
Execution Mode Selection and the Depth Ladder — its wave count meets the
ladder's minimum for the target size, and explicit user constraints always
override the playbook shape.
30 changes: 22 additions & 8 deletions packages/core/src/plugin/command/orchestration-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,17 +52,31 @@ like "review X" never does.

## Execution Mode Selection

Choose the smallest execution mode that can safely complete the request:

1. Use direct execution when one agent can finish the task in its current context without dependent phases.
2. Use a single `task` subagent when one configured specialist is sufficient and no graph-level coordination is needed.
3. Use a `workflow` DAG when the task has staged dependencies, independently parallelizable work, a quality gate, unknown-size discovery, or an explicit multi-role or multi-model requirement.
The parent conversation owns user interaction, requirement and admission
decisions, the macro plan, workflow controls, checkpoint interpretation, and
the final user-facing synthesis. Once work is classified for delegation, the
parent MUST NOT perform executable leaf work itself.

Choose the smallest child execution mode that can safely complete the request:

1. Use direct execution only for conversation, trivial state inspection,
workflow control, final synthesis, or an explicit user opt-out.
2. Use one `task` subagent for one independent non-trivial leaf assignment when
no graph-level coordination is needed. The parent launches it once, consumes
its result, and does not duplicate the leaf work.
3. Use one live `workflow` DAG when one user objective contains staged
dependencies, two or more related workstreams, a quality gate, unknown-size
discovery, adaptive repair, or an explicit multi-role or multi-model
requirement.

"Smallest" is measured against the Depth Ladder: a mode or graph that cannot
deliver the ladder's hard minimum for the target size is not safe, merely
small.

Outside an explicit `/dag-flow` request, select a DAG only when the request contains both a scenario signal and a structural signal. Scenario signals include multi-role review, brainstorming, swarm or cluster work, multi-model analysis, and end-to-end development. Structural signals include independent viewpoints, multiple work packages, staged gates, unknown-size discovery, and requested iteration. A lone keyword such as "review" is not sufficient.
Related flows for one user objective belong to one live DAG. Represent them as
nodes and dependency edges; use `extend` or `control(replan)` when discovery or
a verdict adds work. Start another DAG only after a terminal boundary prevents
live adaptation, and carry the prior outputs into the continuation explicitly.

Explicit user constraints override profile defaults:

Expand Down Expand Up @@ -133,8 +147,8 @@ continue QA, reduce scope, use `standard`, or explicitly waive. A `WAIVED`
start is informed only when both `waiver_reason` and `acknowledged_risks` are
non-empty; preserve them for audit.

Do not supply `protocol_version`, `state`, or `fingerprint` in the YAML
admission input. Those are durable audit fields owned by the workflow boundary:
Do not supply `protocol_version`, `state`, or `fingerprint` in the admission
input. Those are durable audit fields owned by the workflow boundary:
it sets protocol version 1, initializes state from the verdict, normalizes the
Brief for fingerprint computation, and computes the lowercase hexadecimal
SHA-256 hash. A successful deep start alone transitions the durable record to
Expand Down
Loading
Loading