Skip to content

feat(workflows,subagents): progress consumers — loop ledger + attention input (V8, #2489) - #2518

Merged
flora131 merged 8 commits into
mainfrom
verifier/progress-consumers
Aug 19, 2026
Merged

feat(workflows,subagents): progress consumers — loop ledger + attention input (V8, #2489)#2518
flora131 merged 8 commits into
mainfrom
verifier/progress-consumers

Conversation

@flora131

@flora131 flora131 commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Slice V8 of the LLM-as-a-Verifier adoption program (specs/2026-08-17-progress-scoring.md §5.2): the two consumers of the progress-scoring module.

  • loop-until-done: after each iteration, one score_progress call over the iteration-summary prefix (repeats 1); {iteration, score, trend} appended to the loop ledger; the stop/exhaustion report gains the score curve, final trend, and an advisory disclaimer. The stop condition is unchanged — the done boolean and iteration cap decide exactly as before, proven by byte-identical stop decisions with scoring enabled vs disabled.
  • subagent attention: classify_trend duplicated as a tiny pure helper in packages/subagents (mirrored table tests in both packages keep the copies honest); score series arrives via the existing control-config surface; a regressing/flat-low trend may raise attention priority beside the wall-clock signals — it can never mark a run failed, never suppresses a wall-clock signal, and the subagents runtime schedules no model calls of its own.
  • Rider: coding-agent's bundled slash-command metadata for loop-until-done synced to the new inputs.

Base: verifier/progress-scoring (V7, #2514). Stack: V1 → V3 → V7 → V8.

Evidence

Produced by an implement→review→repair goal run (approved turn 1; completion/evidence/risk reviewers all complete; remaining_work: none):

  • npm run check — green (includes the coding-agent tsgo erasableSyntaxOnly pass)
  • Loop + subagents attention suites green: stub scorer → ledger entries + report curve; stop decisions byte-identical with scoring enabled vs disabled; attention raises priority, never failure; mirrored trend tables pass identically in both packages; no code path mutates run status from a TrendResult
  • Size cap: 224 changed source lines (< 500; tests uncapped — 706 total insertions across 14 files)
  • CHANGELOG: both packages/workflows/CHANGELOG.md and packages/subagents/CHANGELOG.md under ## [Unreleased] ### Added

Spec contract: specs/2026-08-17-progress-scoring.md §5.2 (Q2: iteration-summary prefix; Q3: minimal subagent shape).


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Greptile Summary

This change adds advisory progress scoring to loop-until-done and lets progress trends increase subagent attention without changing terminal outcomes. The completed-loop report is inconsistent: callers receive the progress curve, trend, and disclaimer in the returned result, but the documented result_path file omits them. Update the completion artifact to contain the same complete report before merging.

Confidence Score: 4/5

Not safe to merge until the completion artifact preserves the full report returned to workflow callers.

A focused runtime test exercised the completed loop path and directly showed that the returned report and its documented artifact differ. The remaining comment enforces the repository's static typing requirement for the source guard.

Files Needing Attention: packages/workflows/builtin/loop-until-done-runner.ts needs to persist the composed report at result_path; test/unit/subagents-progress-attention.test.ts needs a more precise AST node representation.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex produced a proof for a posted P1 finding and linked it to the review comment.
  • T-Rex produced a second proof for another posted P1 finding and linked it to the review comment.
  • T-Rex performed general contract validation by reviewing the terminal report, progress curve, and final trend, noting that appendix markers were absent in the result_path, and uploading three related artifacts.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. General comment

    P1 Completed result artifact omits the returned progress appendix

    • Bug
      • On the done-decision path, callers receive progress curve, final trend, and disclaimer in result, but the documented result_path artifact contains only the completion-summary stage text. The runtime harness observed all three markers present in result and absent from result_path.
    • Cause
      • completion-summary writes resultPath before line 243 constructs the appended result; the appended string is returned but never written back to resultPath.
    • Fix
      • After composing the final result, write that exact composed content to resultPath (or make the completion-summary stage produce the same appendix before it writes the artifact), then return it consistently.

    T-Rex Ran code and verified through T-Rex

Prompt To Fix All With AI
### Issue 1
packages/workflows/builtin/loop-until-done-runner.ts:243
**Completion artifact omits returned progress report**

On the completed path, `completion-summary` writes `result_path` before this code appends the progress curve, final trend, and disclaimer to `result`. Callers therefore receive a complete returned result but read only the terminal completion text from the documented artifact path. Write the composed result back to `resultPath`, or include the appendix before the completion task writes the artifact.

### Issue 2
test/unit/subagents-progress-attention.test.ts:61-65
**AST guard uses ambiguous types**

The new AST traversal represents parser nodes through broad `unknown` values and an `unknown` index signature, forfeiting useful static checks and making future changes to this source guard more error-prone.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "merge origin/main" | Re-trigger Greptile

Greptile also left 2 inline comments on this PR.

Context used:

  • Context used - AGENTS.md (source)

The V8 loop-until-done consumer added progress_scoring and
progress_repeats inputs. The bundled slash-command completion
metadata and the workflows doc table still listed only prompt and
max_iterations, so /workflow completions omitted the new inputs and
builtin-workflows-builtin-index-manifest-01 failed its mirror
assertion.
@flora131
flora131 marked this pull request as ready for review August 19, 2026 16:53
@flora131
flora131 changed the base branch from verifier/progress-scoring to main August 19, 2026 16:59
@flora131
flora131 merged commit e66fc2a into main Aug 19, 2026
20 of 24 checks passed
// `result_path` below carries it for callers that want the contents.
outputMode: "file-only",
});
const result = `${final.text ? `${final.text.trimEnd()}\n\n` : ""}${formatProgressReport(report)}\n`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Completion artifact omits returned progress report

On the completed path, completion-summary writes result_path before this code appends the progress curve, final trend, and disclaimer to result. Callers therefore receive a complete returned result but read only the terminal completion text from the documented artifact path. Write the composed result back to resultPath, or include the appendix before the completion task writes the artifact.

Artifacts

Focused done-decision runtime harness source

  • The authored Vitest harness drives the loop through a done evaluation, progress scoring, completion artifact write, and comparison of returned text with result_path; it provides the reproducible test setup.

Returned result and result_path comparison from the executed harness

  • The executed harness captured a returned result containing curve, final trend, and disclaimer while the completion artifact contains only the terminal report; the appendix is absent from result_path.

Focused runtime harness pass log

  • Vitest ran the focused done-decision harness successfully from /home/user/repo with exit code 0; the behavior is reproducibly confirmed.

View artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/workflows/builtin/loop-until-done-runner.ts
Line: 243

Comment:
**Completion artifact omits returned progress report**

On the completed path, `completion-summary` writes `result_path` before this code appends the progress curve, final trend, and disclaimer to `result`. Callers therefore receive a complete returned result but read only the terminal completion text from the documented artifact path. Write the composed result back to `resultPath`, or include the appendix before the completion task writes the artifact.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +61 to +65
"canceled",
"aborted",
"stopped",
"terminated",
]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 AST guard uses ambiguous types

The new AST traversal represents parser nodes through broad unknown values and an unknown index signature, forfeiting useful static checks and making future changes to this source guard more error-prone.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: test/unit/subagents-progress-attention.test.ts
Line: 61-65

Comment:
**AST guard uses ambiguous types**

The new AST traversal represents parser nodes through broad `unknown` values and an `unknown` index signature, forfeiting useful static checks and making future changes to this source guard more error-prone.

**Context Used:** AGENTS.md ([source](https://github.com/bastani-inc/atomic/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant