Skip to content

feat(workflows): budget duration enforcement + soft landing (B2, #2212) - #2527

Closed
flora131 wants to merge 50 commits into
budgets/config-reducerfrom
budgets/duration-enforcement
Closed

feat(workflows): budget duration enforcement + soft landing (B2, #2212)#2527
flora131 wants to merge 50 commits into
budgets/config-reducerfrom
budgets/duration-enforcement

Conversation

@flora131

@flora131 flora131 commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Slice B2 of the run-budgets track (specs/2026-08-17-run-budgets.md §5.1 enforce_budget + §5.2b + §8 B2): duration enforcement with the Codex-style one-turn wrap-up and the resumable budget_exceeded outcome.

  • Checkpoints: before stage dispatch, before ctx.tool, after either completes, on durable resume
  • Duration via elapsedRunMs (paused excluded, accumulatedDurationMs included; resume carries prior spend)
  • Warn once per run per dimension at warnAtPercent
  • Exhaustion: one wrap-up injection to the frontier stage (current turn only, no new stages, at most once) → blocked rail with returned status budget_exceeded (system-owned, resumable)
  • Child-scoped budgets: child exhaustion soft-lands only the subtree; root wins simultaneous exhaustion
  • Unbudgeted runs take a zero-cost early return (R10)

Base: budgets/config-reducer (B1, #2505). Stack: B1 → B2 → B3.

Size note

843 changed source lines across 37 files / 50 commits — over the program's 500 cap. The goal run spent 10 turns implementing and repairing real enforcement defects (wrap-up semantics, blocked-rail, resume attribution, child-scope, R10) and ended blocked solely on the size clause, not on missing behavior. Same honest posture as V5 (#2517 at 828 lines): one PR, review commit-by-commit; a post-hoc split would scramble the review history. Token/cost metering stays B3.

Evidence

Produced by a 10-turn implement→review→repair goal run. Final remaining work is only the size-cap finding. Gates exercised throughout (budget unit suite + npm run check); last commits cover child/sync boundaries, root budgets inside child runs, session-restore eligibility, and wrap-up determinism.

  • CHANGELOG: packages/workflows/CHANGELOG.md under ## [Unreleased] ### Added

Spec contract: specs/2026-08-17-run-budgets.md §5.1 / §5.2b / §8 B2 (Q1 child-scoped; Q2 current-turn wrap-up).


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Greptile Summary

This change adds duration-budget enforcement across workflow execution, persists budget metadata for durable sessions, and surfaces budget exhaustion through lifecycle and status handling.

An exhausted engine-owned workflow can resume without restoring the one-time warning and frontier wrap-up markers. A later resume can therefore repeat the lifecycle warning and use another frontier-stage model turn before returning to the budget-exceeded state.

T-Rex validation blocked

Tool blocker: the focused Vitest durable-resume reproduction timed out because the first resumeDurableWorkflow() call did not settle under fake timers. The source run was observed reaching budget_exceeded with one frontier wrap-up and persisted wrapUpCompleted: true, but the required two-resume comparison could not complete.

Confidence Score: 4/5

Durable resume should preserve the completed budget-stop markers before this change is merged.

The restart path and restoration inputs support the reported state-loss behavior, while the focused execution confirmed that the source workflow persists the relevant completion marker. Confidence is limited because the resumed workflow did not settle under the test's fake-timer setup.

Files Needing Attention: packages/workflows/src/engine/run.ts and the durable-resume path that relaunches ended workflow runs.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex ran a focused Vitest reproduction to reproduce the budget_exceeded scenario in an engine-owned source workflow.
  • The reproduction observed budget_exceeded with wrapUpCalls=1 and budgetState.wrapUpCompleted set to true when the workflow ran.
  • The reproduction attempted to resume the durable workflow with the same ID, but the resumed run did not settle under fake timers before the 30-second timeout.
  • The test timeout prevented the first resume from producing a comparison observation, making the claimed repeat-resume behavior unverified.

View all artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix All With AI
### Issue 1
packages/workflows/src/engine/run.ts:190-195
**Durable resume loses budget state**

When an engine-owned `budget_exceeded` run is durably resumed, the resume path removes the ended store snapshot and relaunches without a continuation source, so this code restores elapsed spend but not `wrapUpCompleted` or `warned`. The already-exhausted run therefore emits its warning and invokes another frontier-stage wrap-up on each resume, consuming an additional model turn before blocking again.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "test(workflows): cover budget session re..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

flora131 and others added 30 commits August 18, 2026 10:04
Add the duration checkpoint controller consumed at stage and tool
boundaries: elapsedRunMs semantics, warn-once, one-turn wrap-up, and the
exhaustion report carrying dimension, reading, ceiling, and frontier.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only the enforcement door may produce the status; stage, tool, and model
output cannot set or clear it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Persist and restore run budget state so a resumed run never gets fresh
meters and repeat exhaustion needs no second wrap-up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Check before stage dispatch, before a tool node, and after either
completes; never mid-stream. Child scopes soft-land their own subtree
while the parent continues, and the root wins simultaneous exhaustion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Emit the warn-once lifecycle notice and report per-dimension reading,
ceiling, and percent in workflow status.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The interrupted edit left a truncated import block that failed to parse.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The existing warn-once test injected its own onWarning spy, leaving the
shipped store.recordNotice wiring in run.ts unexercised. Drive six stage
boundaries past the threshold and assert a single deduped warning notice
that does not stop the run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The shipped effective config must carry the documented 80 percent warning
threshold so definition and run budgets warn through the extension path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only an already-live stage turn may consume the one-time wrap-up allowance.
Tool, child-workflow, startup, and before-dispatch exhaustion now stop with
no new stage and no false wrap-up state; the system rechecks after the
workflow body so caught budget errors cannot clear the blocked outcome.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A workflow body cannot swallow the system-owned exhaustion stop. Resume
startup reports the actual failed or resumed stage when available, while
post-body checkpoints keep caught budget errors on the blocked rail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Describe the 80 percent default, live-turn-only wrap-ups, no-live-turn
boundary behavior, and raising a ceiling through workflow resume.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The budget warning was written only to the store notice array, which no
renderer reads, so R9 never reached a user.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ownership keyed on wrap-up delivery left a boundary-only exhaustion
reporting running forever. Key it on an engine-set marker instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A display name was persisted where a stage id belongs, so resuming a
boundary-only exhaustion was rejected as insufficient_state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add durable regressions for startup-checkpoint blocked status, the full
same-budget resume then raised-budget chain, and failedStageId identity.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Remove non-semantic whitespace and use the shared WorkflowBudget type in
the durable-resume interface. Preserve every B2 repair while recording
that the remaining source cap overage cannot be safely golfed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Teach the shared resume gate about the engine-owned budget stop instead
of relying on the extension-only escape hatch. Exercise the shipped
workflow resume action with a raised budget.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…letion

A post-body duration check was newly exhausting successful workflows and
discarding their outputs. Preserve the caught-error guarantee by
rethrowing only a system-owned stop already raised at an earlier boundary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Persisted checkpoint readings are still used for ended runs, while live
runs compute their current elapsed duration and percentage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Capture the wrap-up turn's own model usage without tree-wide metering, and
consume the one-turn allowance only when a summary is actually delivered.
A boundary abort may still own the stop without burning an undelivered
wrap-up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
flora131 and others added 20 commits August 18, 2026 16:40
Drive the agent-facing resume action through createExtensionRuntime and
assert that a raised-budget continuation is launched and completes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A blind 450ms sleep hid a two-phase settle and made the assertion depend
on an idle machine. Poll the observable end state instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The shipped config always materializes a budget object, so testing for
presence left the scan running for every unbudgeted same-id dispatch.
Test the ceiling instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A child ceiling bounds that child run's own boundaries, not its subtree;
only the root scope reaches descendants. Also restore the code spans the
reserved-status paragraph lost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment on lines +190 to +195
const continuationBudgetState =
budgetSnapshot !== undefined &&
continuedBudget?.maxDurationMs === budgetSnapshot.maxDurationMs &&
continuedBudget?.warnAtPercent === budgetSnapshot.warnAtPercent
? (opts.continuation?.source.budgetState ?? priorRun?.budgetState)
: undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Durable resume loses budget state

When an engine-owned budget_exceeded run is durably resumed, the resume path removes the ended store snapshot and relaunches without a continuation source, so this code restores elapsed spend but not wrapUpCompleted or warned. The already-exhausted run therefore emits its warning and invokes another frontier-stage wrap-up on each resume, consuming an additional model turn before blocking again.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/workflows/src/engine/run.ts
Line: 190-195

Comment:
**Durable resume loses budget state**

When an engine-owned `budget_exceeded` run is durably resumed, the resume path removes the ended store snapshot and relaunches without a continuation source, so this code restores elapsed spend but not `wrapUpCompleted` or `warned`. The already-exhausted run therefore emits its warning and invokes another frontier-stage wrap-up on each resume, consuming an additional model turn before blocking again.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant