test(agent): raise the eager tool schema ceiling to 3650 - #877
test(agent): raise the eager tool schema ceiling to 3650#877Vasanthdev2004 wants to merge 1 commit into
Conversation
main is red. TestEagerToolSchemaTokenBudget fails at 3578 tokens against a 3550 ceiling, and every open PR inherits it. Bisected: edf660a and 4ca4fa7 pass, cd0eb19 (#843, which added view_image to the core set) fails at 3588. #867 later trimmed it to 3578. #843's own CI was green, because the ceiling is only crossed once the tool set is combined, so no per-PR check could have caught it. Raised rather than paid down. The overrun is 28 tokens, and view_image costs 82, the second cheapest tool in the set against exec_command's 909 and request_permissions' 397. It is simply the one that crossed the line. The alternative was trimming a description to reclaim 28 tokens, degrading a tool's usability to satisfy a line drawn against a 2026-07 measurement that predates two tools. The ratchet worked exactly as intended: it caught the creep and forced a decision instead of a drift. Also corrects the failure message. It advised "defer a tool", but this test pins DeferThreshold at 0 so deferral is inactive and marking a tool deferred leaves it exposed here, changing nothing. Deferring is still worth doing for real sessions; it just cannot move this number. The levers that can are a smaller schema, one fewer core tool, or a deliberate raise, and the message now says so.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThe prompt budget test raises the eager tool schema token ceiling from 3550 to 3650. It documents the measured count and updates failure guidance to identify schema reduction, core tool removal, or deliberate ceiling increases as remediation options. ChangesPrompt budget validation
Estimated code review effort: 1 (Trivial) | ~5 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Zero automated PR reviewVerdict: No blockers found Blockers
Validation
ScopeHead: This deterministic review checks validation status and basic diff hygiene. A human reviewer still owns product judgment and design quality. |
|
@anandh8x @jatmn this one is small and it unblocks everyone, so it is worth jumping the queue.
This raises the ceiling to 3650 with the reasoning written into the constant. One file, test-only. Two things worth your eyes rather than a rubber stamp: The choice to raise instead of trim. The overrun is 28 tokens and I also corrected the failure message. It advised "defer a tool", which cannot work: the test pins Separately, and not part of this PR: CI reported success on |
mainis red and every open PR inherits it.TestEagerToolSchemaTokenBudgetfails at 3578 tokens against a 3550 ceiling.Where it came from
Bisected:
edf660aband4ca4fa75pass,cd0eb194(#843, which addedview_imageto the core set) fails at 3588. #867 later trimmed it to 3578, still over.#843's own CI was green. The ceiling is only crossed once the tool set is combined, so no per-PR check could have caught it, and I merged it on the strength of "CLEAN, no failing checks". Worth knowing that is a blind spot for anything measuring a whole-set budget.
Raised rather than paid down
The overrun is 28 tokens. Per-tool cost in the eager set:
view_imageis the second cheapest tool in the set. It is not the problem, it is just what crossed the line. The alternative was trimming a description to reclaim 28 tokens, degrading a tool's usability to satisfy a line drawn against a 2026-07 measurement that predates two tools since added.The ratchet did its job: it caught the creep and forced this to be a decision rather than a drift. Keeping it that way means raising it deliberately, with the reason written down, which is what this does.
The failure message was wrong
It advised "defer a tool". This test pins
DeferThresholdat 0, so deferral is inactive and marking a tool deferred leaves it exposed here, changing nothing. I confirmed that by actually deferringview_image: the count stayed at 3578 across 14 tools, and it brokeregistry_test.gobesides, since the deferrable-builtin list is curated.Deferring is still worth doing for real sessions, it just cannot move this number. The message now names the levers that can: a smaller schema, one fewer core tool, or a deliberate raise.
Validation
go build ./...,go vet,gofmt, and the fullinternal/agentandinternal/toolssuites pass. One file changed.Summary by CodeRabbit