Skip to content

feat(mcp): detect and withhold a tool-call-free reasoning-loop reply - #131

Merged
alpibrupa merged 1 commit into
mainfrom
reasoning-loop-watchdog
Sep 5, 2026
Merged

feat(mcp): detect and withhold a tool-call-free reasoning-loop reply#131
alpibrupa merged 1 commit into
mainfrom
reasoning-loop-watchdog

Conversation

@alpibrupa

Copy link
Copy Markdown
Contributor

Summary

  • A live run today (bar mode via a "thinking" model, OpenCode's `kimi-k2.7-code`) burned its whole step budget on a ~36,000-character reply that was pure self-narration with zero real tool calls. `bridge.outcome_of_steps` returned that text as a normal, completed reply — a caller had no signal the run had degenerated.
  • Scoped to lex-code rather than lex-llm: true mid-generation cancellation would need lex-llm to expose an abort, and is moot for MCP anyway — `make_handler` already calls `iter.to_list` on the whole `run_loop` before any `Step` is visible here, so the full (wasted) generation has already happened by the time detection could run.
  • `is_degenerate_ramble(steps)` flags a turn with zero `StepToolExec` steps whose final text exceeds 8,000 characters. `make_handler` now checks this before handing off to `bridge.outcome_of_steps`, replacing a flagged reply with an explicit error naming the character count and pointing at `LEX_CODE_PROVIDER_` (Per-mode provider defaults / escalation for reliability-sensitive modes #116) as the fix.

Fixes #114

Test plan

  • `lex check`/`lex fmt --check` — ok
  • Full repo sweep — all pass
  • `lex test` — 4/4 passed
  • `lex doc-sync --check` — current
  • Constructed-case verification covering the three cases that matter: a long zero-tool-call reply (flagged), a short zero-tool-call reply (not flagged — avoids a false positive on legitimate short chat answers), a long reply that did call a tool (not flagged — tool use is itself evidence of real work).
  • Live sanity check: a normal short MCP request end to end still passes through unchanged, confirming no regression on ordinary replies.

🤖 Generated with Claude Code

…114)

A live run today (bar mode via a "thinking" model, opencode's
kimi-k2.7-code) burned its whole step budget on a ~36,000-character
reply that was pure self-narration ("OK. Let's go. Now. Done. I'll
output now...") with zero real tool calls. bridge.outcome_of_steps
returned that text as a normal, completed reply — a caller had no
signal the run had degenerated rather than succeeded.

Scoped this to lex-code rather than lex-llm: true mid-generation
cancellation would need lex-llm to expose an abort, and would be moot
for MCP specifically anyway — make_handler already calls iter.to_list
on the whole run_loop before any Step is visible here, so the full
(wasted) generation has already happened by the time detection could
run. What's fully in scope on lex-code's side: never hand the raw
ramble back as if it were a real answer.

is_degenerate_ramble(steps) flags a turn with zero StepToolExec steps
whose final StepDone text exceeds 8,000 characters — picked well below
today's 36,000-character failure and comfortably above the longest
genuine final answers seen live this session (a few thousand
characters). make_handler now checks this before handing off to
bridge.outcome_of_steps, replacing a flagged reply with an explicit
"[degenerate response detected...]" message naming the character count
and pointing at LEX_CODE_PROVIDER_<MODE> as the fix.

Verified with constructed Step lists covering the three cases that
matter: a long zero-tool-call reply (flagged), a short zero-tool-call
reply — e.g. plain chat with no work to do — (not flagged, avoiding a
false positive on legitimate short answers), and a long reply that DID
call a tool (not flagged, since tool use is itself evidence of real
work regardless of final text length). Also live-tested a normal short
MCP request end to end afterward to confirm an ordinary reply still
passes through unchanged.

Fixes #114

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alpibrupa
alpibrupa merged commit 3fc5e2a into main Sep 5, 2026
1 check passed
@alpibrupa
alpibrupa deleted the reasoning-loop-watchdog branch September 5, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Detect and abort a tool-call-free reasoning loop

1 participant