Skip to content

fix: error on ipython calls missing 'code' instead of silent no-op - #129

Open
hallerite wants to merge 1 commit into
mainfrom
fix/ipython-silent-noop-args
Open

fix: error on ipython calls missing 'code' instead of silent no-op#129
hallerite wants to merge 1 commit into
mainfrom
fix/ipython-silent-noop-args

Conversation

@hallerite

@hallerite hallerite commented Aug 16, 2026

Copy link
Copy Markdown
Member

Problem

IpythonTool.execute reads args.get("code", ""), so a tool call that puts the code under any other key executes nothing, returns an empty result, and raises no error. The model then hits NameErrors on variables it believes it created, and burns turns rediscovering state.

Observed routinely during large-scale eval: a model emits the code under cmd (a {'cmd': ...} cell), and the silent no-op appeared across many rollouts. Under a turn budget it injects failure that lands stochastically later than the malformed call, so it corrupts credit assignment rather than being locally learnable.

Fix

If code is absent or empty, return a loud error naming the wrong keys ("got keys: 'cmd'. Pass your Python under the 'code' key.") instead of executing an empty string. Now a learnable signal — the model/policy can correct on the next turn, like it already does for the invalid-JSON-args error path.

Tests

test_ipython_missing_code_key_errors_not_noop, test_ipython_empty_code_errors. Full suite 114 passed.

See also the depth-limit and skill-preload issues filed separately.

🤖 Generated with Claude Code


Note

Low Risk
Localized validation in the ipython tool with clearer errors; no auth, data, or execution-path changes beyond skipping empty runs.

Overview
Fixes silent IPython no-ops when models pass Python under the wrong argument (e.g. Laguna’s cmd instead of code). Previously execute treated a missing or empty code as "", ran nothing, and returned an empty success—leading to later NameErrors and poor RL credit assignment.

IpythonTool.execute now fails fast with an explicit error that lists the unexpected keys and tells the caller to use code. Metrics still record an IpythonExecuted event with zero input size on this path.

Adds test_ipython_missing_code_key_errors_not_noop and test_ipython_empty_code_errors plus a small _ipython_ctx() helper for REPL-free error-path tests.

Reviewed by Cursor Bugbot for commit 8f865f8. Bugbot is set up for automated code reviews on this repo. Configure here.

…lent no-op

The ipython tool read args.get("code", ""), so a call that put the code under
any other key executed nothing and returned an empty result with no error — the
model then hit NameErrors on state it believed it had created. Laguna-S-2.1
does this routinely, emitting the code under 'cmd'. Return a loud, learnable
error naming the wrong keys instead, so the model (or an RL policy) can correct.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant