Skip to content

feat: describe the sandbox to the model in a tagged <code-mode> block - #14

Merged
benclarkeio merged 1 commit into
mainfrom
feat/code-mode-metadata-block
Jul 23, 2026
Merged

feat: describe the sandbox to the model in a tagged <code-mode> block#14
benclarkeio merged 1 commit into
mainfrom
feat/code-mode-metadata-block

Conversation

@benclarkeio

@benclarkeio benclarkeio commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Why

The model was never told which Python version or third-party packages the sandbox image has. Worse, execute_code's description actively claimed "The Python Standard Library and a set of custom tools are available" — so a model with pandas baked into the image would still hand-roll CSV parsing.

The README already documents RUN pip install pandas as the supported way to extend the image; this closes the loop to the prompt.

How

The sandbox reports its own environment on connect, so anything baked into the image is advertised with no host-side configuration.

Wire protocol

ReadyFrame gains python_version and packages, populated by a new ready_frame() helper used by both the TCP and WebSocket transports. packages maps each top-level import name to the distribution or distributions that provide it and their versions, so names such as yaml / PyYAML are unambiguous.

These are additive fields with defaults, so no PROTOCOL_VERSION bump: decode already drops unknown keys, and a sandbox predating the fields just leaves them empty. Mixed host/sandbox pins keep working.

Caching

adk_code_mode.metadata caches each sandbox's reported environment keyed by backend identity (RemoteBackend.url / UnsafeLocalDockerBackend.image). Module-level rather than on ExecuteCodeTool, so one warm process serves every agent pointing at the same image and the value survives a tool being rebuilt.

The report only arrives once a container has booted, so the block changes exactly once per process — at the second model call of the first turn that runs code. Recording is monotonic, while import names and their providing distributions are both sorted before rendering so the block stays byte-stable from then on. This keeps the provider's cached prompt prefix intact.

Block structure

<code-mode>
  <how-to-use>
This section describes the sandbox that `execute_code` runs in: …
  </how-to-use>
  <python-version>3.13.2</python-version>
  <installed-packages>
pandas: pandas 2.3.3
yaml: PyYAML 6.0.2
  </installed-packages>
  <tools-package>
# tools.slack

from tools.slack import list_channels, send_message
…
  </tools-package>
</code-mode>

Tags are indented, their content is not — the <tools-package> body is Python source, where leading whitespace is meaningful. Unknown or empty tags are omitted rather than rendered blank, since an empty tag reads as "none exist".

Tiering

The all-or-nothing size gate is gone. A block over budget now degrades:

  1. Full catalog
  2. Import lines only (render_catalog(detail="names"))
  3. A pointer to /tools/unconditional, so an oversized tool surface no longer costs the model the Python version and package list

Two surfaces, kept apart

The tool description is now the invariant contract of calling execute_code; the block is the per-deployment inventory. Its discovery pointer is swapped at construction so it never references a block that wasn't appended.

API renames

Before After
append_function_stubs_to_system_instruction append_code_mode_metadata_to_system_instruction
max_catalog_chars max_code_mode_metadata_chars (now budgets the whole block, tags included)

Testing

  • tests/test_metadata.py: cache isolation per identity, monotonic recording, deterministic rendering regardless of input order, omitted empty/unknown tags, all three tiers, exact-budget boundary, indentation.
  • tests/test_sandbox_entry.py: top-level import mapping, multiple providers, deterministic provider ordering, runtime-package filtering.
  • tests/test_docker_integration.py: the Python version reaches the block only after a real container boots.
  • make ci green locally: 148 passed, Ruff and mypy clean.

🤖 Generated with Claude Code

The sandbox now reports its Python environment after boot so models can use packages baked into the image without host-side configuration.

- Add Python version and deterministic import-to-distribution version mappings to `ReadyFrame` for both TCP and WebSocket transports.

- Cache reported environments by backend identity and render sorted `<python-version>` and `<installed-packages>` metadata.

- Add full, names-only, and discovery-pointer catalog tiers so oversized tool surfaces retain environment metadata.

- Rename the metadata-related `ExecuteCodeTool` options and keep the tool description aligned with the configured prompt surface.

- Document the tagged block and add coverage for protocol compatibility, cache isolation, deterministic rendering, and tier boundaries.
@benclarkeio
benclarkeio force-pushed the feat/code-mode-metadata-block branch from 9beab96 to 0978036 Compare July 23, 2026 23:04
@benclarkeio benclarkeio changed the title feat!: describe the sandbox to the model in a tagged <code-mode> block feat: describe the sandbox to the model in a tagged <code-mode> block Jul 23, 2026
@benclarkeio
benclarkeio merged commit 662a281 into main Jul 23, 2026
6 checks passed
@benclarkeio
benclarkeio deleted the feat/code-mode-metadata-block branch July 23, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant