eval(harbor): forward stored API keys into the task container - #756
Merged
Conversation
WebSearch resolves TAVILY_API_KEY through get_secret(), which reads the "env" block of the global config (src/tool_system/tools/web_search.py:124). The adapter seeded only settings.effort into the container's ~/.clawcodex/config.json, so that block was absent and every WebSearch call failed with "Web search is not configured" -- on terminal-bench 2.1 tasks that need a lookup, the agent simply could not do it. _seed_container_settings now also writes the host config's "env" block. Values travel through the exec ENV, never the command string, so they stay out of process listings and Harbor's trial logs. Opt out with --ak forward_keys=false. This is a capability, not guidance. The distinction matters for the comparison the eval exists to make: instructions given to one agent and not the other (the deadline injection removed in #751) skew trajectories and would not exist on a scored run. Key material does not -- Harbor's built-in claude-code agent likewise runs with its own credentials available, and a harness that cannot reach a tool it ships is measuring the wrong thing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WebSearch resolves
TAVILY_API_KEYthroughget_secret(), which reads theenvblock of the global config (src/tool_system/tools/web_search.py:124). The adapter seeded onlysettings.effortinto the container's~/.clawcodex/config.json, so that block was absent and every WebSearch call failed with "Web search is not configured" — on terminal-bench 2.1 tasks needing a lookup, the agent could not do it at all._seed_container_settingsnow also writes the host config'senvblock. Values travel through the exec ENV, never the command string, so they stay out of process listings and Harbor's trial logs. Opt out with--ak forward_keys=false.This is a capability, not guidance — a distinction that matters for the comparison the eval exists to make. Instructions given to one agent and not the other (the deadline injection removed in #751) skew trajectories and wouldn't exist on a scored run. Key material doesn't: Harbor's built-in claude-code agent likewise runs with its own credentials available, and a harness that can't reach a tool it ships is measuring the wrong thing.
Verified: reads the host key,
forward_keys=falseopts out, both adapters import under Harbor's interpreter.🤖 Generated with Claude Code