feat: add Grok Build harness adapter

[alexzhang1030]

Summary

Adds reach grok so Grok Build (verified 1.0.5) runs shell tools on the session target instead of the local machine.

Grok does not walk PATH for bash. It spawns $SHELL by absolute path and wraps each run_terminal_command in a __grok_user_cmd envelope (-O extglob -c '<envelope>' -- <command>). Login-environment snapshots ($SHELL -lc) stay local; the shim unwraps the payload after -- and runs only that command on the target. Native file tools (read_file, search_replace, list_dir, grep) still hit the local disk, so they are denied at launch and subagents are disabled.

Test plan

• go test ./cmd/reach/ ./internal/harnessprobe/ (envelope unwrap + snapshot classification)
• reach grok against a live session (reach up then a remote hostname)
• reach harness verify grok once a session target hostname differs from local
• make conformance on a machine with grok installed (SHELL-wrapper probe)

[bojieli]

Thanks for this — I installed grok 1.0.5 and verified the whole adapter against a live target (an Alpine container via docker://, hostname REMOTEBOX, so local-vs-remote is unambiguous). The seam reverse-engineering is correct and holds up under measurement. Two things block it landing as-is.

Confirmed working

• $SHELL is the seam. Four configurations tested: SHELL=<shim>/bash intercepts; GROK_SHELL alone does not; PATH-prepend alone does not. cmdGrok sets SHELL, so the mechanism is right. (Grok appears to check the basename — a wrapper not named bash was ignored.)
• The envelope is exactly as documented: bash -O extglob -c '<envelope>' -- <command>, __grok_user_cmd="$1", fd-3 snapshot, plus the two $SHELL -lc bashrc snapshots that must stay local. unwrapGrokEnvelope, isGrokLocalSnapshot and shellCommandArg all handle the real captured argv correctly.
• End to end: reach grok → hostname → REMOTEBOX.

Blocking 1 — the --deny rules also gate the shell

Grok maps shell commands that touch files onto the same permission prefixes as the native tools:

rule	blocks native tool	also blocks in the shell
--deny Read	read_file, list_dir	cat
--deny Write	write	echo > file
--deny Edit	search_replace	sed -i
--deny Grep	grep	—

So the adapter denies the file tools, tells the model via grokExecGuidance to use the shell instead (read cat -- FILE, write cat > FILE <<'EOF', edit sed -i), and then the same rules block exactly those. Measured through reach grok against the container: cat /work/remote_marker.txt came back "denied by the permission policy". The adapter can run hostname, but cannot read, write or edit anything on the target by any route.

A mechanism that does work in the TUI is an agent profile — --agent <file.md> with disallowedTools:. Verified: it removes read_file, search_replace, list_dir and grep while leaving shell cat working. Worth noting it also catches write, a local-filesystem tool that is in grok's live tool list and in none of its documentation.

(Also: --agent-profile from the docs does not exist in the 1.0.5 binary; the flag is --agent.)

Blocking 2 — reach harness verify grok can never pass

Three separate bugs. Because cmdGrok calls guardHarnessSeam, this means reach grok does not launch at all without -force:

1. grokPrepare writes base_url = "http://127.0.0.1/unused". A model's own base_url overrides GROK_CLI_CHAT_PROXY_BASE_URL completely — confirmed with two logging HTTP servers on separate ports, where only base_url was ever dialled. The probe therefore talks to nothing and times out after 2 minutes. Omitting base_url lets the model resolve through the cli-chat-proxy, which the env var points at the mock.
2. The mock's chat-dialect tool call omits description, which grok's run_terminal_command requires: Failed to parse arguments for tool run_terminal_command: missing field 'description'. pickChatTool needs a per-tool argument shape rather than one generic {"command": ...}.
3. baseProbeEnv never pins REACH_HOME. grokEnv replaces HOME, so reach's shim looks for the session store under the throwaway home, finds no session, and the seam gets reported as broken when it is fine. geminiEnv has the identical shape, so this one predates this PR and likely affects the gemini probe too.

With those three fixed the probe reaches verdict: ok — tool output reports hostname "REMOTEBOX", the target.

Smaller points

• docs/harnesses/grok.md says the shim directory is prepended to PATH "as a fallback" — PATH alone is never consulted, so it is not a fallback.
• README and ARCHITECTURE.md list Grok as verified. The seam is verified; the adapter as a whole is not usable until the deny-rule issue is resolved.
• The probe passes --tools run_terminal_command, but grok's documented filter ID is run_terminal_cmd. It appeared to work, but an unknown name may simply no-op the allowlist — worth confirming, since a silently-ignored allowlist is not the isolation the probe intends.

Next

I have pushed a merge commit resolving the CHANGELOG.md conflict (both sides added under [Unreleased] / Added; both entries kept), so the branch is mergeable again. I will follow up with the three probe fixes and the agent-profile change, and re-run the end-to-end test.

[bojieli]

Merged — thank you, @alexzhang1030. This is a genuinely good piece of reverse engineering, and I want to be specific about what held up, because it was the hard part.

Every claim you made about the seam was correct when measured against a live grok 1.0.5:

• $SHELL is the seam, and nothing else is. I tested four configurations: SHELL pointing at the shim intercepts, GROK_SHELL alone does not, and a shim reachable only through PATH is never consulted. Your adapter sets SHELL, so it works. (One detail worth adding to the notes: grok checks the basename — a wrapper not named bash gets ignored in favour of a system shell, which is why the shim's name is load-bearing.)
• The envelope is exactly as you documented it — bash -O extglob -c '<envelope>' -- <command>, __grok_user_cmd="$1", snapshot on fd 3 — and both -lc bashrc snapshots show up just as described. unwrapGrokEnvelope, isGrokLocalSnapshot and shellCommandArg all handle the real captured argv correctly, first try.

Working out that shape from a stripped binary is not easy, and getting it right to the argument is why the rest was a matter of fixing rather than rethinking.

Two things needed changing before it could ship, both only findable by running it:

The --deny rules also gated the shell. Grok classifies a shell command that reads a file under the same permission prefix as read_file, so --deny Read denies cat, and Write/Edit deny cat > file and sed -i. Those are the commands grokExecGuidance tells the model to use once the native tools are gone, so the adapter could run hostname and essentially nothing else. The tools are now removed with a generated agent profile's disallowedTools, which takes them out of the model's view without teaching the permission layer anything about shell commands. That list also includes write — grok advertises it in a live session and documents it nowhere, so a list built from the docs leaves the agent a local file writer.

The seam probe could not reach a verdict, which meant reach grok refused to launch without -force. Three causes: the probe's config.toml pinned the model's base_url, which overrides GROK_CLI_CHAT_PROXY_BASE_URL outright, so it dialled a dead address; the mock sent {"command": ...} and grok's run_terminal_command rejects a call with no description; and a probe that replaces HOME left the shim hunting for the session store under the throwaway home. That last one was latent in the gemini probe too, so your PR ended up fixing a bug it did not cause.

Verified end to end against a container target and a real ssh host: the probe returns the target's hostname, and a session reads, creates and edits files on the target with nothing written locally and a host-only path correctly absent from inside the session.

One thing deliberately left open, recorded in docs/harnesses/grok.md: the profile's disallowedTools is measured in headless mode, not in the interactive TUI. The evidence is good — grok marks --tools/--disallowed-tools/--max-turns as headless-only and --agent carries no such note — but it is inference. It matters because that failure is silent and looks like success. If you drive it in the TUI, asking the model to use the read_file tool to read /etc/hosts settles it; "no such tool" is the expected answer.

Separately, your PR surfaced a bug in reach itself that had nothing to do with grok: every harness launcher reported "not installed or not in PATH" for a binary found through a relative PATH entry, because exec.LookPath returns those with exec.ErrDot. That is #20.

Thanks again — the seam research is the part that could not have been guessed, and it was right.