Skip to content

fix(tokenless): anchor rtk prefix in Hermes and OpenClaw rewrite paths - #2248

Closed
Forrest-ly wants to merge 1 commit into
alibaba:mainfrom
Forrest-ly:fix/tokenless-anchor-hermes-openclaw-rewrite
Closed

fix(tokenless): anchor rtk prefix in Hermes and OpenClaw rewrite paths#2248
Forrest-ly wants to merge 1 commit into
alibaba:mainfrom
Forrest-ly:fix/tokenless-anchor-hermes-openclaw-rewrite

Conversation

@Forrest-ly

Copy link
Copy Markdown
Collaborator

Description

Follow-up to #1975. That PR fixed the bare rtk prefix in common/hooks/rewrite_hook.py but two adapters shipped independent rewrite paths with the same latent defect:

  • Hermes (adapters/tokenless/hermes/__init__.py, _try_rewrite): returned proc.stdout.strip() verbatim without anchoring.
  • OpenClaw (adapters/tokenless/openclaw/index.ts, tryRtkRewrite): returned result.stdout?.trim() verbatim without anchoring.

In agent runtimes whose tool-shell PATH omits the rtk install location, the rewritten commands fail with exit 127 despite the hook resolving rtk successfully.

Changes

  • common/hooks/hook_utils.py: Extract anchor_rtk_prefix (with _SEGMENT_OPS, _is_env_assignment) from rewrite_hook.py into the shared utilities module so all Python adapters share one implementation.
  • common/hooks/rewrite_hook.py: Import and delegate to hook_utils.anchor_rtk_prefix; remove the now-redundant local copy.
  • hermes/__init__.py: Import anchor_rtk_prefix from hook_utils; call it on the rewritten command in _try_rewrite before returning the block message.
  • openclaw/index.ts: Port the token-level anchor as anchorRtkPrefix (same posix=False-style lexer: preserves quotes/globs verbatim, disables comment stripping); call it from tryRtkRewrite.
  • openclaw/test_anchor_rtk_prefix.mjs: 15 node:test unit tests covering wrapper (sudo rtk), env assignments, single &, quoted patterns, unquoted globs, fd redirections (2>&1, 2>/dev/null), command substitutions, path-with-spaces quoting, and double-anchor prevention.

Testing

# Python adapter tests (29 passed, 15 skipped)
cd src/tokenless
python3 -m pytest tests/test_rewrite_hook.py tests/test_hermes_plugin_import.py tests/test_resolve_agent_id.py -v

# OpenClaw anchor unit tests (15 passed)
node src/tokenless/adapters/tokenless/openclaw/test_anchor_rtk_prefix.mjs

All tests pass. The fix is identical in behaviour to #1975: anchoring is conservative (never corrupts unrecognised patterns), and unparseable input is returned untouched.

Related

Closes #2123

Both adapters returned bare `rtk` wrappers verbatim from `rtk rewrite`,
coupling the rewritten command to the agent tool-shell's PATH. On IDE
runtimes whose PATH omits the rtk install location the commands fail
with exit 127 despite a successful hook resolve.

Extract anchor_rtk_prefix from rewrite_hook.py into hook_utils.py so
Hermes and the shared hook share one implementation. Wire Hermes
__init__._try_rewrite to call it before returning the block message.
Port the token-level anchor to openclaw/index.ts as anchorRtkPrefix
(same posix=False-style lexer: preserves quotes/globs, disables comment
stripping) and call it from tryRtkRewrite. Add 15 node:test unit tests
covering wrapper (sudo), env assignments, single &, quoted patterns,
unquoted globs, fd redirections, and command substitutions.

Assisted-by: Qoder
Signed-off-by: 林生 <linyan.lin@alibaba-inc.com>
@Forrest-ly
Forrest-ly requested a review from ikunkun-sys as a code owner August 6, 2026 01:05
@github-actions github-actions Bot added the component:tokenless src/tokenless/ label Aug 6, 2026

@qoderai qoderai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. [P2] openclaw 测试文件中内联复制 anchorRtkPrefix 实现,后续易与生产实现漂移,降低回归有效性。
  2. [P2] openclaw TypeScript 版 anchorRtkPrefix 的 shell 特殊字符匹配集合需与 Python 版保持长期一致,避免微妙行为差异。

🤖 Generated by QoderView workflow run

return /^[A-Za-z0-9_]+$/.test(name);
}

function anchorRtkPrefix(rewritten, resolvedRtkPath) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] anchorRtkPrefix 测试实现重复

openclaw 的单元测试文件中内联复制了一份与 index.ts 中几乎相同的 anchorRtkPrefix 实现(测试文件 15-63 行),后续如主实现修订而遗漏同步,测试将与生产逻辑发生偏差,降低回归价值。建议改为通过纯函数导出+依赖注入方式引入,或在测试中只覆盖输入/输出用例并减少重复实现,以降低漂移风险。


🤖 Generated by QoderFix in Qoder

i = j;
}

// Shell-quote the resolved binary path only if it contains special chars

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] anchorRtkPrefix shell quoting 行为与 Python 版本略有差异

openclaw/index.ts:231-234 的 shell 特殊字符判定与 hook_utils.anchor_rtk_prefix 的 shlex.quote 行为略有差异,例如 $!# 等字符的匹配集合需要长期保持语义一致,否则在跨语言迁移时可能出现微妙行为差异。建议在 Python/TS 两侧各自 docstring 中清晰列出受支持字符集合,并在后续调整时同时更新两端实现和回归用例。


🤖 Generated by QoderFix in Qoder

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a32af30b8a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

// Unquoted token — collect until whitespace
let j = i;
while (j < rewritten.length && rewritten[j] !== " " && rewritten[j] !== "\t") {
if (rewritten[j] === "'" || rewritten[j] === '"') break; // switch to quoted

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve inline quoted tokens when anchoring rtk

In the OpenClaw rewrite path, commands that contain quotes inside an otherwise unquoted shell word are now split at the quote and later rejoined with an extra space. For example an RTK rewrite such as rtk grep --include="*.rs" foo becomes /path/to/rtk grep --include= "*.rs" foo, and FOO="bar baz" rtk cargo test becomes FOO= "bar baz" /path/to/rtk cargo test, changing argv/env semantics before the agent re-executes the suggested command. The lexer needs to keep embedded quoted spans as part of the same token instead of breaking out here.

Useful? React with 👍 / 👎.

@Forrest-ly

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #2249 which covers the same fix with a cleaner branch name. Please track progress on #2249.

@Forrest-ly Forrest-ly closed this Aug 6, 2026
@Forrest-ly
Forrest-ly deleted the fix/tokenless-anchor-hermes-openclaw-rewrite branch August 6, 2026 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:tokenless src/tokenless/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hermes / OpenClaw rewrite paths emit bare rtk prefix (same PATH-coupling as #1974)

1 participant