fix(coding-agent): explain eval language selection (fixes #1395) - #1396
fix(coding-agent): explain eval language selection (fixes #1395)#1396MoerAI wants to merge 1 commit into
Conversation
…u#1395) Describe the run-only language requirement without choosing a default kernel. Report unsupported values separately from omissions. Verify the regression red-to-green, 35 related tests, full checks and build, and an isolated real CLI eval turn.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 380046ceb3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const LANGUAGE_FIELD_DESCRIPTION = | ||
| "REQUIRED for run. Choose a kernel explicitly; there is no default. Omit for peek/stop."; |
There was a problem hiding this comment.
Document the language requirement in the README
This introduces a user-visible contract stating that every run must select a language and that no default kernel exists, but packages/senpi-codemode/README.md still only lists available kernels and never documents that requirement. Users and extension integrators relying on the shipped README therefore remain exposed to the ambiguity this change is intended to resolve; update the README alongside the schema.
AGENTS.md reference: packages/senpi-codemode/AGENTS.md:L76-L77
Useful? React with 👍 / 👎.
Summary
Fixes #1395.
Explain that eval runs require an explicit
language, with no default kernel, whilepeekandstopdo not need it. Apply the description to both schema definitions and distinguish missing languages from unsupported values in request parsing.Root cause
languagemust remain optional at the schema level because the same tool handles control requests. Unlikesummary, it had no description explaining its run-only requirement. The parser also reported every unsupported value as an omission.Verification
From
packages/senpi-codemode:eval run requires language.npm run checkandnpm run build: passed.node .agents/skills/senpi-qa/scripts/rpc-drive.mjs --self-test: passed.An isolated source CLI with a scripted local provider made three eval calls: omitted language, invalid
python, then validjswithreturn 1395. The captured wire schema matched the source description; the first two calls were rejected distinctly, the real JavaScript kernel returned1395, and the final model turn completed. Real auth was unchanged and no paid provider was contacted.Existing QA failure
The broader
mock-loop.mjs --self-testpasses the loopback OpenAI Completions, Anthropic Messages, and OpenAI Responses turns, but its text-tool-leak recovery cases fail. The same complete-leak failure was reproduced in a clean, independently built upstream4df67dc87checkout with:Those unrelated recovery checks are not changed here.
Summary by cubic
Explains that eval runs require an explicit
languagewith no default kernel, so callers no longer get a generic error when it’s missing or unsupported.languageoptional in the shared schema since the same tool servespeekandstop, and documents the run-only requirement in both live and exported schemas.Written for commit 380046c. Summary will update on new commits.