fix(tools): make write/edit's implicit lex check attest itself - #132
Merged
Conversation
write and edit already run lex fmt + lex check internally via linter.run, but that pass is a plain proc.run inside the tool's own body — invisible to lex-llm's dispatch_one_traced, which only sees separately-dispatched tool calls. A model that leans on this implicit check-and-repair loop produces correct, type-checked code that a task spec's verified.type_check criterion still reports unmet, for reasons that have nothing to do with correctness. Reproduced live via #115's eval run across three providers on the same two tasks. linter.record_verified writes a verified.type_check record directly to .lex/verified.jsonl (tool: "write"/"edit", preserving honest provenance against the existing lex_check tool's own records) whenever a .lex file's implicit check passes. mkdir -p .lex first via proc, not std.fs, since Tool.execute's row is fixed at [net, io, proc] and this runs from inside it. Fixes #90 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
writeandeditalready runlex fmt+lex checkinternally vialinter.run, but that pass is a plainproc.runinside the tool's own body — invisible to lex-llm'sdispatch_one_traced, which only sees separately-dispatched tool calls whose own result indicates a pass.verified.type_checkcriterion still reports unmet, for reasons that have nothing to do with correctness. Reproduced live via Ship the eval harness scoped in #86 #115's eval run across three providers on the same two tasks.linter.record_verifiednow writes averified.type_checkrecord directly to.lex/verified.jsonl(reusing the existing kind so no task spec needs to change) whenever a.lexfile's implicit check passes, taggingtool: "write"/"edit"to keep provenance honest against the existinglex_checktool's own records.mkdir -p .lexruns first viaproc, notstd.fs, sinceTool.execute's row is fixed at[net, io, proc](record-field effect rows unify by equality) and this call happens from insidewrite/edit'sexecute— a fresh project (no prior memory-consolidation cycle, e.g. every eval-harness worktree) has no.lex/directory yet, andverification.append_all's plainio.writeinto a missing directory fails silently once itsResultis discarded.Fixes #90
Test plan
lex checkon all three modified files, plus a full repo sweep (lex checkover every trackedsrc/file)lex fmt --check src/lex test(4/4 passing)lex doc-sync --check.lex/directory, invokedwrite.executedirectly and confirmed.lex/verified.jsonlis created with{"kind":"verified.type_check","tool":"write","target":"verify_target.lex","ts_ms":0}— no separatelex_checktool call involvedgit statusclean of stray worktrees/artifacts after verification🤖 Generated with Claude Code