Skip to content

fix(tools): make write/edit's implicit lex check attest itself - #132

Merged
alpibrupa merged 1 commit into
mainfrom
write-check-attestation
Sep 5, 2026
Merged

fix(tools): make write/edit's implicit lex check attest itself#132
alpibrupa merged 1 commit into
mainfrom
write-check-attestation

Conversation

@alpibrupa

Copy link
Copy Markdown
Contributor

Summary

  • write and edit already run lex fmt + lex check internally via linter.run, but that pass is a plain proc.run inside the tool's own body — invisible to lex-llm's dispatch_one_traced, which only sees separately-dispatched tool calls whose own result indicates a pass.
  • A model that leans on this implicit check-and-repair loop (common under load — a smaller model, a truncated response, a tool-count squeeze per The local-model toolset is sized by guess, not measurement #87) produces correct, type-checked code that a task spec's verified.type_check criterion still reports unmet, for reasons that have nothing to do with correctness. Reproduced live via Ship the eval harness scoped in #86 #115's eval run across three providers on the same two tasks.
  • linter.record_verified now writes a verified.type_check record directly to .lex/verified.jsonl (reusing the existing kind so no task spec needs to change) whenever a .lex file's implicit check passes, tagging tool: "write"/"edit" to keep provenance honest against the existing lex_check tool's own records.
  • mkdir -p .lex runs first via proc, not std.fs, since Tool.execute's row is fixed at [net, io, proc] (record-field effect rows unify by equality) and this call happens from inside write/edit's execute — a fresh project (no prior memory-consolidation cycle, e.g. every eval-harness worktree) has no .lex/ directory yet, and verification.append_all's plain io.write into a missing directory fails silently once its Result is discarded.

Fixes #90

Test plan

  • lex check on all three modified files, plus a full repo sweep (lex check over every tracked src/ file)
  • lex fmt --check src/
  • lex test (4/4 passing)
  • lex doc-sync --check
  • Live end-to-end verification: in a fresh git worktree with no .lex/ directory, invoked write.execute directly and confirmed .lex/verified.jsonl is created with {"kind":"verified.type_check","tool":"write","target":"verify_target.lex","ts_ms":0} — no separate lex_check tool call involved
  • Confirmed git status clean of stray worktrees/artifacts after verification

🤖 Generated with Claude Code

write and edit already run lex fmt + lex check internally via
linter.run, but that pass is a plain proc.run inside the tool's own
body — invisible to lex-llm's dispatch_one_traced, which only sees
separately-dispatched tool calls. A model that leans on this implicit
check-and-repair loop produces correct, type-checked code that a task
spec's verified.type_check criterion still reports unmet, for reasons
that have nothing to do with correctness. Reproduced live via #115's
eval run across three providers on the same two tasks.

linter.record_verified writes a verified.type_check record directly
to .lex/verified.jsonl (tool: "write"/"edit", preserving honest
provenance against the existing lex_check tool's own records) whenever
a .lex file's implicit check passes. mkdir -p .lex first via proc, not
std.fs, since Tool.execute's row is fixed at [net, io, proc] and this
runs from inside it.

Fixes #90

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alpibrupa
alpibrupa merged commit f559be4 into main Sep 5, 2026
1 check passed
@alpibrupa
alpibrupa deleted the write-check-attestation branch September 5, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Write's implicit lex check never reaches the attestation trail, so a correct fix can still fail verification

1 participant