Skip to content

feat(ai): flip tool-argument validation to native#357

Merged
zmaril merged 1 commit into
mainfrom
flip/ai-validation
Jul 22, 2026
Merged

feat(ai): flip tool-argument validation to native#357
zmaril merged 1 commit into
mainfrom
flip/ai-validation

Conversation

@zmaril

@zmaril zmaril commented Jul 21, 2026

Copy link
Copy Markdown
Owner

What

Flips pi's packages/ai/src/utils/validation.ts (tool-argument validation +
JSON-Schema coercion) to the native Rust engine. validateToolArguments and
validateToolCall now run in crates/pidgin-ai (utils/validation.rs);
packages/ai/test/validation.test.ts passes fully native.

Before / After

Before: validation.ts ran pi's own TypeScript — TypeBox Value.Convert +
the hand-rolled coerceWithJsonSchema + compiled Value.Check. Manifest row
status: "original". Native module count: 52.

After: a native shim overlays the module and delegates to the addon. Every
decision runs in Rust: the AJV-compatible primitive-coercion rules
(coercePrimitiveByType), the recursive allOf/anyOf/oneOf + object + array
coercion driver, the pass/fail check over the JSON-Schema subset, and the
Validation failed for tool "<name>": … throw-envelope. Manifest row
status: "native", tests: ["test/validation.test.ts"]. Native module count: 53.

How

  • crates/pidgin-napi/src/ai_validation.rs (new): exposes both functions as
    free #[napi] fns. Tool / ToolCall cross the boundary as JSON strings —
    the tool's parameters is a TypeBox TSchema, and JSON.stringify drops its
    hidden [TypeBox.Kind] symbol, leaving exactly the JSON-Schema projection the
    port models as serde_json::Value. Each fn returns a discriminated outcome
    envelope: {ok:true,value} on success, {ok:false,error} carrying pi's thrown
    message verbatim.
  • crates/pidgin-napi/src/lib.rs: registers pub mod ai_validation; (pub
    for the free-fn crate-reachability rule).
  • conformance/shims/packages/ai/src/utils/validation.ts (new): re-exports
    the preserved pi original, overrides both fns to JSON.stringify their inputs,
    delegate, return the coerced value, and re-throw new Error(envelope) on
    failure so pi's .toThrow("Validation failed") / Tool "<name>" not found
    contract holds. .toEqual deep-equality means the freshly parsed return is fine.
  • conformance/manifest.json: the one validation.ts row flipped to native.

The symbol-drop is behaviour-preserving

pi's coercion path has two branches gated on [TypeBox.Kind]: TypeBox schemas
run only Value.Convert, plain serialized schemas additionally run the
hand-rolled coercion. Rust Values carry no Kind, so the port always runs the
hand-rolled coercion — and for the shapes these utils see it reproduces the same
observable result Value.Convert would have produced (e.g. {count:"42"} ->
{count:42} for a Type.Number() field). The tests assert only plain coerced
values (.toEqual) and, on failure, only the substring "Validation failed"
(pi's own envelope prefix) — never TypeBox's per-error message text — so the
reimplementation satisfies the full test contract.

Verification

  • test/validation.test.ts: 3/3 pass, native (all cases).
  • Full packages/ai suite: zero new regressions — 553 passed / 3 failed /
    740 skipped; the 3 failures are pre-existing model-catalog phantoms
    (Anthropic adaptive-thinking + Kimi pricing, from generated model data),
    unrelated to this change.
  • Codegen missing: 0, native 52 -> 53.
  • cargo fmt --all --check, cargo clippy --all-targets (no warnings), and
    whole-repo straitjacket v0.2.3 (crates conformance scripts .github, vendor
    excluded): 0 errors.

🤖 Generated with Claude Code


Generated by Claude Code

@zmaril
zmaril marked this pull request as ready for review July 21, 2026 23:10
Back pi's packages/ai/src/utils/validation.ts with the Rust port in
crates/pidgin-ai (utils/validation.rs). validateToolArguments and
validateToolCall now run natively: the AJV-compatible primitive coercion,
the recursive allOf/anyOf/oneOf + object + array coercion driver, the
pass/fail check over the JSON-Schema subset, and the
`Validation failed for tool "<name>": …` throw-envelope all live in Rust.

- Add crates/pidgin-napi/src/ai_validation.rs exposing both functions as
  free #[napi] fns. Tool/ToolCall cross as JSON strings (the TypeBox schema's
  JSON-Schema projection, the [TypeBox.Kind] symbol dropped by JSON.stringify);
  each returns a discriminated outcome envelope ({ok,value}/{ok,error}).
- Register `pub mod ai_validation;` in lib.rs (pub for the free-fn reachability
  rule).
- Add the native shim conformance/shims/packages/ai/src/utils/validation.ts:
  re-export the pi original, override both fns to marshal via JSON.stringify,
  return the coerced value, and re-throw new Error(envelope) on failure.
- Flip the manifest row to status:"native", tests:["test/validation.test.ts"].

test/validation.test.ts passes 3/3 native. Full ai suite: zero new
regressions (the 3 failing tests are pre-existing model-catalog phantoms).
Codegen missing:0, native 52 -> 53.
@zmaril
zmaril force-pushed the flip/ai-validation branch from 0402d24 to 0e46a63 Compare July 22, 2026 00:18
@github-actions

Copy link
Copy Markdown

Conformance smoke: agent + ai + tui

  • pi_sha: 3da591ab74ab9ab407e72ed882600b2c851fae21
  • manifest_native_modules: 55

Headline is rust-backed: passing cases in files whose module-under-test is a native (Rust addon) module. Raw all-pass is shown secondary — it is inflated by unflipped TypeScript that passes without touching any Rust.

Package rust-backed Native modules raw pass (secondary) Failing Skipped
agent 73/180 (40.6%) 7 180/180 0 0
ai 84/1296 (6.5%) 12 553/1296 3 740
tui 375/678 (55.3%) 14 678/678 0 0
smoke total 532/2154 (24.7%) 33 1411/2154 3 740

agent: 73/180 (40.6%) · ai: 84/1296 (6.5%) · tui: 375/678 (55.3%)

rust-backed = passing / total tests run, per the manifest's per-native-row tests lists. The Native modules column counts modules served by the Rust addon. This is the agent+ai+tui smoke subset; the full baseline lives in committed conformance.json.

Attribution

A pi test file is rust-backed iff the module it primarily exercises — its module-under-test — is status=native in conformance/manifest.json. A package's rust-backed count is the passing cases in its rust-backed files. Transitive or infrastructure use does not count (e.g. ~30 ai files construct the native faux provider but test other things — only faux-provider.test.ts, whose subject is faux, counts). A file that substantially tests both a native and an original module is excluded rather than counted, so the number under-reports rather than over-claims.

Per-file decisions (from each native manifest row's tests list):

Native module Test file Decision
ai/api/anthropic-messages.ts test/anthropic-sse-parsing.test.ts counted
ai/providers/faux.ts test/faux-provider.test.ts counted
coding-agent/utils/ansi.ts test/ansi-utils.test.ts counted
coding-agent/utils/changelog.ts test/changelog.test.ts counted
coding-agent/utils/git.ts test/git-ssh-url.test.ts counted
coding-agent/core/tools/path-utils.ts test/path-utils.test.ts counted
tui/keys.ts test/keys.test.ts counted
tui/utils.ts test/truncate-to-width.test.ts, test/regression-regional-indicator-width.test.ts counted
coding-agent/utils/mime.ts test/image-process.test.ts mixed — excluded (2 of 3 cases test processImage/original)
coding-agent/utils/version-check.ts test/version-check.test.ts mixed — excluded (fetch cases mock the original; not separable per-file)
coding-agent/core/export-html/ansi-to-html.ts test/export-html-whitespace.test.ts mixed — excluded (asset-grep + tool-renderer/original dominate)
coding-agent/core/tools/truncate.ts no dedicated test (exercised via tools.test.ts, subject is the read/edit factory/original)
coding-agent/core/tools/edit-diff.ts no dedicated test (edit-tool tests' subject is the edit tool/original)

CLI conformance (black-box, against the pidgin binary)

  • CLI conformance: 15/15 pass against target/release/pidgin (pass delta ±0, fail delta ±0).
File Passing Failing Skipped
packages/coding-agent/test/session-file-invalid.test.ts 1 0 0
packages/coding-agent/test/session-id-readonly.test.ts 7 0 0
packages/coding-agent/test/startup-session-name.test.ts 2 0 0
packages/coding-agent/test/stdout-cleanliness.test.ts 5 0 0

The four repointed coding-agent CLI test files spawn the compiled pidgin binary via $PIDGIN_BIN instead of pi's own cli.ts. This is a separate signal from the module smoke table and is never folded into the per-package Native count.

@zmaril
zmaril merged commit f054e5d into main Jul 22, 2026
13 checks passed
@zmaril
zmaril deleted the flip/ai-validation branch July 22, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants