docs(sdk): clarify custom assertion terminology

README.md

@@ -279,7 +279,8 @@ export default defineEval({
 Full docs at [agentv.dev/docs](https://agentv.dev/docs/getting-started/introduction/).
 
 - [Eval files](https://agentv.dev/docs/evaluation/eval-files/) — format and structure
-- [Custom graders](https://agentv.dev/docs/graders/custom-graders/) — script graders in any language
+- [Custom assertions](https://agentv.dev/docs/graders/custom-assertions/) — reusable assertion types
+- [Script graders](https://agentv.dev/docs/graders/script-graders/) — command-backed graders in any language
 - [Rubrics](https://agentv.dev/docs/evaluation/rubrics/) — structured criteria scoring
 - [Targets](https://agentv.dev/docs/targets/configuration/) — configure agents and providers
 - [Compare results](https://agentv.dev/docs/tools/compare/) — A/B testing and regression detection

apps/web/src/content/docs/docs/next/evaluation/sdk.mdx

@@ -46,7 +46,8 @@ Use the simplest surface that matches the job:
 - **`defineEval()` / `evalSuite()`** when you want a `.eval.ts` file that mirrors YAML concepts and lowers back to the canonical snake_case contract.
 - **`evaluate({ specFile })`** when you want library control around an existing YAML suite.
 - **Inline `evaluate({ tests })`** when the eval definition truly belongs inside application code. The programmatic API mirrors YAML, but uses current TypeScript naming such as `expectedOutput`.
-- **`defineAssertion` / `defineScriptGrader`** when the grading logic itself must execute code.
+- **`defineAssertion`** when you want a reusable assertion type discovered from `.agentv/assertions/`.
+- **`defineScriptGrader`** when you need a command-backed grader with explicit score and assertion-result control.
 - **`agentv eval <verifier.test.ts>`** for deterministic workspace checks that fit normal Vitest `expect(...)` tests.
 
 There is no separate first-party Python authoring SDK today. Python-facing workflows should either emit canonical YAML/JSONL or implement executable graders that consume the standard `snake_case` wire format.
@@ -209,6 +210,8 @@ assert:
 
 Use `defineAssertion` from `@agentv/sdk` to create reusable assertion types. Place them in `.agentv/assertions/` — they're auto-discovered by filename.
 
+This is the custom assertion path, not the custom grader path. It matches Promptfoo's assertion terminology for normal eval checks, while extending Promptfoo's fixed custom logic types (`javascript`, `python`, `ruby`, `webhook`) with arbitrary discovered AgentV type names.
+
 ### Pass/Fail Pattern
 
 ```typescript
@@ -312,7 +315,7 @@ export default defineWorkspaceGrader(async ({ workspace }) => [
 ]);
 ```
 
-`defineScriptGrader`, `defineVitestWorkspaceGrader`, and `defineWorkspaceGrader` custom scripts are referenced in YAML with `type: script` and `command: [bun, run, grader.ts]`. Plain Vitest verifier files can use `command: [agentv, eval, graders/check.test.ts]` without a custom wrapper; use `agentv eval vitest` when you need adapter flags. `defineAssertion` uses convention-based discovery instead — just place in `.agentv/assertions/` and reference by name.
+`defineScriptGrader`, `defineVitestWorkspaceGrader`, and `defineWorkspaceGrader` custom scripts are graders referenced in YAML with `type: script` and `command: [bun, run, grader.ts]`. Plain Vitest verifier files can use `command: [agentv, eval, graders/check.test.ts]` without a custom wrapper; use `agentv eval vitest` when you need adapter flags. `defineAssertion` uses convention-based discovery instead — just place it in `.agentv/assertions/` and reference it by assertion type name.
 
 For detailed patterns, input/output contracts, and language-agnostic examples, see [Script Graders](/docs/graders/script-graders/).
 

apps/web/src/content/docs/docs/next/getting-started/quickstart.mdx

@@ -72,5 +72,5 @@ Results appear in `.agentv/results/<run_id>/.internal/index.jsonl` with scores,
 
 - Learn about [eval file formats](/docs/evaluation/eval-files/)
 - Configure [targets](/docs/targets/configuration/) for different providers
-- Create [custom graders](/docs/graders/custom-graders/)
+- Choose [custom assertions](/docs/graders/custom-assertions/) or [script graders](/docs/graders/script-graders/)
 - If setup drifts, rerun: `agentv init`

apps/web/src/content/docs/docs/next/graders/custom-assertions.mdx

@@ -6,23 +6,29 @@ sidebar:
 slug: docs/graders/custom-assertions
 ---
 
-Custom assertions let you add evaluation logic that goes beyond built-in types. Define a TypeScript function, drop it in `.agentv/assertions/`, and reference it by name in your YAML eval files.
+Custom assertions let you add reusable assertion types that go beyond built-in types. Define a TypeScript function, drop it in `.agentv/assertions/`, and reference it by name in your YAML eval files.
 
 ## When to Use Each Approach
 
 AgentV provides two SDK functions for custom evaluation logic:
 
 | Function | Best For | Discovery |
 |----------|----------|-----------|
-| `defineAssertion()` | Pass/fail checks, reusable assertion types | Convention-based (`.agentv/assertions/`) |
-| `defineScriptGrader()` | Full scoring control with explicit assertions array | Referenced via `type: script` + `command:` |
+| `defineAssertion()` | Reusable assertion types with pass/fail plus optional score | Convention-based (`.agentv/assertions/`) |
+| `defineScriptGrader()` | Command-backed scorer with full score and assertion-result control | Referenced via `type: script` + `command:` |
 
 **Use `defineAssertion()`** when you want a named assertion type that can be referenced across eval files without specifying a command path. It uses a simplified result contract focused on `pass` and optional `score`.
 
-**Use `defineScriptGrader()`** when you need full control over scoring with explicit `assertions` arrays, or when the grader is a one-off grader tied to a specific eval. See [Script Graders](/docs/graders/script-graders/) for details.
+**Use `defineScriptGrader()`** when the scoring component is a command-backed grader: it needs explicit score calculation, custom assertion-result arrays, workspace commands, or LLM calls through a grader target. See [Script Graders](/docs/graders/script-graders/) for details.
 
 Both functions handle stdin/stdout JSON parsing, snake_case-to-camelCase conversion, Zod validation, and error handling automatically.
 
+## Promptfoo Terminology
+
+Promptfoo calls normal eval checks assertions. Its custom code paths use fixed assertion types such as `javascript`, `python`, `ruby`, and `webhook`, and its Node API exposes assertion-oriented helpers such as `runAssertion()` and `runAssertions()`.
+
+AgentV follows that framing for `assert:` entries and `defineAssertion()`. The AgentV extension is convention discovery: any file in `.agentv/assertions/` becomes an assertion type name such as `word-count` or `has-citation`. Reserve custom grader or script grader wording for command-backed or LLM-backed scoring components, especially `type: script` entries built with `defineScriptGrader()`.
+
 ## Installation
 
 ```bash

apps/web/src/content/docs/docs/next/graders/custom-graders.mdx

@@ -6,7 +6,7 @@ sidebar:
 slug: docs/graders/custom-graders
 ---
 
-AgentV supports multiple grader types that can be combined for comprehensive evaluation.
+AgentV supports multiple grader types that can be combined for comprehensive evaluation. Use this page for command-backed or LLM-backed scoring components. For reusable assertion types discovered from `.agentv/assertions/`, see [Custom Assertions](/docs/graders/custom-assertions/).
 
 ## Grader Types
 
@@ -83,5 +83,6 @@ If any grader has `required: true` and scores below its required threshold, the
 - **Use plain assertion strings first for semantic checks** — AgentV treats them as rubric criteria
 - **Use script graders for deterministic checks** — exact value matching, format validation, schema compliance
 - **Use `llm-rubric` for semantic evaluation** — meaning, quality, helpfulness, or weighted itemized scoring
+- **Use custom assertions for reusable pass/fail types** — define them with `defineAssertion()` and reference them by discovered type name
 - **Combine grader types** for comprehensive coverage
 - **Test script graders locally** before running full evaluations

apps/web/src/content/docs/docs/next/graders/script-graders.mdx

@@ -8,6 +8,8 @@ slug: docs/graders/script-graders
 
 Script graders are scripts that evaluate agent responses deterministically. Write them in any language — Python, TypeScript, Node, or any executable.
 
+Use script graders when you need a command-backed scoring component with explicit score control. If you only need a reusable assertion type that can be referenced by name from `.agentv/assertions/`, use [Custom Assertions](/docs/graders/custom-assertions/) instead.
+
 ## Contract
 
 Script graders receive eval context via stdin JSON and return a result via stdout.

apps/web/src/content/docs/docs/v4.42.4/evaluation/sdk.mdx

@@ -12,13 +12,13 @@ YAML remains AgentV's canonical, portable eval format. The SDK surfaces below ar
 
 AgentV currently provides two npm packages for programmatic use:
 
-- **`@agentv/sdk`** — YAML-aligned eval authoring, custom assertions, and code graders
+- **`@agentv/sdk`** — YAML-aligned eval authoring, custom assertions, and script graders
 - **`@agentv/core`** — programmatic evaluation API and typed configuration
 
 ## Installation
 
 ```bash
-# Lightweight SDK (defineEval, graders, defineAssertion, defineCodeGrader)
+# Lightweight SDK (defineEval, graders, defineAssertion, defineScriptGrader)
 npm install @agentv/sdk
 
 # Programmatic API (evaluate, defineConfig)
@@ -35,7 +35,7 @@ npm install @agentv/sdk
 ```
 
 ```typescript
-import { defineCodeGrader } from '@agentv/sdk';
+import { defineScriptGrader } from '@agentv/sdk';
 ```
 
 The general policy is hard convergence for same-week or unreleased surface names: use the correct package, field, or wire name instead of carrying aliases. The package rename is the exception because `@agentv/eval` was already published. It remains a temporary deprecated compatibility package that re-exports `@agentv/sdk` for existing consumers, but it should not appear in new docs, examples, scaffolds, or skills except as migration guidance.
@@ -48,11 +48,12 @@ Use the simplest surface that matches the job:
 - **`defineEval()` / `evalSuite()`** when you want a `.eval.ts` file that mirrors YAML concepts and lowers back to the canonical snake_case contract.
 - **`evaluate({ specFile })`** when you want library control around an existing YAML suite.
 - **Inline `evaluate({ tests })`** when the eval definition truly belongs inside application code. The programmatic API mirrors YAML, but uses current TypeScript naming such as `expectedOutput` and `assert`.
-- **`defineAssertion` / `defineCodeGrader`** when the grading logic itself must execute code.
+- **`defineAssertion`** when you want a reusable assertion type discovered from `.agentv/assertions/`.
+- **`defineScriptGrader`** when you need a command-backed grader with explicit score and assertion-result control.
 
 There is no separate first-party Python authoring SDK today. Python-facing workflows should either emit canonical YAML/JSONL or implement executable graders that consume the standard `snake_case` wire format.
 
-For example, the repo-local helper in `examples/features/sdk-python/` can build YAML-shaped cases while keeping `assertions` as the durable contract:
+For example, the repo-local helper in `examples/features/sdk-python/` can build YAML-shaped cases while keeping `assert` as the durable authored contract:
 
 ```python
 from agentv_py.evals import EvalDefinition, JsonlCase, write_eval_yaml, write_jsonl
@@ -61,7 +62,7 @@ from agentv_py.evals import EvalDefinition, JsonlCase, write_eval_yaml, write_js
 def rag_faithfulness():
     return {
         "name": "rag-faithfulness",
-        "type": "llm-grader",
+        "type": "llm-rubric",
         "target": "grader-target",
         "prompt": "Grade whether the answer is supported by the retrieved context.",
     }
@@ -74,7 +75,7 @@ write_jsonl(
             id="grounded-answer",
             input=[{"role": "user", "content": "Answer using the retrieved context."}],
             expected_output=[{"role": "assistant", "content": "The answer cites the source material."}],
-            extra={"assertions": [rag_faithfulness()]},
+            extra={"assert": [rag_faithfulness()]},
         )
     ],
 )
@@ -113,7 +114,7 @@ export default defineEval({
       input: 'Say hello',
       inputFiles: ['../fixtures/per-test-note.md'],
       expectedOutput: 'Hello from the mock target',
-      assertions: [graders.contains('Hello')],
+      assert: [graders.contains('Hello')],
     },
   ],
 });
@@ -124,11 +125,11 @@ Useful companion helpers:
 - `toEvalYamlObject()` returns the canonical snake_case object.
 - `serializeEvalYaml()` returns YAML text using the same canonical field names.
 
-The durable field remains `assertions`. This helper does not introduce a second YAML vocabulary.
+The durable authored field remains `assert`. This helper does not introduce a second YAML vocabulary.
 
 ## Built-In Grader Helpers
 
-`@agentv/sdk` includes a small `graders` catalog for common deterministic and LLM-backed grader configs. These helpers return ordinary `assertions` entries and serialize to the same canonical YAML you could write by hand.
+`@agentv/sdk` includes a small `graders` catalog for common deterministic and LLM-backed grader configs. These helpers return ordinary `assert` entries and serialize to the same canonical YAML you could write by hand.
 
 ```typescript
 import { defineEval, graders } from '@agentv/sdk';
@@ -139,35 +140,35 @@ export default defineEval({
     {
       id: 'json-greeting',
       input: 'Return a JSON greeting.',
-      assertions: [
+      assert: [
         graders.contains('Hello', { name: 'mentions-hello' }),
         graders.exact('{"message":"Hello"}', { name: 'exact-json', minScore: 1 }),
         graders.regex(/"message"\s*:/, { name: 'message-key' }),
         graders.json({ name: 'valid-json', required: true }),
-        graders.rubrics(['Greets the user'], { name: 'rubric-review' }),
-        graders.llmGrader({
+        graders.llmRubric(['Greets the user'], { name: 'rubric-review' }),
+        graders.llmRubric(undefined, {
           name: 'llm-review',
           prompt: 'Grade whether the answer is useful.',
           target: 'grader-target',
         }),
-        graders.codeGrader(['bun', 'run', 'graders/check.ts'], { name: 'scripted-check' }),
+        graders.scriptGrader(['bun', 'run', 'graders/check.ts'], { name: 'scripted-check' }),
       ],
     },
   ],
 });
 ```
 
-The catalog covers `contains`, `equals`/`exact`, `regex`, `is-json`/`json`, `rubrics`, `llm-grader`, and `code-grader`. CamelCase SDK options such as `minScore`, `maxSteps`, and rubric `scoreRanges` lower to `min_score`, `max_steps`, and `score_ranges` when AgentV loads or serializes the suite.
+The catalog covers `contains`, `equals`/`exact`, `regex`, `is-json`/`json`, `llm-rubric`, and `script`. CamelCase SDK options such as `minScore`, `maxSteps`, and rubric `scoreRanges` lower to `min_score`, `max_steps`, and `score_ranges` when AgentV loads or serializes the suite.
 
 ## AgentV-Native Helper Factories
 
-If you are coming from Braintrust `scores` or DeepEval metrics, keep the reusable logic AgentV-native: write helper factories that return `graders.*` configs, then let `defineEval()` lower them to ordinary `assertions`.
+If you are coming from Braintrust `scores` or DeepEval metrics, keep the reusable logic AgentV-native: write helper factories that return `graders.*` configs, then let `defineEval()` lower them to ordinary `assert` entries.
 
 ```typescript
 import { defineEval, graders } from '@agentv/sdk';
 
 function ragFaithfulness() {
-  return graders.llmGrader({
+  return graders.llmRubric(undefined, {
     name: 'rag-faithfulness',
     target: 'grader-target',
     prompt: [
@@ -184,7 +185,7 @@ export default defineEval({
       id: 'grounded-answer',
       input: 'Answer the question using the retrieved context.',
       expectedOutput: 'The answer cites the source material.',
-      assertions: [
+      assert: [
         graders.contains('source', { name: 'mentions-source' }),
         ragFaithfulness(),
       ],
@@ -196,12 +197,12 @@ export default defineEval({
 The helper above serializes to the same shape you could write by hand:
 
 ```yaml
-assertions:
+assert:
   - name: mentions-source
     type: contains
     value: source
   - name: rag-faithfulness
-    type: llm-grader
+    type: llm-rubric
     target: grader-target
     prompt: |-
       Grade whether the answer is supported by the retrieved context.
@@ -212,6 +213,8 @@ assertions:
 
 Use `defineAssertion` from `@agentv/sdk` to create reusable assertion types. Place them in `.agentv/assertions/` — they're auto-discovered by filename.
 
+This is the custom assertion path, not the custom grader path. It matches Promptfoo's assertion terminology for normal eval checks, while extending Promptfoo's fixed custom logic types (`javascript`, `python`, `ruby`, `webhook`) with arbitrary discovered AgentV type names.
+
 ### Pass/Fail Pattern
 
 ```typescript
@@ -260,20 +263,20 @@ Convention-based discovery maps filename → assertion type:
 Reference directly in your eval file — no `command:` needed:
 
 ```yaml
-assertions:
+assert:
   - type: word-count
   - type: contains
     value: "Hello"
 ```
 
-## Code Graders
+## Script Graders
 
-Use `defineCodeGrader` from `@agentv/sdk` for full control over scoring with an explicit assertions array:
+Use `defineScriptGrader` from `@agentv/sdk` for full control over scoring with an explicit assertions array:
 
 ```typescript
-import { defineCodeGrader } from '@agentv/sdk';
+import { defineScriptGrader } from '@agentv/sdk';
 
-export default defineCodeGrader(({ output, traceSummary }) => ({
+export default defineScriptGrader(({ output, traceSummary }) => ({
   score: (output ?? '').length > 0 && (traceSummary?.eventCount ?? 0) <= 5 ? 1.0 : 0.5,
   assertions: [
     { text: 'Answer is not empty', passed: (output ?? '').length > 0 },
@@ -282,9 +285,9 @@ export default defineCodeGrader(({ output, traceSummary }) => ({
 }));
 ```
 
-`defineCodeGrader` graders are referenced in YAML with `type: code-grader` and `command: [bun, run, grader.ts]`. `defineAssertion` uses convention-based discovery instead — just place in `.agentv/assertions/` and reference by name.
+`defineScriptGrader` scripts are graders referenced in YAML with `type: script` and `command: [bun, run, grader.ts]`. `defineAssertion` uses convention-based discovery instead — just place it in `.agentv/assertions/` and reference it by assertion type name.
 
-For detailed patterns, input/output contracts, and language-agnostic examples, see [Code Graders](/docs/v4.42.4/graders/code-graders/).
+For detailed patterns, input/output contracts, and language-agnostic examples, see [Code Graders](/docs/v4.42.4/graders/code-graders/) in this versioned doc set.
 
 ## Wire Format vs SDK Format
 

apps/web/src/content/docs/docs/v4.42.4/getting-started/quickstart.mdx

@@ -57,9 +57,9 @@ tests:
 
     expected_output: "42"
 
-    assertions:
+    assert:
       - name: math_check
-        type: code-grader
+        type: script
         command: [./validators/check_math.py]
 ```
 
@@ -75,5 +75,5 @@ Results appear in `.agentv/results/runs/<timestamp>/index.jsonl` with scores, re
 
 - Learn about [eval file formats](/docs/v4.42.4/evaluation/eval-files/)
 - Configure [targets](/docs/v4.42.4/targets/configuration/) for different providers
-- Create [custom graders](/docs/v4.42.4/graders/custom-graders/)
+- Choose [custom assertions](/docs/v4.42.4/graders/custom-assertions/) or command-backed [custom graders](/docs/v4.42.4/graders/custom-graders/)
 - If setup drifts, rerun: `agentv init`

apps/web/src/content/docs/docs/v4.42.4/graders/code-graders.mdx

@@ -10,6 +10,8 @@ pagefind: false
 
 Code graders are scripts that evaluate agent responses deterministically. Write them in any language — Python, TypeScript, Node, or any executable.
 
+Use command-backed graders when you need explicit score control. If you only need a reusable assertion type that can be referenced by name from `.agentv/assertions/`, use [Custom Assertions](/docs/v4.42.4/graders/custom-assertions/) instead.
+
 ## Contract
 
 Code graders receive eval context via stdin JSON and return a result via stdout.