feat(grader): support agent-rubric compatibility

apps/web/src/content/docs/docs/next/graders/llm-graders.mdx

@@ -46,6 +46,40 @@ assert:
 
 Use `provider:` when you want different `llm-rubric` entries in the same eval to run on different grader models. This is useful for grader panels, majority-vote ensembles, and grader A/B benchmarks.
 
+## Agent Rubrics
+
+Use `type: agent-rubric` when a Promptfoo-style rubric needs an agentic grader
+that can inspect the workspace instead of only judging the final answer text.
+AgentV routes `agent-rubric` through the same `llm-rubric` / `llm-grader`
+scoring path, so results still appear as normal `EvaluationScore` entries.
+
+```yaml
+assert:
+  - name: workspace-evidence
+    type: agent-rubric
+    provider: codex-grader
+    value: The answer's claims are backed by concrete files in the workspace.
+```
+
+The resolved grader provider must be agent-capable, such as a Codex, Claude,
+Copilot, Pi, VS Code, or `agentv` provider. If `provider:` resolves to a
+plain LLM grader provider, AgentV fails clearly instead of silently downgrading
+the check.
+
+For agent-backed grading, AgentV creates a temporary `verdict.json` path for
+each grading call and instructs the agent grader to write exactly one JSON
+object there:
+
+```json
+{ "pass": true, "score": 1, "reason": "Evidence found in src/logger.ts." }
+```
+
+AgentV reads that file before parsing the final assistant message. The `score`
+must be a finite number from `0` to `1`; malformed verdict files fail closed as
+grader failures. If the file is missing, AgentV may still parse the final
+assistant text using the same structured JSON fallback used by other agent
+grader modes.
+
 ## Prompt Files
 
 The prompt file defines evaluation criteria and scoring guidelines. It can be a markdown text template or a TypeScript/JavaScript dynamic template.

apps/web/src/content/docs/docs/next/reference/promptfoo-parity.mdx

@@ -72,7 +72,7 @@ implements equivalent semantics directly.
 | Authored preprocessors | Not Promptfoo's canonical output-shaping surface. | Rejected in current authored YAML. | Removed/rejected surface | Use `transform` at `default_test.options`, `tests[].options`, or the assertion that needs the shaped output. Historical versioned docs may still show old preprocessor examples. |
 | Suite assertions | `assert` entries can be strings or typed assertion objects. | `assert` entries can be strings, typed assertion objects, script graders, or AgentV extension graders. | Align with Promptfoo | Plain strings become semantic rubric checks. Use `assert`, not `assertions`, in current authored eval YAML. |
 | Assertion grouping | `type: assert-set` with child `assert` entries, optional `config`, `metric`, `weight`, and `threshold`. | `type: assert-set` with child `assert`, optional `config`, metric names, weights, and parent threshold. | Align with Promptfoo | Parent `config` is inherited by child assertions; child `config` keys override shared parent keys. Without `threshold`, pass/fail follows nonzero-weight child assertions. With `threshold`, the weighted aggregate score determines pass/fail. `type: composite` is rejected; use `assert-set`. |
-| Deterministic assertion vocabulary | Common Promptfoo types include `contains`, `icontains`, `contains-any`, `contains-all`, `starts-with`, `regex`, `is-json`, `equals`, `latency`, `cost`, `javascript`, `python`, `webhook`, `similar`, and `llm-rubric`. | AgentV accepts the implemented overlap, including `contains`, `icontains`, `contains-any`, `contains-all`, `starts-with`, `regex`, `is-json`, `equals`, `latency`, `cost`, `javascript`, `python`, `webhook`, `similar`, and `llm-rubric`. | Align with Promptfoo | Unsupported Promptfoo assertion names error instead of silently becoming custom assertion names. |
+| Deterministic and rubric assertion vocabulary | Common Promptfoo types include `contains`, `icontains`, `contains-any`, `contains-all`, `starts-with`, `regex`, `is-json`, `equals`, `latency`, `cost`, `javascript`, `python`, `webhook`, `similar`, `llm-rubric`, and `agent-rubric`. | AgentV accepts the implemented overlap, including `contains`, `icontains`, `contains-any`, `contains-all`, `starts-with`, `regex`, `is-json`, `equals`, `latency`, `cost`, `javascript`, `python`, `webhook`, `similar`, `llm-rubric`, and `agent-rubric`. | Align with Promptfoo | `agent-rubric` runs through AgentV-native agent grader providers and writes a structured verdict file before fallback text parsing. Unsupported Promptfoo assertion names error instead of silently becoming custom assertion names. |
 | Custom assertion terminology | Promptfoo calls normal eval custom logic assertions, with fixed code assertion types such as `javascript`, `python`, `ruby`, and `webhook`. | `defineAssertion()` files in `.agentv/assertions/` become reusable assertion type names. | Keep AgentV extension | AgentV keeps assertion terminology and extends discovery to arbitrary assertion type names such as `has-citation`. |
 | Script/custom grader terminology | Promptfoo custom code assertions are still assertion types. | `defineScriptGrader()` powers command-backed graders referenced with `type: script` and `command:`. | Keep AgentV divergence | Use script grader wording only for command-backed or LLM-backed scoring components that need explicit score and assertion-result control. |
 | Skill assertions | Promptfoo includes `skill-used` for checking whether an agent invoked a named skill. | `type: skill-used` and `type: not-skill-used` with `value: <skill>` or a matcher object. | Align with Promptfoo | AgentV evaluates these against normalized tool-call and skill-use trace data. |