feat(config)!: hard-deprecate the graders: config block — a grader is just a target

[christso]

Summary

Follow-up to PR #1703 (wired defaults.grader as a config-level fallback). This session's research (peer frameworks + this repo's own design principles + the live execution path) converged on: a grader is not a distinct entity, it's a target selected for a grading role.

• Margin-Lab/evals and Harbor were checked (local clones + DeepWiki): neither maintains a separate grader/judge schema type. Margin-Lab's own docs say it outright — "there is no separate grader abstraction."
• This repo's .agents/product-boundary.md already lists "config-level grader targets selected through defaults.grader... not target-level grader configuration" as a preferred pattern.
• The live .agentv/targets.yaml execution path already resolves grader_target/defaults.grader against the same target-id namespace — there was never a real separate grader concept there.
• The only place a separate "grader" entity still existed was config-graph.ts's graders: top-level array (NormalizedGraderConfig) — confirmed zero implementation: nothing ever turned it into a Provider.
• A grader can itself need grading (oracle/calibration checks) — that only works for free if it's addressable the same way as any other target, not siloed in a separate schema/namespace.

Changes

• config-graph.ts: removed NormalizedGraderConfig/parseGraders/the graders field. Authoring a graders: block now throws: Field 'graders' in ... has been removed. A grader is just a target — move each entry into 'targets' and select it via 'defaults.grader' or an assertion's target override. defaults.grader now validates against the same graph.targets pool as defaults.target (previously checked the now-removed graph.graders).
• eval-file.schema.ts: same hard-rejection via z.never(), matching the existing pattern for other removed fields (eval_cases, providerPromptMap) — this backs the published eval.schema.json IDE-autocomplete surface.
• config-loader.ts: dropped the dead graders pass-through.
• Docs (targets/configuration.mdx, targets/coding-agents.mdx, targets/cli-provider.mdx, targets/custom-providers.mdx, targets/llm-providers.mdx, evaluation/running-evals.mdx), root README.md, and the readme-quickstart example all had a graders: block — folded each into targets:. (Swept every other graders:-in-README hit first and confirmed the rest are an unrelated, still-valid pattern: per-assertion script graders like {name, type: script, command}, not this schema.)
• readme-quickstart's .agentv/targets.yaml/graders.yaml also still had the legacy ${{ VAR }} env syntax (pre-existing, unrelated breakage from an earlier hard-deprecation pass that missed this example) — fixed to {{ env.VAR }} in the same pass since the file needed touching anyway.
• Added a CONCEPTS.md entry disambiguating grader-selection (a target) from grader-method (an assertion type like llm-rubric/code-grader) and grader-prompt (criteria/rubric value) — three senses of "grader" that collide in this codebase.

Known pre-existing gap noticed but NOT fixed here (out of scope)

This repo's own root .agentv/config.yaml currently fails agentv validate for an unrelated reason: execution.verbose/execution.keep_workspaces aren't supported execution fields (only max_concurrency is). Untouched by this PR — flagging separately rather than scope-creeping this change.

Test plan

• New tests: config-graph.ts rejects an authored graders: block (direct loadComposableConfigGraph test); eval-file.schema.ts rejects it too (Zod schema test). Both also updated their sibling "accepts composable graph fields" tests to fold grader into targets:.
• bun test packages/core/test/evaluation/loaders packages/core/test/evaluation/validation — 656 pass. Plus the 4 files I hand-edited individually — 279 pass (includes overlap).
• bun run typecheck (full workspace) — clean.
• bunx biome check on touched source/test files — clean.
• bun run validate:examples — 109/109 example eval files valid.
• Live CLI check: a scratch .agentv/config.yaml with an authored graders: block fails with the new error message end-to-end via bun apps/cli/src/cli.ts validate; the folded-into-targets: version validates cleanly.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying agentv with    Cloudflare Pages

Latest commit:	6e7c322
Status:	✅  Deploy successful!
Preview URL:	https://4f84be5a.agentv.pages.dev
Branch Preview URL:	https://fix-deprecate-graders-schema.agentv.pages.dev

View logs