Skip to content

fix(eval): accept target id in targets files - #1631

Merged
christso merged 1 commit into
mainfrom
fix/targets-id-validation
Jul 4, 2026
Merged

fix(eval): accept target id in targets files#1631
christso merged 1 commit into
mainfrom
fix/targets-id-validation

Conversation

@christso

@christso christso commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Live quickstart evals can now execute against checked-in targets.yaml files that use the current id target identity. Before this, the merged README/example contract validated but agentv eval run still rejected targets.yaml at runtime with Missing or invalid 'label' field.

This keeps name as the internal resolver key, rejects authored label/name at validation boundaries, and migrates the tracked target templates/examples to id so new projects and examples use the supported schema directly.

Related: av-kyv3

Validation

  • bun test packages/core/test/evaluation/providers/targets-file.test.ts packages/core/test/evaluation/providers/targets.test.ts packages/core/test/evaluation/validation/targets-validator.test.ts packages/core/test/evaluation/loaders/config-loader.test.ts
  • bun --filter @agentv/core build
  • bun run validate:examples (107 valid, 0 invalid)
  • bun run lint
  • bun run typecheck
  • git diff --check
  • Live quickstart with local OpenAI-compatible endpoint and concurrency 2:
    LOCAL_OPENAI_PROXY_BASE_URL=http://127.0.0.1:10531/v1 LOCAL_OPENAI_PROXY_API_KEY=local-dummy-key LOCAL_OPENAI_PROXY_MODEL=gpt-5.3-codex-spark bun apps/cli/src/cli.ts eval run examples/features/readme-quickstart/evals/my-eval.eval.yaml --workers 2 --threshold 0.8 --verbose
  • Live run bundle: .agentv/results/2026-07-04T03-57-29-236Z; result was PASS, 1/1 tests, mean score 100%, grading.json score 1 verdict pass with 5/5 assertion_results passing.

Compound Engineering
GPT--5

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 4, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: a59fa76
Status: ✅  Deploy successful!
Preview URL: https://c7d909e2.agentv.pages.dev
Branch Preview URL: https://fix-targets-id-validation.agentv.pages.dev

View logs

@christso
christso force-pushed the fix/targets-id-validation branch from cfa9236 to dbb30c2 Compare July 4, 2026 04:06
Entire-Checkpoint: ef8f54a28ee7
@christso
christso force-pushed the fix/targets-id-validation branch from dbb30c2 to a59fa76 Compare July 4, 2026 04:31
@christso
christso merged commit 6f2110b into main Jul 4, 2026
8 checks passed
@christso
christso deleted the fix/targets-id-validation branch July 4, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant