RFC 008 slice 0a: core types, normalized manifest, and fixtures

[zkwentz]

Checkpoint

Stacked on #1041 (RFC). First half of the slice-0 contracts (split from the original #1042 for reviewability; the registries/report/policy half follows in #1042).

PYTHONPATH=src:envs uv run pytest tests/test_validation/ -v   # 26 passed
python scripts/sync_validation_schemas.py --check              # schemas in sync

How to review (~450 hand-written lines)

Read in this order:

1. src/openenv/validation/types.py (~95 lines) — the seven enums. Carries the core invariant: graders emit CheckStatus; only the severity policy assigns Severity.
2. src/openenv/validation/manifest.py (~300 lines) — NormalizedManifest + component models, the entire interface between a package format and every grader. The validators are the substance: judge pin + variance tolerance iff llm_judged; set_state required for injected-state oracles; verifier entry iff kind == "script"; NetworkPolicy per the Harbor task.toml 1.4 precedent (mode public default — egress allowed — / no-network / allowlist); GPU declarations (gpus/gpu_types) are resource fields, not disqualifiers; a missing oracle is a valid manifest (graded FAIL later, not a parse error).
3. tests/test_validation/test_manifest.py — the validator behaviors, one test each.

Skim only:

• schemas/manifest.schema.json (~450 lines) — generated from the pydantic model, CI-checked by scripts/sync_validation_schemas.py.
• tests/fixtures/validation/ (20 files, ~480 lines) — ten fixture packages; read served_min_pass/ and broken_manifest/, the other eight are variations. Each normalized_manifest.json doubles as the golden parse result for the parser slices.

Refs #778, #898.

🤖 Generated with Claude Code

---

Note

Low Risk
New additive package and tests only; no changes to runtime CLI behavior or existing env execution paths.

Overview
Introduces the RFC 008 validation contract foundation: a new openenv.validation package with shared enums and the normalized manifest Pydantic models that parsers will emit and graders will consume.

Core types (types.py) define validation levels, local vs hub lanes, grader CheckStatus vs policy Severity, verdicts, package SignatureKind (openenv.yaml, task.toml, task.md), and provider capabilities.

NormalizedManifest and nested models encode reward/oracle/verifier rules, resource and network declarations, capabilities (including optional oracle — valid at parse time, graded later), and type tags. Cross-field validators enforce LLM judge pins + variance tolerance, set_state for injected-state oracles, and verifier entry only for script verifiers.

Packaging ships committed manifest.schema.json (generated from models) via scripts/sync_validation_schemas.py (--check / --fix for CI). Ten fixture packages under tests/fixtures/validation/ supply golden normalized manifests for future parser work plus pass/fail schema cases; tests/test_validation/ covers enum contracts, manifest validation, and schema sync.

^{Reviewed by Cursor Bugbot for commit e2e84a8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Golden manifests invent package version

Medium Severity

Served fixture goldens set version to 0.1.0, but the matching openenv.yaml sources never declare a package version. NormalizedManifest.version defaults to None, so a faithful parser would emit null. Harbor/PostTrain fixtures keep source and golden aligned; these served ones do not, so later parser golden checks will encode the wrong expected result.

 

^{Reviewed by Cursor Bugbot for commit 5cc099f. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e2e84a8. Configure here.}

[cursor]

Incomplete llm_judged invariant

Medium Severity

_judge_pin_iff_llm_judged treats the judge pin as present iff llm_judged, but only requires reward.variance_tolerance when llm_judged is true — it never rejects a non-null variance_tolerance when llm_judged is false. That breaks the stated bidirectional invariant and can let non-judged manifests carry a variance bound that only variance-mode determinism should use.

 

^{Reviewed by Cursor Bugbot for commit e2e84a8. Configure here.}