Skip to content

feat: add RubricConfig.load_bundle() for rubric bundle manifests - #177

Draft
luca-belli wants to merge 1 commit into
mainfrom
feat/rubric-bundle-manifest-helper
Draft

feat: add RubricConfig.load_bundle() for rubric bundle manifests#177
luca-belli wants to merge 1 commit into
mainfrom
feat/rubric-bundle-manifest-helper

Conversation

@luca-belli

Copy link
Copy Markdown
Collaborator

Summary

  • Adds RubricConfig.load_bundle(), a classmethod that loads a rubric from a rubric bundle manifest JSON file (validates required keys, resolves relative paths against the manifest's folder, delegates to the existing RubricConfig.load())
  • Adds data/rubric_manifest.json, the production default manifest pointing at the existing data/rubric.tsv bundle
  • Adds tests/fixtures/rubric_manifest_simple.json, a test fixture manifest for the existing rubric_simple.tsv fixture

This is Phase 0 step 1 of 2 (helper only). A follow-up PR wires judge.py's currently-disconnected --rubrics flag to this helper.

Test plan

  • uv run pytest -m "not live" passes (963 passed)
  • New unit tests cover: successful load via manifest, FileNotFoundError on missing manifest, ValueError on missing required manifest key
  • uv run ruff check / uv run ruff format --check clean

@luca-belli
luca-belli marked this pull request as draft July 22, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant