Skip to content

Normalize LLM judge scores in result JSON#39

Open
Sehlani042 wants to merge 1 commit into
Accenture:mainfrom
Sehlani042:codex/normalize-llm-judge-scores
Open

Normalize LLM judge scores in result JSON#39
Sehlani042 wants to merge 1 commit into
Accenture:mainfrom
Sehlani042:codex/normalize-llm-judge-scores

Conversation

@Sehlani042

Copy link
Copy Markdown

Summary

  • normalize raw LLM judge subdimension scores from the 0-10 rubric to the 0-1 result scale
  • recalculate aggregate judge scores from normalized subdimensions for both standard and stability evaluation paths
  • document the score scale and add unittest coverage for helper, stability averaging, and TaskEvaluator output

Tests

  • python3 -m unittest discover -s tests
  • python3 -m compileall benchmark tests/test_score_normalization.py
  • git diff --check

Fixes #15.
Addresses #23.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Computation/rescale for the overall and each-item score

1 participant