Skip to content

fix(harness): correct math_08 reference answer - #629

Open
pepuscz wants to merge 1 commit into
Luce-Org:mainfrom
pepuscz:fix/harness-math08-answer
Open

fix(harness): correct math_08 reference answer#629
pepuscz wants to merge 1 commit into
Luce-Org:mainfrom
pepuscz:fix/harness-math08-answer

Conversation

@pepuscz

@pepuscz pepuscz commented Aug 18, 2026

Copy link
Copy Markdown

Summary

Correct the math_08 reference answer from 20 to 20/3.

Squaring the stated equation gives 9(3x+5) = 5(6x+5), hence 27x + 45 = 30x + 25 and x = 20/3. The current gold answer marks a correct model response as wrong.

The reproducible benchmark setup that exposed the scoring mismatch is published at https://github.com/pepuscz/strix-halo-deepseek-v4-flash.

Validation

  • Parsed the complete JSONL file with jq
  • Re-derived the answer independently from the prompt
  • git diff --check

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant