Skip to content

Severity tier for review comments#82

Open
dangng2004 wants to merge 1 commit into
mainfrom
wip/comment-severity
Open

Severity tier for review comments#82
dangng2004 wants to merge 1 commit into
mainfrom
wip/comment-severity

Conversation

@dangng2004
Copy link
Copy Markdown
Contributor

Summary

  • Adds a `severity` field (`minor`/`moderate`/`major`) to `Comment` and to the JSON output of all three review prompts.
  • New shared `SEVERITY_RUBRIC` block keeps the tier definitions in one place.
  • Parser normalizes the value and defaults to `moderate` when missing/invalid, so older outputs stay loadable.
  • Viz JSON now carries severity per comment.

Status

Draft. Functional but no tests or downstream consumers (e.g. ranking, filtering, viz UI styling) wired up yet.

Test plan

  • Sanity-check a real run produces severity on every comment
  • Viz UI: surface severity on comment cards
  • Decide whether evaluation metrics should be severity-weighted

Each comment now carries a "severity" field with values "minor",
"moderate", or "major" alongside the existing technical/logical type.

- models.py: new SEVERITY_VALUES constant and Comment.severity field;
  serialized in to_dict when set.
- prompts.py: shared SEVERITY_RUBRIC injected into the deep-check,
  zero-shot, and zero-shot-chunk prompts; output schemas include the
  field.
- utils.py: parse_comments_from_list normalizes the severity string
  and defaults to "moderate" when missing or invalid.
- cli.py: severity propagated into the viz JSON output.
@dangng2004 dangng2004 marked this pull request as ready for review May 14, 2026 17:02
@dangng2004 dangng2004 changed the title WIP: severity tier for review comments Severity tier for review comments May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant