Skip to content

fix(core): Emit structured warning for skipped zero-baseline metrics - #26

Open
khatayondev wants to merge 1 commit into
flameiq:v2-devfrom
khatayondev:main
Open

fix(core): Emit structured warning for skipped zero-baseline metrics#26
khatayondev wants to merge 1 commit into
flameiq:v2-devfrom
khatayondev:main

Conversation

@khatayondev

@khatayondev khatayondev commented Aug 22, 2026

Copy link
Copy Markdown

Description

This PR resolves Issue #13 where metrics with a 0.0 baseline were being silently skipped during comparison, resulting in them being completely missing from the final JSON report and console summary.

What was changed?

  • Created a new SkippedMetric data class model to explicitly track skipped metrics.
  • Updated the ComparisonResult to include a skipped list.
  • Modified the compare_snapshots logic in flameiq/core/comparator.py to catch ComparisonError exceptions (thrown by compute_change_percent) and append the skipped metric to the skipped list with a detailed reason, rather than just ignoring it.
  • Updated to_dict() so the skipped metrics are included in the serialized JSON output.
  • Updated the console summary to explicitly state if metrics were skipped (e.g. All 2 metric(s) within threshold. (1 skipped)).

Testing

  • ✅ Added a new unit test test_zero_baseline_skipped_metrics to tests/unit/test_core/test_comparator.py.
  • ✅ All 145 tests pass locally via make test.

Closes #13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zero-baseline guard silently skips metrics — should emit a structured warning in JSON output

1 participant