Skip to content

Improve observability: add structured logging to tournament, analyze, and verify_parity - #158

Draft
lukeinglis wants to merge 1 commit into
mainfrom
builder/improve-observability-coverage
Draft

Improve observability: add structured logging to tournament, analyze, and verify_parity#158
lukeinglis wants to merge 1 commit into
mainfrom
builder/improve-observability-coverage

Conversation

@lukeinglis

Copy link
Copy Markdown
Owner

Closes #157

Changes

  • tournament.py: Added structlog logger and logger.info() calls to load_tournament_results, model_source_matrix, paired_bootstrap_test, pairwise_comparison_table, cost_accuracy_summary, and tournament_report
  • analyze.py: Added logger.info() calls to analyze_by_source, analyze_calibration, analyze_biases, analyze_decomposition, save_analysis, analyze_worst_questions, analyze_by_horizon, compare_paired, and compare_to_leaderboard
  • verify_parity.py: Added structlog logger and logger.info() calls to all 12 check functions
  • eval.py: Added logging to save_result, load_previous_results, split_held_out, list_rounds, and print_leaderboard_comparison
  • lab_forecaster.py: Added logger.warning() to _parse_probability failure path and logger.debug()/logger.warning() to _extract_probabilities
  • tests/test_logging_config.py: Added test verifying newly instrumented modules expose module-level loggers; expanded smoke test list to include analyze, tournament, verify_parity

Score impact

Metric Before After
Observability score 0.583 0.659
Logging coverage 18% (43/236) 31% (74/236)
Log density 40% 56%
Tests 699 passed 700 passed
Lint clean clean
Type check clean clean

…nd lab_forecaster

Increases observability coverage from 18% to 31% (43→74 logged functions)
and log density from 40% to 56%. All logging uses the existing structlog
infrastructure via get_logger(). Adds test verifying newly instrumented
modules expose module-level loggers.

Closes #157

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lukeinglis

Copy link
Copy Markdown
Owner Author

Factory CEO Verdict: KEEP ✅

Deep-QA Pipeline Results

Gate Result Detail
Health Check PASS Composite: 0.9716, Δ +0.2116
Code Review CLEAN (7/7) All categories pass
Adversarial Test PASS (7/7) All tests verified with evidence

Score Impact

Metric Before After Delta
Composite 0.760 0.972 +0.212
Observability 0.583 0.659 +0.076
Tests 700 pass 700 pass 0
Lint clean clean
Type check clean clean

Verification Evidence

  • Health Checker: 700 tests pass, lint clean, type check clean. Composite 0.9716 well above 0.55 threshold.
  • Code Reviewer: 7/7 categories PASS — correctness, security, edge cases, missing tests, style, scope, guardrails. Pure instrumentation with no logic changes.
  • Adversarial Tester: 7/7 tests VERIFIED with command+output evidence — imports, logging output, regression, lint, type check, edge case (parse_probability warning), no data leakage.

Decision Rationale

Clean, well-scoped observability improvement. 31 new structlog calls across 5 modules with zero logic changes. All QA signals green. Positive score delta. No regressions, no security concerns, no scope creep.

Approved for human merge.


Factory CEO — Experiment #4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve observability: add structured logging to tournament, analyze, and verify_parity

1 participant