Skip to content

optimise batch report#475

Open
rhaegar325 wants to merge 2 commits into
mainfrom
optimise_batchprocessing_log_info
Open

optimise batch report#475
rhaegar325 wants to merge 2 commits into
mainfrom
optimise_batchprocessing_log_info

Conversation

@rhaegar325

Copy link
Copy Markdown
Collaborator

Improve JSON readability of batch report failures

Background

In the moppy_batch_report.json produced by batch_processing, each entry in the failures section is hard to read: both error_message and stderr_tail hold multi-line text. Once JSON-escaped (newlines become \n), they collapse into a single very long line.

Changes

Multi-line fields in failure entries are now emitted as arrays of lines, so that with json.dumps(indent=2) each line renders on its own row.

  • Add _as_lines() helper: empty → None, single line → plain string, multi-line → list of lines.
  • Change _stderr_tail return type to str | list[str] | None (string for a single line, list for multiple lines); no longer uses "\n".join.
  • Apply _as_lines to error_message in both tasks and failures, keeping the field type consistent.
  • Bump SCHEMA_VERSION from v1 to v2 (field shape changed).

Before / After

Before (one very long line):

"stderr_tail": "line1\nline2\nline3\n...(20 lines)...",
"error_message": "job 1 | exit_status=1 | err_tail:\nTraceback\n  boom"

After (one line per entry):

"stderr_tail": [
  "line1",
  "line2",
  "line3"
],
"error_message": [
  "job 1 | exit_status=1 | err_tail:",
  "Traceback",
  "  boom"
]

Short messages (e.g. "input missing") stay plain strings.

Compatibility

  • The dashboard reads directly from the database and does not parse this JSON, so the field type change does not affect other functionality.
  • Existing report files are not updated automatically; regenerate to get the new format.

Testing

  • Updated stderr_tail and schema-version assertions.
  • Added _as_lines behavior test and a multi-line error_message splitting test.
  • tests/unit/test_batch_report.py: 25 passed.

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.4%. Comparing base (ed7102d) to head (61c9189).

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #475   +/-   ##
=====================================
  Coverage   78.4%   78.4%           
=====================================
  Files         33      33           
  Lines       6483    6491    +8     
  Branches    1226    1228    +2     
=====================================
+ Hits        5082    5090    +8     
  Misses      1128    1128           
  Partials     273     273           
Flag Coverage Δ
unit 78.4% <100.0%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant