Phase 2: assembly version summary generation

[fchen13]

Adds flows/lib/generate_assembly_summary.py, a post-parse aggregation step that combines assembly_current.tsv and assembly_historical.tsv into a per-base-accession version summary (assembly_version_summary.tsv). Emits generate.assembly_summary.completed for downstream Phase 3 taxon milestone flows to trigger from.

Also adds phase-specific run guides (tests/README_phase_0_backfill.md, tests/README_phase_1_daily_updates.md, tests/README_phase_2_assembly_summary.md) and removes the assembly version walkthrough from flows/README.md — that content now lives in the phase-specific READMEs.

Summary by Sourcery

Generate a per-base-accession assembly version summary from current and historical TSVs and document the multi-phase assembly version pipeline.

New Features:

• Add a flow to aggregate assembly_current.tsv and assembly_historical.tsv into assembly_version_summary.tsv and emit a completion event for downstream Phase 3 taxon milestone flows.

Documentation:

• Add phase-specific README guides for Phase 0 backfill, Phase 1 daily updates, and Phase 2 assembly summary, and remove the assembly version walkthrough from the main flows README.

[sourcery-ai]

Reviewer's Guide

Implements a new Phase 2 aggregation flow that reads assembly_current.tsv and assembly_historical.tsv, normalizes accessions, computes per-base-accession version history metrics, writes assembly_version_summary.tsv, emits a completion event for downstream flows, and refactors documentation into phase-specific README guides for backfill, daily updates, and assembly summary while removing the old walkthrough from flows/README.md.

Flow diagram for Phase 2 assembly version summary generation

flowchart LR
  current[assembly_current.tsv]
  hist[assembly_historical.tsv]
  flow_node[generate_assembly_summary flow]
  summary[assembly_version_summary.tsv]
  event[generate.assembly_summary.completed]

  current --> flow_node
  hist --> flow_node
  flow_node --> summary
  flow_node --> event

Loading

File-Level Changes

Change	Details	Files
Add Phase 2 aggregation flow that builds a per-base-accession assembly_version_summary.tsv from current and historical TSVs and emits a completion event for downstream orchestration.	Introduce generate_assembly_summary flow that loads assembly_current.tsv and assembly_historical.tsv from a work directory and writes assembly_version_summary.tsv. Normalize accession fields from differing column names (accession vs genbankAccession) when loading TSV rows so downstream logic can assume a consistent key. Group rows by base accession using existing accession parsing, sort by version, and compute summary metrics including first/current version, counts, superseded versions, first EBP metric, and version gaps. Detect EBP metric presence using multiple possible column names for date fields following different naming conventions. Emit a structured completion event with summary statistics via conditional_import.emit_event for downstream Phase 3 milestone flows, and provide a CLI entrypoint using shared_args.WORK_DIR.	flows/lib/generate_assembly_summary.py
Restructure assembly version tracking documentation into phase-specific run and test guides and remove the detailed assembly version walkthrough from the main flows README.	Remove the monolithic assembly version tracking walkthrough from flows/README.md to declutter the main documentation. Add a Phase 0 backfill README describing prerequisites, CLI usage, and how to run tests and a small full backfill over fixture data. Add a Phase 1 daily updates README that documents the incremental parse and updater, their data flow, a minimal end-to-end walkthrough, and how to run tests. Add a Phase 2 assembly summary README that explains the new aggregation step, its position in the pipeline, expected inputs/outputs, example run, and test notes.	flows/README.md tests/README_phase_0_backfill.md tests/README_phase_1_daily_updates.md tests/README_phase_2_assembly_summary.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• The Phase 2 README and pipeline description still refer to aggregate_assembly_summary.py and the aggregate.assembly_summary.completed event, but the implementation is generate_assembly_summary.py emitting generate.assembly_summary.completed; aligning filenames and event names between code and docs will avoid confusion and miswired triggers.
• The helper _get_accession() is currently unused and the accession-normalisation logic is duplicated in load_assemblies; consider either using _get_accession() there or removing it to keep the module minimal and consistent.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The Phase 2 README and pipeline description still refer to `aggregate_assembly_summary.py` and the `aggregate.assembly_summary.completed` event, but the implementation is `generate_assembly_summary.py` emitting `generate.assembly_summary.completed`; aligning filenames and event names between code and docs will avoid confusion and miswired triggers.
- The helper `_get_accession()` is currently unused and the accession-normalisation logic is duplicated in `load_assemblies`; consider either using `_get_accession()` there or removing it to keep the module minimal and consistent.

## Individual Comments

### Comment 1
<location path="flows/lib/generate_assembly_summary.py" line_range="32-39" />
<code_context>
+EBP_METRIC_COLUMNS = {"ebpStandardDate", "ebp_standard_date", "ebpMetricDate", "ebp_metric_date"}
+
+
+def _get_accession(row: dict) -> str:
+    """Return the accession value from a row, handling both column name conventions.
+
+    assembly_current.tsv uses 'accession'; assembly_historical.tsv written by
+    the Phase 0 backfill uses 'genbankAccession'.  Both may appear in the
+    merged historical TSV depending on which phase wrote each row.
+    """
+    return row.get("accession") or row.get("genbankAccession") or ""
+
+
</code_context>
<issue_to_address>
**suggestion:** Consider removing or inlining `_get_accession` since it is unused and duplicates logic in `load_assemblies`.

This helper duplicates the accession normalisation logic already in `load_assemblies` and is never called. To reduce redundancy and avoid confusion about the preferred access pattern, either remove `_get_accession` or update `load_assemblies` to use it consistently.

Suggested implementation:

```python
EBP_METRIC_COLUMNS = {"ebpStandardDate", "ebp_standard_date", "ebpMetricDate", "ebp_metric_date"}

```

The helper `_get_accession` is currently unused and duplicates logic in `load_assemblies`, so removing it is sufficient to reduce redundancy and avoid confusion. No further changes are required unless you want to centralise accession normalisation in a shared helper, in which case `load_assemblies` would need to be updated to call that helper consistently.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}