Fix N+1 queries in grids, views and tasks #1723

[davmlaw]

🤖 Written by Claude.

Addresses part of #1723 — the fixes that are just select_related/prefetch_related or hoisting a query out of a loop.

Allele Groupings grid

The worst of them. render_allele runs a ClassificationGrouping query per row, then c_hgvs_for walked latest_classification_modification -> classification -> allele_info -> allele_info[gb] -> ri.genome_build as lazy FK loads for every grouping of every row.

ClassificationGrouping.update() sets latest_allele_info from best_classification.classification.allele_info in the same breath as latest_classification_modification, so the stored field is exactly what that chain resolves to — read it directly and select_related the resolved variant info for both builds.

The Labs column had the same problem from the other end: allele_origin_dict prefetched classificationgrouping_set but not lab, and sorted(labs) calls Lab.__lt__, which reaches for organization. Both now come through the prefetch.

DiscordanceReport.update()

Runs on every classification publish/withdraw touching a discordant clinical context. Two problems: classification_original.classification was dereferenced per row (two queries each, one of them the wide ClassificationModification), and each newly added classification was fetched with its own .get(). Now one select_related queryset for the existing rows and one bulk query for the added ones.

Behaviour note for review: the old .get(is_last_published=True, classification=vcm_id) raised DoesNotExist if a classification had no published modification, failing the whole update. The bulk query skips it instead. I think skipping is the better behaviour, but it is a change, so flagging it.

Variant details page

• The sample classifications table used latest_for_user, which applies no select_related, so each row refetched Classification — including the wide evidence JSONB — plus lab.
• The locus counts table fetched Variant rows without their locus/sequence rows, and str(v) then costs three queries each via locus.contig.name, locus.ref.seq and alt.seq.

Others

• The transcript table (variant page, classification autopopulate, classification detail) loaded gene_version lazily over a loop that pulls every transcript version for the variant's gene symbols — often 50-300 rows.
• update_all_dirty — update() dirties allele_origin_grouping immediately, so it is worth bringing along; this runs over tens of thousands of groupings after a bulk import.
• The nightly ClinVar prepare ran two COUNT queries per allele; collapsed into one aggregate with a filtered Count.

Not included

The remaining grid items in the issue (imported_allele_info_view, clinvar_export_view, the AlleleLiftover grids, the variant tag can_write) all need the pre_render(qs) hook to bulk-load a page's objects into a dict. That is a consistent little pattern rather than a one-line change, so it is better as its own PR than bolted onto this one.

Testing

python3 manage.py test --keepdb classification.tests snpdb.tests annotation.tests — 649 tests pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LC9Z2HMXw1FFbmvSLjcJXz

[davmlaw]

🤖 Written by Claude.

Split into two per-app PRs, together identical to this diff:

• Classification: fix N+1 queries in the allele groupings grid and tasks #1729 #1731 — classification (allele groupings grid, DiscordanceReport.update(), update_all_dirty, ClinVar prepare counts) against N+1 queries in classification grids and tasks #1729
• snpdb/annotation: fix N+1 queries on the variant details page #1730 #1732 — snpdb/annotation (variant details page sample + locus counts tables, transcript table gene_version) against N+1 queries in snpdb/annotation/variantopedia grids and views #1730

This PR is superseded by those two.