Liftover page - retry failed liftovers per tool #1273 - #1845
Open
davmlaw wants to merge 2 commits into
Open
Conversation
A tool that has ever errored on an allele/build was skipped forever by AlleleLiftover.get_failed_conversion_tools, so the liftover page's "Liftover variants" button, the allele page's "Create Variant" button and the admin action all silently did nothing for previously-failed alleles. Add retry_conversion_tools, subtracted from the per-allele failed set in _get_build_liftover_dicts. Earlier AlleleLiftover rows are kept as history - a retry that fails again lands as a new row under a new LiftoverRun. - Liftover page: a per-tool "Retry failed liftovers" button under each "Failed Liftover to <build>" heading, with the count of alleles it would re-attempt (Allele.failed_liftover_for_build) - liftover_alleles / liftover_allele_batch tasks take the tool value - "Create Variant" and the admin Liftover action retry every tool, so the button also appears for an allele all tools have failed on - manage.py liftover_alleles --retry-tool <value>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #1273 (plan:
claude/plans/1273_liftover_retry_failed_plan.md).Problem
AlleleLiftover.get_failed_conversion_tools()returns every tool that has ever errored on an allele/build, and_get_build_liftover_dicts()skips those permanently. After fixing a bug or config problem (bad chain file, missing bcftools binary, aLiftoverRun.error()that marked a whole batch failed) there was no way to re-run those alleles - the liftover page's "Liftover variants" button, the allele page's "Create Variant" button and the admin "Liftover" action all silently did nothing.The only workaround was deleting
AlleleLiftoverrows by hand in the admin, which throws away the error messages you need to diagnose the failure.Approach
The failed-tool memory is the right default (it stops a ClinGen-less allele being re-queried on every import), so this adds a deliberate per-tool override rather than removing it.
retry_conversion_toolsis subtracted from each allele's failed set in_get_build_liftover_dicts; earlier failures are kept as history and a retry that fails again lands as a newAlleleLiftoverunder a newLiftoverRun.Alleles that have since acquired a variant in the destination build are already excluded by the existing
existing_buildscheck, so the retry queryset can be generous.Changes
snpdb/liftover.pyretry_conversion_tools=oncreate_liftover_pipelines,_create_liftover_pipelines_for_batch,_get_build_liftover_dicts,liftover_alleles,allele_can_attempt_liftoversnpdb/models/models_variant.pyAllele.failed_liftover_for_build(build, tool)- still-missing alleles that tool has errored onsnpdb/tasks/liftover_tasks.pyretry_conversion_tool(enum value, so it survives the Celery message) through both tasks;_alleles_to_liftover()shared by the batcher and the batch re-querysnpdb/views/views_liftover.pyretry_{build}_{tool}buttons, addsretry_countscontextsnpdb/templates/snpdb/liftover/liftover_runs.htmlvariantopedia/views_allele.py,classification/variant_card.py,snpdb/admin.pysnpdb/management/commands/liftover_alleles.py--retry-tool {SC,CA,DB,NR,PC,CM,BL}for kicking a retry from a shell after a deployAlso a gotcha line in
snpdb/CLAUDE.mdand a changelog entry. No migration - no model fields change.Testing
--keepdb --parallel 4)snpdb/tests/test_liftover.py: the retry override on_get_build_liftover_dicts(the retried tool comes back, other failed tools stay skipped),failed_liftover_for_build, andallele_can_attempt_liftoverwith retryvg page /snpdb/liftover --as admin --queriesagainst live data on vg-test2: renders, 18 production queries (3 added, one per build). Real counts e.g. GRCh38 - ClinGen Allele Registry 97, BCFtools/liftover 10scripts/vg map --checkandscripts/vg docs checkpass🤖 Generated with Claude Code
https://claude.ai/code/session_01XeLN2A486UE31zfdk9SrRG