feat: support max_indels=2 for homogeneous insertions or deletions#30
feat: support max_indels=2 for homogeneous insertions or deletions#30Roman-Young wants to merge 2 commits into
Conversation
A 2-indel match uses two insertions or two deletions, never one of each — a mixed pair is a substitution, which mismatch search already covers. The two edits need not be adjacent. dfs gains allow_del/allow_ins branch masks; extend_bidirectional runs two homogeneous passes (deletions-only, then insertions-only) and unions them via the existing dedup. Fixing the mask across both extensions of a seed is what keeps an alignment homogeneous — masking per-side would still let a left-deletion pair with a right-insertion. No prev_was_* substitution guards are needed because mixing is structurally impossible per pass. 1-indel output is unchanged: at budget 1 only one edit can fire, so the deletion pass unioned with the insertion pass reproduces the previous behavior. Validated against the committed brute-force oracle (pepmatch/tests/indel2_brute_force.py).
The matrix is an explicit per-file allowlist, so test_indel2_property.py would never execute in CI without a row of its own.
718dd30 to
5fb940f
Compare
There was a problem hiding this comment.
Nice work, Roman. Approved pending a couple things.
The short-query claim is inaccurate. k = max(2, min_len//(edits+1)) floors k to 2, but the pigeonhole guarantee needs l ≥ k·(max_indels+1). Below that (min_len < 6 for 2 indels), two edits spoil every seed and matches are silently missed, like:
ABCDE → ACE and ABCD → AD.
This isn't a bug, the mismatch algorithm has the identical limit, and it's the correct inverse of our paper's formula with m → max_indels. Just warn user about it.
- In the PR description: drop "results stay correct (pigeonhole, 3 seeds)"; document the real bound that it has (min_len ≥ k·(max_indels+1)), and noting it matches mismatch behavior.
- Add a shared warning (used by mismatch and indel, not best_match) when the chosen k can't guarantee recall for some query lengths. Flag any query with len < k·(edits+1). Something like this helper function:
def _recall_warning(self, k, edits):
unguaranteed = sorted({len(s) for _, s in self.query if len(s) < k * (edits + 1)})
too_short = sorted({len(s) for _, s in self.query if len(s) < k})
if unguaranteed:
print(f"k={k}, edits={edits}: complete recall is not guaranteed for this query "
f"peptides shorter than {k*(edits+1)} residues (lengths: {unguaranteed}) "
f"a match may exist but be missed.")I should have done this for mismatching a long time ago, but didn't. I am also granting that you can touch the mismatching code here to give this warning as well. Same behavior as the indel search.
The reviewer agent also said for tests:
- Nice-to-have: test_indel2_property.py:32 uses qlen 9–14, so k is always ≥3 and never exercises this regime — that's why it slipped through.
- Add a few qlen 4–8 configs, and inject a repeat-rich protein into the fuzzed proteome (current ones are random, so no homopolymers/periodic
repeats hit the engine).
Once these changes are made, it's ready to merge.
Summary
Extends indel search to
max_indels=2, restricted to homogeneous edits — two insertions or two deletions, never one of each. A mixed 1-insertion + 1-deletion pair is a substitution, which mismatch search already covers, so it is excluded (deferred to a follow-up PR). The two edits need not be adjacent — consecutive and non-consecutive both match.max_indels > 2still raises.Approach
Reuses the 1-indel bidirectional DFS, run as two homogeneous passes:
dfsgainedallow_del/allow_insbranch masks;extend_bidirectionalruns a deletions-only pass and an insertions-only pass, unioned through the existingseendedup. Fixing the mask across both extensions of a seed is what keeps each alignment homogeneous (masking per side would still let a left-deletion pair with a right-insertion). Noprev_was_*substitution guards are needed — mixing is structurally impossible within a single pass. 1-indel output is bit-for-bit unchanged (at budget 1 only one edit can fire, so del-pass ∪ ins-pass equals the prior behavior), proven by the pre-existing suite.Validation
test_indel2_property.py, 15 randomized proteome/query configurations, run in CI) asserting exact result-set equality — catching missed hits, phantom hits, and homogeneity violations together.itertools.combinations+ exact substring match) over 450,000 randomized trials with 0 disagreements, plus hand-verified barred/boundary cases and a separate annotation differential.Known limitation (follow-up PR)
For 2 indels,
k = max(2, min_len // 3), so query lengths ≤ 8 fall tok = 2. Two-mers are extremely common in a proteome, so seeding degenerates toward brute force and runtime blows up on short queries. Results stay correct — the pigeonhole guarantee still holds (3 seeds) — only performance is affected. A follow-up PR will address the k=2 short-query regime.Notes
k-handling change in fix: honor explicit k in indel search, clamp above the pigeonhole optimal #29 (Issue indel matching still optimizes for k even if k is explicitly passed #28).