Skip to content

feat: support max_indels=2 for homogeneous insertions or deletions#30

Open
Roman-Young wants to merge 2 commits into
IEDB:masterfrom
Roman-Young:feature/indel-2-homogeneous
Open

feat: support max_indels=2 for homogeneous insertions or deletions#30
Roman-Young wants to merge 2 commits into
IEDB:masterfrom
Roman-Young:feature/indel-2-homogeneous

Conversation

@Roman-Young

Copy link
Copy Markdown
Contributor

Summary

Extends indel search to max_indels=2, restricted to homogeneous edits — two insertions or two deletions, never one of each. A mixed 1-insertion + 1-deletion pair is a substitution, which mismatch search already covers, so it is excluded (deferred to a follow-up PR). The two edits need not be adjacent — consecutive and non-consecutive both match. max_indels > 2 still raises.

Approach

Reuses the 1-indel bidirectional DFS, run as two homogeneous passes: dfs gained allow_del / allow_ins branch masks; extend_bidirectional runs a deletions-only pass and an insertions-only pass, unioned through the existing seen dedup. Fixing the mask across both extensions of a seed is what keeps each alignment homogeneous (masking per side would still let a left-deletion pair with a right-insertion). No prev_was_* substitution guards are needed — mixing is structurally impossible within a single pass. 1-indel output is bit-for-bit unchanged (at budget 1 only one edit can fire, so del-pass ∪ ins-pass equals the prior behavior), proven by the pre-existing suite.

Validation

  • The engine is differential-tested against an independent brute-force oracle (test_indel2_property.py, 15 randomized proteome/query configurations, run in CI) asserting exact result-set equality — catching missed hits, phantom hits, and homogeneity violations together.
  • The oracle is itself cross-validated against a third, algorithmically-independent edit-enumeration oracle (itertools.combinations + exact substring match) over 450,000 randomized trials with 0 disagreements, plus hand-verified barred/boundary cases and a separate annotation differential.
  • Full suite green (68 tests), including targeted cases for consecutive deletions, non-consecutive deletions, insertions, mixed-not-found, and 2-indel annotation (chunks, homopolymers, and the periodic-repeat case that must not collapse).

Known limitation (follow-up PR)

For 2 indels, k = max(2, min_len // 3), so query lengths ≤ 8 fall to k = 2. Two-mers are extremely common in a proteome, so seeding degenerates toward brute force and runtime blows up on short queries. Results stay correct — the pigeonhole guarantee still holds (3 seeds) — only performance is affected. A follow-up PR will address the k=2 short-query regime.

Notes

@dmx2
dmx2 self-requested a review July 18, 2026 03:59
A 2-indel match uses two insertions or two deletions, never one of each — a
mixed pair is a substitution, which mismatch search already covers. The two
edits need not be adjacent.

dfs gains allow_del/allow_ins branch masks; extend_bidirectional runs two
homogeneous passes (deletions-only, then insertions-only) and unions them via
the existing dedup. Fixing the mask across both extensions of a seed is what
keeps an alignment homogeneous — masking per-side would still let a
left-deletion pair with a right-insertion. No prev_was_* substitution guards
are needed because mixing is structurally impossible per pass.

1-indel output is unchanged: at budget 1 only one edit can fire, so the
deletion pass unioned with the insertion pass reproduces the previous
behavior. Validated against the committed brute-force oracle
(pepmatch/tests/indel2_brute_force.py).
The matrix is an explicit per-file allowlist, so test_indel2_property.py would
never execute in CI without a row of its own.
@Roman-Young
Roman-Young force-pushed the feature/indel-2-homogeneous branch from 718dd30 to 5fb940f Compare July 19, 2026 07:08

@dmx2 dmx2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, Roman. Approved pending a couple things.

The short-query claim is inaccurate. k = max(2, min_len//(edits+1)) floors k to 2, but the pigeonhole guarantee needs l ≥ k·(max_indels+1). Below that (min_len < 6 for 2 indels), two edits spoil every seed and matches are silently missed, like:

ABCDE → ACE and ABCD → AD.

This isn't a bug, the mismatch algorithm has the identical limit, and it's the correct inverse of our paper's formula with m → max_indels. Just warn user about it.

  • In the PR description: drop "results stay correct (pigeonhole, 3 seeds)"; document the real bound that it has (min_len ≥ k·(max_indels+1)), and noting it matches mismatch behavior.
  • Add a shared warning (used by mismatch and indel, not best_match) when the chosen k can't guarantee recall for some query lengths. Flag any query with len < k·(edits+1). Something like this helper function:
def _recall_warning(self, k, edits):
  unguaranteed = sorted({len(s) for _, s in self.query if len(s) < k * (edits + 1)})
  too_short    = sorted({len(s) for _, s in self.query if len(s) < k})
  if unguaranteed:
    print(f"k={k}, edits={edits}: complete recall is not guaranteed for this query "
            f"peptides shorter than {k*(edits+1)} residues (lengths: {unguaranteed}) "
            f"a match may exist but be missed.")

I should have done this for mismatching a long time ago, but didn't. I am also granting that you can touch the mismatching code here to give this warning as well. Same behavior as the indel search.

The reviewer agent also said for tests:

  • Nice-to-have: test_indel2_property.py:32 uses qlen 9–14, so k is always ≥3 and never exercises this regime — that's why it slipped through.
  • Add a few qlen 4–8 configs, and inject a repeat-rich protein into the fuzzed proteome (current ones are random, so no homopolymers/periodic
    repeats hit the engine).

Once these changes are made, it's ready to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants