Gremlins mutation testing tracker

[cplieger]

Gremlins mutation testing tracker

Auto-updated weekly by cplieger/ci/.github/workflows/weekly-gremlins.yaml.
Last update: 2026-08-31 07:56

This week: 97.2% efficacy (±0.0% across 3 runs), 97.5% mutant coverage. 10 confirmed live mutants.
Trend: ↘ -0.6% from 12-week mean (97.8%).

Rolling 12-week history

Run (UTC)	Mean efficacy	Stddev	Mutant coverage	Live mutants	Δ efficacy
2026-08-31 07:56	97.2%	±0.0%	97.5%	10	+0.0%
2026-08-24 05:00	97.2%	±0.0%	97.5%	10	+5.7%
2026-08-21 15:35	91.5%	±0.0%	97.5%	30	-7.2%
2026-07-13 02:16	98.7%	±0.0%	96.8%	4	-0.3%
2026-07-07 23:28	99.0%	±0.0%	98.7%	3	+2.6%
2026-06-29 19:29	96.4%	±0.0%	98.7%	11	-1.8%
2026-06-29 00:22	98.2%	±0.0%	96.6%	5	-1.1%
2026-06-22	99.3%	±0.0%	93.6%	2	+0.0%
2026-06-22	99.3%	±0.0%	93.6%	2	+0.0%
2026-06-18	99.3%	±0.0%	93.6%	2	+0.0%
2026-06-18	99.3%	±0.0%	93.6%	2	+1.8%
2026-06-17	97.5%	±0.0%	93.6%	7	+0.0%

Current live mutants (10, this week)

Solid gap — LIVED in all 3 runs (known coverage hole) — 10

internal/cache/cache.go

• L435 — CONDITIONALS_BOUNDARY

internal/cache/contract.go

• L75 — ARITHMETIC_BASE
• L75 — ARITHMETIC_BASE

internal/cache/processed.go

• L24 — CONDITIONALS_BOUNDARY
• L44 — CONDITIONALS_BOUNDARY
• L70 — CONDITIONALS_BOUNDARY

internal/deepscan/deepscan.go

• L188 — CONDITIONALS_BOUNDARY
• L271 — CONDITIONALS_BOUNDARY
• L277 — CONDITIONALS_BOUNDARY

internal/streams/score.go

• L38 — CONDITIONALS_BOUNDARY

How to read

• Mean efficacy: % of runnable mutants killed (or timed-out, treated as caught), averaged across the N runs
• Stddev: variance across runs — high stddev (>3%) signals flaky tests
• Mutant coverage: % of mutants reached by the test suite (test depth)

The "Current live mutants" section is bucketed by how many of the N runs the
mutant LIVED in (N = attempts that week, normally 3):

• Rare flake (1/N): a mutant your tests usually KILL but occasionally let
  through. Most actionable — almost always means a flaky test that's not
  reliable. Open the section to see the file:line; expect to fix the test, not
  the production code.
• Weakly flaky (k/N for 1<k<N): tests catch this mutant inconsistently.
  Less ideal than rare-flake but still worth investigating.
• Solid gap (N/N): every run lets this mutant through. Either there's no
  test exercising the path, or the test asserts the wrong thing. Action: add a
  test or strengthen an existing assertion.

The mutation-regression label is added when this week's mean efficacy drops

5% below the rolling 12-week mean.

A ⚠️ line under "This week" means the number describes the measurement, not the
suite: either the attempts disagreed about which mutants survive (one verdict per
disagreement is false), or the score jumped to a flawless 100% from a week with
live mutants. Equivalent mutants are a permanent noise floor, so a reproducible
100% is not something a test suite can reach.

Free-form notes

Add anything below — won't be touched by the auto-updater.