fix(coverage): never let a partial run shrink the baseline; UNMEASURED vs DROPPED#20
Merged
Merged
Conversation
…tected-but-unmeasured languages as UNMEASURED A path-filtered CI run (wardnet/wardnet#892: Rust-only PR skips the frontend coverage job) left typescript unmeasured, so the gate printed the misleading "[DROPPED] typescript: no longer measured" — and worse, the same partial report recorded on main (or cached on a PR cache miss) would silently delete the language's baseline entry, un-gating it permanently. The empty-baseline guard doesn't catch this: the report isn't empty, just partial. Both baseline writers (record-on-main and computeBaselineAt's cache-miss path) now carry detected-but-unmeasured languages forward from the nearest prior baseline via gitstate.PriorBaselines, which starts at the recorded SHA itself — so re-runs and concurrent per-language jobs never clobber a fresher same-commit entry, and a depth-1 checkout still sees it — before walking first-parent ancestors, skipping poisoned {} entries and stopping once every needed language is filled. Anything no prior had is named in a stderr warning instead of vanishing. A language that is genuinely gone (source deleted, or enabled: false — now honored via enabledEcosystems) stays undetected and its entry dies with the next record. diffReport splits the old DROPPED case: still-detected languages get "[UNMEASURED] <lang>: not measured this run (baseline X%)", reserving DROPPED for source that actually left the tree. action.yml's cov_detail grep becomes tag-generic so the new tag (and any future one) reaches the PR comment instead of being silently filtered. Claude-Session: https://claude.ai/code/session_01QMTWVM4pUB4xy6F6P9wWYA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
wardnet/wardnet#892 (a Rust-only PR) showed
typescript: no longer measured (baseline was 93.8%)even though no TypeScript code was touched — wardnet's CI path-filters its coverage jobs, so the frontend coverage job was skipped and bulwark saw no TS report. Two defects behind that symptom:[NEW], gated against nothing, permanently. The "never cache an empty baseline" guard doesn't catch this — the report isn't empty, just partial.Fix
carryForwardBaselinecopies each detected-but-unmeasured language's entry from the nearest prior baseline via the newgitstate.PriorBaselines, which starts at the recorded SHA itself (a re-run or concurrent per-language job never clobbers a fresher same-commit entry; works even on depth-1 checkouts) before walking first-parent ancestors, skipping poisoned{}entries and stopping once every needed language is filled.recorded coverage baselineline names anything carried.enabled: falsein.bulwark.yml, now actually honored viaenabledEcosystems— is never carried; its entry dies on the next record.[UNMEASURED]vs[DROPPED].diffReportreports a still-detected-but-unmeasured language as[UNMEASURED] <lang>: not measured this run (baseline X%), reserving[DROPPED]for source that actually left the tree. Neither fails the gate on its own.action.yml— the enumeratedPASS|FAIL|NEW|DROPPEDlist would have silently filtered the new tag out of the sticky comment.Testing
Test-first throughout:
TestPriorBaselinesNearestCommitWinsPerLanguage(real git fixture: same-SHA entry wins, per-language nearest fill, empty-entry skip, maxDepth, missing-branch tolerance),TestUnmeasuredLanguages,TestMergeCarried,TestEnabledEcosystemsDropsDisabledLanguages,TestDiffReportUnmeasuredWhenLanguageStillDetected(reproduces #892's report shape). Fullgo test -race ./...,go vet,gofmt,golangci-lintclean. Thecov_detailpipeline was simulated against real gate output to confirm[UNMEASURED]reaches the PR comment.Merge Commit Message
fix(coverage): carry detected-but-unmeasured languages forward in baseline writes (never shrink on partial runs) and report them as [UNMEASURED] instead of [DROPPED]
https://claude.ai/code/session_01QMTWVM4pUB4xy6F6P9wWYA