Skip to content

feat(skills): add gap-analysis verification-gate skill + evals#14

Merged
kreneskyp merged 1 commit into
mainfrom
feat/gap-analysis-skill
Jun 22, 2026
Merged

feat(skills): add gap-analysis verification-gate skill + evals#14
kreneskyp merged 1 commit into
mainfrom
feat/gap-analysis-skill

Conversation

@kreneskyp

Copy link
Copy Markdown
Contributor

What

A new gap-analysis skill — a post-implementation verification gate over one targeted plan — plus an eval suite.

Skill (skills/gap-analysis/): SKILL.md + 6 step references. It checks:

  1. Plan completion — every Task is status: done
  2. Test-matrix verification — each matrix TC-xxx is backed by a real tracking tag in the tests
  3. Underspecified code — code/behavior with no owning StR/US/FR (reverse gap) + stub detection
  4. (optional) Semantic review — intent↔test↔code agreement (user-gated)

Output: one quire-validated SpecReview (analysis: gap-analysis) at reviews/YY-MM-DD-<slug>.md with a Verdict (PASS/CONDITIONAL/FAIL) and a findings table.

Evals (evals/): EV-030..EV-033 cover happy + sad branches via a shared seedGapBundle fixture, plus a new fileContains assertion (evals/lib/assert.mjs) that checks the actual Verdict/findings:

Eval Path
EV-030 sad (combined) → FAIL
EV-031 happy → non-blocking verdict (not FAIL)
EV-032 sad (medium-only) → untraced code flagged, CONDITIONAL
EV-033 optional semantic review catches a hollow test

Validation

  • make test green (102/102); make lint clean for the changed files.
  • All four evals run 4/4 PASS locally (sonnet, 0 validation rejections).

Dependency

Requires agent-ix/spec-artifacts-process v0.4.0 (the gap-analysis analysis enum — agent-ix/spec-artifacts-process#5). The evals are RED-in-CI until v0.4.0 is tagged and the default-modules.yaml pin is bumped to it (same release-coupling as EV-026); proven green locally by temporarily sourcing the module from the local working tree.

🤖 Generated with Claude Code

New `gap-analysis` skill: a post-implementation gate over one targeted
plan. It verifies (1) every task is done, (2) each Test Matrix TC is
backed by a real tracking tag in the tests, (3) code is fully traced to
spec (flagging underspecified code), with an optional (4) semantic
review of intent<->test<->code. Emits one quire-validated SpecReview
(analysis: gap-analysis) to reviews/YY-MM-DD-<slug>.md with a Verdict
(PASS/CONDITIONAL/FAIL) and a findings table.

Evals EV-030..EV-033 cover the happy + sad branches (combined FAIL,
clean not-FAIL, medium-only CONDITIONAL, optional semantic catches a
hollow test) via a shared seedGapBundle fixture, plus a new
`fileContains` assertion that checks the actual Verdict/findings.

Requires spec-artifacts-process v0.4.0 (the gap-analysis analysis enum).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kreneskyp kreneskyp requested a review from a team as a code owner June 22, 2026 15:10
@kreneskyp kreneskyp merged commit 8937f5e into main Jun 22, 2026
5 of 6 checks passed
@kreneskyp kreneskyp deleted the feat/gap-analysis-skill branch June 22, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant