Skip to content

Test debt: persist and improve skill-validation fixtures #5

Description

@chrispezza

The blind-validation fixtures for a11y-audit, deps-audit, and perf-audit lived in session-scratch space and are gone after session cleanup; re-validation after any skill edit means rebuilding them. Two sub-tasks:

  1. Decide a home for reusable fixtures (e.g. a fixtures/ dir in this repo, excluded from plugin payloads, or a dedicated fixtures repo) and recreate the three twin-sets there.
  2. Fix the realism gaps both validation runs flagged (test-side, not skill-side):
    • deps fixtures: hand-written lockfiles lacked transitive trees and integrity hashes, and an orphan send@2.0.0/ms subtree (carrying a real CVE) existed in BOTH twins — use npm-generated lockfiles and remove orphans from the fixed twin.
    • perf fixtures: CI trigger was push: branches: [main] in BOTH twins, so the fixed twin still drew a (true) low finding about post-merge-only enforcement — the fixed twin should trigger on pull_request.

Seed definitions are recorded in the v0.10.0 and v0.11.0 release commit bodies.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions