Skip to content

ci(coverage): establish test-coverage baseline for Node/Vite apps in src/500-application #653

Description

@katriendg

Problem Statement

PR #633 introduced the first Node/TypeScript application (Vite/esbuild + Vitest) under src/500-application/: 516-chat-with-your-factory. The app already ships Vitest with the @vitest/coverage-v8 provider and passing unit tests, but nothing in CI runs those tests, enforces them, or reports coverage to Codecov.

Rust crates in the same folder have a complete coverage + registration + enforcement contract (see .github/instructions/rust-crate-registration.instructions.md). Node/Vite apps have no equivalent, so unit tests can silently rot and coverage is invisible. This gap will repeat for every future Node/Vite app unless a shared baseline is established.

Current State (gap analysis)

Concern Rust (exists) Node/Vite (missing)
Test + coverage runner in CI rust-tests.yml matrix runs cargo llvm-cov → Codecov (flags: rust) No workflow runs npm test / vitest --coverage for 500-application apps (docusaurus-tests.yml is docs-only)
PR gating / change detection rust-tests gated by changesInRust from scripts/build/Detect-Folder-Changes.ps1 application-matrix-builds.yml only builds Docker images; never runs unit tests/coverage
Codecov flag + paths codecov.yml has flags.rust.paths + project/patch status No js/node flag, paths, or coverage status; 516 not registered
Registration enforcement validate-rust-registration.yml + Validate-RustCrateRegistration.ps1 No equivalent gate for Node packages
Authoring docs / instructions rust-crate-registration.instructions.md No instruction file for registering a Node app

What 516 already has (local only)

  • test, test:watch, test:coverage scripts and @vitest/coverage-v8 in package.json
  • A vitest.config.ts with a v8 coverage provider (default text reporter only — no Cobertura/lcov output)
  • At least one real test: src/server/factoryTool.test.ts

Proposed Solution

Build a generic baseline for all Vite/Node apps under src/500-application/, not a one-off for 516.

  1. Reusable Node test/coverage workflow (e.g. .github/workflows/node-tests.yml)

    • Matrix over registered Node packages under src/500-application/
    • npm cinpm run test:coverage → upload Cobertura/lcov to Codecov under a new js (or node) flag
    • Standardize a coverage reporter (Cobertura/lcov) in each app's vitest.config.ts (516 currently emits only the default text reporter) and a consistent test:coverage script convention
  2. PR gating in pr-validation.yml

    • Add a changesInNode-style output to scripts/build/Detect-Folder-Changes.ps1 so the workflow runs only when Node app files (or the workflow/codecov config) change
  3. Codecov registration in codecov.yml

    • Add flags.js with per-app paths, plus coverage.status.project/patch entries, or an explicit ignore opt-out path — matching the Rust flag model
  4. Registration enforcement gate

    • New validator workflow + script (mirroring Validate-RustCrateRegistration.ps1) that fails a PR when a Vite/Node app under src/500-application/ is neither fully registered (workflow matrix + codecov flag) nor explicitly opted out
  5. Authoring instructions

    • New node-package-registration.instructions.md with applyTo: **/src/500-application/**/package.json (+ workflow/codecov globs), documenting the registration/opt-out contract, analogous to the Rust instructions

Benefits

  • Node/Vite apps get the same coverage visibility and enforcement guarantees as Rust crates
  • Prevents silent test rot and untracked coverage as more Node apps land under src/500-application/
  • Establishes a documented, self-service registration path for future app authors

Acceptance Criteria

  • Reusable Node test/coverage workflow runs vitest with coverage for registered apps and uploads to Codecov under a dedicated flag
  • 516-chat-with-your-factory is registered and reporting coverage as the first consumer
  • Node tests are PR-gated on Node-relevant changes via Detect-Folder-Changes.ps1
  • codecov.yml has a Node/JS flag with paths + project/patch status (and an opt-out path)
  • A CI gate fails PRs that add an unregistered/opted-out Vite/Node app
  • Instruction file documents the registration and opt-out process
  • A standard test:coverage script + coverage reporter convention is documented for new apps

Notes / Open Questions

  • Flag naming: reuse js to align with the existing fuzz-js-* scheme, or use node?
  • Node version: standardize on 24 (used across existing workflows) vs. per-app engines
  • Should coverage thresholds start informational: true (like Rust patch) to avoid blocking early adopters?

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestjavascriptPull requests that update javascript codetestingTest infrastructure and test filestoolingDeveloper tooling and utilities

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions