[feature] Track skill provenance, usage, and health

[paudley]

Problem statement

coding-ethos generates skills and emits skill IDs in remediation payloads, but it does not yet measure whether a skill actually helped resolve an issue, whether a skill is stale, or whether a skill repeatedly appears in failed remediation loops.

ECC prior art worth porting: skill provenance, learned/imported/evolved placement policy, and skill health reporting. See:

• https://github.com/affaan-m/ECC
• https://raw.githubusercontent.com/affaan-m/ECC/main/docs/SKILL-PLACEMENT-POLICY.md

Proposed solution

Track skill provenance, usage, and health in the repo-local evidence/code-intel layer.

The first version should record:

• skill id and generated source/provenance
• triggering policy/tool/finding
• surface that recommended or used the skill
• outcome when observable: success, repeated failure, abandoned, unknown
• retry count and recurrence window
• last used time and recent success/failure trend

Add a skill health CLI/MCP view that reports unused, frequently failing, improving, and stale skills.

Acceptance criteria

• Skill use/outcome rows are recorded when remediation payloads or MCP skill tools are used.
• Health report includes 7-day/30-day or equivalent recent windows.
• Output is available as JSON and TOON.
• No learned/evolved skill is automatically promoted into generated repo artifacts without explicit future policy.
• Tests cover outcome recording, unknown skill ids, and health trend computation.

Implementation notes

Prefer code-intel tables over a separate store. Generated skills remain source-derived artifacts; this issue is about measurement and reporting, not automatic skill mutation.

[paudley]

Stage 1 plan:

1. Build skill health on the existing code-intel evidence store instead of creating a separate skill database. Use remediation outcome rows and skill tool observations as the v1 source of truth.
2. Add code-intel skill-health records with 7-day and 30-day windows, classifying unused, frequently failing, improving, stale, and unknown-skill states.
3. Add code-intel skill-health JSON/TOON output and an MCP code_intel_skill_health view.
4. Record observable MCP skill usage from skill_lookup and skill_recommend as unknown-outcome observations tied to the selected skill IDs and surface.
5. Cover outcome recording, unknown skill IDs, and health trend computation with focused Go tests, then run the repo gate.

[paudley]

Stage 1 complete.

Implemented skill provenance, usage, and health reporting in PR #257: #257

Commit: 5bb6418 feat(codeintel): add skill health reporting

Validation completed:

• go test ./internal/codeintel ./internal/codeintelcli ./internal/mcp
• ./bin/lint --staged
• make build
• make check

Note: make check and commit/push hooks pass with the existing non-blocking testing.go_coverage_goal warning.

[paudley]

Stage 2 update

PR #257 has been updated after review feedback:

• Addressed Gemini review feedback in af53138.
• Addressed CodeRabbit follow-up feedback in a29d660: summary counts now use the full unpaginated record set, known skill catalog membership is explicit, and telemetry write failures are logged without failing skill tools.
• Resolved all reviewed threads after verification.
• Validation: go test ./internal/codeintel ./internal/codeintelcli ./internal/mcp from the Go module root; ./bin/lint --staged; make check. The managed gates pass with the existing non-blocking testing.go_coverage_goal warning.

PR #257 is ready for merge once required review and pending checks complete.