Skip to content

Retire mcp-csharp-create, mcp-csharp-test, and mcp-csharp-publish skills#943

Open
leslierichardson95 wants to merge 1 commit into
mainfrom
leslierichardson95/retire-mcp-csharp-skills
Open

Retire mcp-csharp-create, mcp-csharp-test, and mcp-csharp-publish skills#943
leslierichardson95 wants to merge 1 commit into
mainfrom
leslierichardson95/retire-mcp-csharp-skills

Conversation

@leslierichardson95

Copy link
Copy Markdown
Contributor

What

Retires the remaining three mcp-csharp-* skills from the dotnet-ai plugin: mcp-csharp-create, mcp-csharp-test, and mcp-csharp-publish. Follows the cut of mcp-csharp-debug (#938) — this completes retirement of the family.

Why

Based on the cross-family skills-eval (#889) and the 30-day telemetry review (gist):

Skill Eval (impact / families ✓) Real demand
mcp-csharp-test +0.65 but 0/5 families pass 2 users, 0 with ≥5 invocations
mcp-csharp-publish +0.56 but 0/5 families pass 0 real users
mcp-csharp-create 0.40, wins only 2/5 (Sonnet, Haiku) and regresses both frontier models; avgN 2.6 (directional) absent from telemetry report

None has demonstrated real demand. The only positive signal is a low-confidence eval that would require a full 5-family sweep to validate, with no usage to justify the investment. Retiring now (reversible via git history) is the pragmatic call over speculative strengthening.

Changes

  • Delete plugins/dotnet-ai/skills/mcp-csharp-{create,test,publish}/ (SKILL.md + references)
  • Delete tests/dotnet-ai/mcp-csharp-{create,test,publish}/ (eval.yaml)
  • Remove the dotnet-ai CODEOWNERS block for the three skills and their tests
  • Repoint the skill-validator doc example (InvestigatingResults.md) from the now-deleted mcp-csharp-create to a surviving skill (technology-selection)

dotnet-ai retains technology-selection. plugin.json auto-discovers ./skills/, so no manifest change is needed. Verified no remaining mcp-csharp-* references repo-wide (git grep). Rebased on main after #938 merged, so there are no conflicts with the debug cut.

Refs #889. Supersedes #939, #940, #941 (the earlier strengthen/fix PRs, now obsolete).

Removes the remaining three mcp-csharp-* skills from the dotnet-ai plugin.
Following the cut of mcp-csharp-debug (#938), this retires the rest of the
family based on the cross-family skills-eval (#889) and the 30-day telemetry
review (https://gist.github.com/AbhitejJohn/db57f38051ebf4e7de5556a4bd98ec15):

- mcp-csharp-test: STRENGTHEN 0/5 families; 2 real users, 0 with >=5 invocations.
- mcp-csharp-publish: STRENGTHEN 0/5 families; 0 real users.
- mcp-csharp-create: wins only 2/5 families (Sonnet, Haiku) and REGRESSES both
  frontier models; impact 0.40 at avgN 2.6 (directional only); absent from the
  telemetry report. Validating it would require a full 5-family eval sweep with
  no evidence of real demand to justify the investment.

Changes:
- Delete plugins/dotnet-ai/skills/mcp-csharp-{create,test,publish}/ (SKILL.md + references)
- Delete tests/dotnet-ai/mcp-csharp-{create,test,publish}/ (eval.yaml)
- Remove the dotnet-ai CODEOWNERS block for the three skills and their tests
- Repoint the skill-validator doc example to a surviving skill (technology-selection)

dotnet-ai retains technology-selection. plugin.json auto-discovers ./skills/,
so no manifest change is needed. Verified no remaining mcp-csharp-* references
repo-wide (git grep).

Refs #889. Supersedes #939, #940, #941.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6a8ce111-58e5-48a7-9f7f-8de972bdc693
@github-actions

Copy link
Copy Markdown
Contributor

Skill Coverage Report

Plugin Skill Covered Coverage
dotnet-ai mcp-csharp-create error
dotnet-ai mcp-csharp-publish error
dotnet-ai mcp-csharp-test error

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Retires the remaining mcp-csharp-* skills from the dotnet-ai plugin by removing their skill content and associated eval coverage, and updates repo documentation/ownership to match.

Changes:

  • Removed the mcp-csharp-create, mcp-csharp-test, and mcp-csharp-publish skill directories (including reference docs).
  • Removed the corresponding tests/dotnet-ai/mcp-csharp-*/eval.yaml evaluation files.
  • Updated CODEOWNERS and the skill-validator “Investigating results” documentation to avoid referencing retired skills.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/dotnet-ai/mcp-csharp-test/eval.yaml Removed eval coverage for the retired mcp-csharp-test skill.
tests/dotnet-ai/mcp-csharp-publish/eval.yaml Removed eval coverage for the retired mcp-csharp-publish skill.
tests/dotnet-ai/mcp-csharp-create/eval.yaml Removed eval coverage for the retired mcp-csharp-create skill.
plugins/dotnet-ai/skills/mcp-csharp-test/SKILL.md Removed retired skill content.
plugins/dotnet-ai/skills/mcp-csharp-test/references/test-patterns.md Removed retired skill reference material.
plugins/dotnet-ai/skills/mcp-csharp-test/references/evaluations.md Removed retired skill reference material.
plugins/dotnet-ai/skills/mcp-csharp-publish/SKILL.md Removed retired skill content.
plugins/dotnet-ai/skills/mcp-csharp-publish/references/nuget-packaging.md Removed retired skill reference material.
plugins/dotnet-ai/skills/mcp-csharp-publish/references/mcp-registry.md Removed retired skill reference material.
plugins/dotnet-ai/skills/mcp-csharp-publish/references/docker-azure.md Removed retired skill reference material.
plugins/dotnet-ai/skills/mcp-csharp-create/SKILL.md Removed retired skill content.
plugins/dotnet-ai/skills/mcp-csharp-create/references/transport-config.md Removed retired skill reference material.
plugins/dotnet-ai/skills/mcp-csharp-create/references/api-patterns.md Removed retired skill reference material.
eng/skill-validator/src/docs/InvestigatingResults.md Updated example job name to reference a remaining skill (technology-selection).
.github/CODEOWNERS Removed CODEOWNERS entries for the retired skills/tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@AbhitejJohn AbhitejJohn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving but tagging @luisquintanilla to take a look as well.

@AbhitejJohn

Copy link
Copy Markdown
Collaborator

@leslierichardson95 , @luisquintanilla : If this does get merged in leaving technology-selection as the only skill in this plugin, would it make sense to move that over to dotnet-advanced plugin? If we foresee more skills in this space in the near term, it would make sense to keep dotnet-ai around, otherwise it would be great to consolidate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants