Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .specify/feature.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"feature_directory": "specs/006-publish-npmjs"
"feature_directory": "specs/007-ai-support"
}
33 changes: 20 additions & 13 deletions .specify/memory/constitution.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,24 @@
<!--
SYNC IMPACT REPORT
==================
Version change: 1.2.0 → 1.3.0
Version change: 1.3.0 → 1.4.0

Principles modified: none

Sections added:
- Development Workflow: one new mandatory constraint added:
The quality-gate constraint from v1.2.0 is now enforced by a mandatory
after_implement hook (speckit.quality-gate in .specify/extensions.yml).
A failing gate blocks the git-commit hook and the Completion Report.
/speckit-implement MUST fix all failures and re-run before completing.
- AI Integration Requirements: new top-level section (parallel to CI/CD Integration
Requirements) that requires any AI tooling integration to explicitly verify
compatibility with Claude Code and GitHub Copilot (VS Code Agent mode).
Single-host implementations do not satisfy this requirement.

Sections removed: none

Templates reviewed:
- .specify/templates/plan-template.md ✅ — No update needed; plan's Quality Gate
Requirement section already documents the gate commands.
- .specify/templates/plan-template.md ✅ — No update needed; Constitution Check
table in plans will capture the new AI Integration gate when relevant.
- .specify/templates/spec-template.md ✅ — No alignment changes required.
- .specify/templates/tasks-template.md ✅ — Task categories unaffected.
- .specify/templates/commands/ ⚠ — Directory not present; skipped.
- .specify/extensions.yml ✅ — Hook registered as mandatory after_implement entry.
- .claude/skills/speckit-quality-gate/SKILL.md ✅ — Skill created; defines gate
stages and blocking behaviour on failure.
- .specify/extensions.yml ✅ — No hook changes required.

Deferred TODOs: none
-->
Expand Down Expand Up @@ -117,6 +113,17 @@ Diagnostic output MUST explain *why* a finding matters, not just *what* was foun
**Rationale**: A tool that only scores an API without explaining the reasoning does not
help developers improve their practice.

## AI Integration Requirements

The following constraints apply to any feature that delivers AI tooling integration:

- Any AI integration capability MUST be explicitly verified to function with **Claude Code** and **GitHub Copilot (VS Code Agent mode)** as the two primary supported targets.
- An implementation that functions in only one of these environments does not satisfy this requirement.
- MCP tool definitions MUST be self-describing: any capable MCP host MUST be able to discover and invoke all tools using only the tool definitions, with no additional documentation or configuration required beyond registering the server.
- All prerequisites for AI integration MUST have zero monetary cost, consistent with Principle V.

**Rationale**: AI tooling is rapidly diversifying across agentic IDEs, coding assistants, and enterprise agent platforms. Anchoring verification to a concrete set of three tools (covering CLI/agentic, IDE, and enterprise Copilot scenarios) prevents narrow implementations that work only in the tool the author happened to test against.

## CI/CD Integration Requirements

The CLI MUST be usable in CI/CD pipelines. The following constraints apply to any
Expand Down Expand Up @@ -168,4 +175,4 @@ All pull requests and code reviews MUST verify compliance with the principles ab
Complexity violations MUST be recorded in the plan's Complexity Tracking table with
explicit justification.

**Version**: 1.3.0 | **Ratified**: 2026-06-12 | **Last Amended**: 2026-06-18
**Version**: 1.4.0 | **Ratified**: 2026-06-12 | **Last Amended**: 2026-06-18
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!-- SPECKIT START -->
For additional context about technologies to be used, project structure,
shell commands, and other important information, read the current plan
at specs/006-publish-npmjs/plan.md
at specs/007-ai-support/plan.md
<!-- SPECKIT END -->
19 changes: 18 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,12 @@ src/
rulesets/
loader.ts # Loads the default ruleset or a custom one

packages/
api-grade-core/ # @dawmatt/api-grade-core — standalone grading library
api-grade-mcp/ # @dawmatt/api-grade-mcp — MCP server exposing six AI tools
backstage-plugin-api-grade/ # Backstage frontend card plugin
backstage-plugin-api-grade-backend/ # Backstage backend grading plugin

tests/
unit/ # Unit tests for individual modules
integration/ # End-to-end grading tests against fixture specs
Expand All @@ -56,14 +62,25 @@ tests/
rulesets/ # Test rulesets: custom-ruleset.yaml, missingfunction.yaml, unreachable.yaml
```

## Monorepo packages

| Package | Path | Description |
|---------|------|-------------|
| `@dawmatt/api-grade` | `/` (root) | CLI tool (`api-grade` binary) |
| `@dawmatt/api-grade-core` | `packages/api-grade-core/` | Standalone grading library used by all other packages |
| `@dawmatt/api-grade-mcp` | `packages/api-grade-mcp/` | MCP server exposing six AI tools (`grade-api`, `grade-api-detailed`, `assert-api-grade`, `grade-api-quick-fixes-only`, `set-ruleset-config`, `get-ruleset-config`) |
| `@dawmatt/backstage-plugin-api-grade` | `packages/backstage-plugin-api-grade/` | Backstage frontend card plugin |
| `@dawmatt/backstage-plugin-api-grade-backend` | `packages/backstage-plugin-api-grade-backend/` | Backstage backend grading plugin |

## Scripts

| Command | Description |
|---------|-------------|
| `npm run build` | Compile TypeScript to `dist/` |
| `npm run build` | Compile TypeScript to `dist/` (root CLI and all packages) |
| `npm test` | Run all tests once |
| `npm run test:watch` | Run tests in watch mode |
| `npm run test:coverage` | Run tests and generate a coverage report |
| `npm run -w packages/api-grade-mcp test` | Run MCP server tests only |

## Development process — Specification-Driven Development (SDD)

Expand Down
23 changes: 21 additions & 2 deletions GOAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,31 @@ Feature 6 - npmjs
- Make sure the GitHub release description reflects the changes in the release. Gather this information from commit descriptions included in the release, ignoring any commit that only includes a release tag.
- Update documentation to cover npmjs. Includes both the user oriented documentation and the contribution guide

Feature 7 - AI support
Feature 7 - Local AI support

- Allow API grading to be performed directly from LLMs and agentic AI tooling
- Allow API grading to be performed directly (locally) from LLMs and agentic AI tooling
- Support all of the standard functions of the api-grade CLI, including: grading an API (both overall and detailed levels), and asserting whether an API is at or above a particular grade (e.g. >= C)
- Use the api-grade's JSON format output so the AI is able to process and reformat the information in a way that suits its requirements
- Leverage the AI support to not just grade the API, but also resolve the "non-breaking change" issues highlighted by the grading that are bringing down the result
- Any local AI tooling support must explicitly include Claude Code and GitHub Copilot
- Add support for rulesets hosted on GitHub private repos (via PAT) and Entra ID protected environments (e.g. SharePoint, OneDrive)
- Support both direct installed and containerised execution of the functionality.

Feature 8 - CLI GitHub PAT

- Add CLI support for rulesets hosted on GitHub private repos (via PAT)

Feature 9 - Remote AI support

- Allow API grading to be performed directly (remotely) from LLMs and agentic AI tooling
- Update AI support to include remote access via streamable/HTTP transport
- Any remote AI tooling support must explicitly include Claude Code, GitHub Copilot and Copilot Studio
- Support both direct installed and containerised execution of the functionality.

Feature 10 - Entra and CLI

- Confirm Entra ID ruleset functionality in MCP is operational, for Entra ID protected environments (e.g. SharePoint, OneDrive)
- Add Entra ID functionality to CLI as well

## Constitution

Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@ npm install @dawmatt/api-grade-core

**[Backstage Plugins](docs/backstage-plugins/README.md)** — Display API grades on Backstage API entity pages. A frontend card shows the grade summary; a backend plugin computes grades server-side using `@dawmatt/api-grade-core`.

**[MCP Server](docs/mcp/quick-start.md)** — Expose api-grade capabilities to AI tools via the Model Context Protocol. Register in Claude Code, GitHub Copilot (VS Code Agent mode), or any MCP-compatible AI host and let the AI grade specs directly.

```bash
claude mcp add api-grade -- npx -y @dawmatt/api-grade-mcp
```

Can also be run via the published Docker image instead of `npx`/`node` — see [Docker invocation](docs/mcp/quick-start.md#run-via-docker).

---

## Documentation
Expand All @@ -56,6 +64,7 @@ Full documentation is available in the **[Documentation Index](docs/index.md)**:
- [CLI Reference](docs/cli/README.md) — installation, commands, and CI/CD setup
- [Package Reference](docs/package/README.md) — integrate `api-grade-core` into your tooling
- [Backstage Plugins](docs/backstage-plugins/README.md) — display grades in your developer portal
- [MCP Server](docs/mcp/README.md) — grade specs from AI tools via MCP

---

Expand Down
38 changes: 30 additions & 8 deletions docs/contributing/release-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ When in doubt, use `minor` — it signals new functionality without breaking exi

### When to assign the version

Version assignment happens **in the feature branch**, before the PR is opened. This makes the version bump part of the reviewed change, keeps the main branch clean of direct commits, and ensures the release pipeline only triggers after the change has been reviewed and approved.
Version assignment happens **in the feature branch**, before the PR is opened. This makes the version bump part of the reviewed change and keeps it visible in the PR diff.

> **Important — squash merges**: GitHub squash-merges a PR into a single new commit on `main`. The version bump commit from the feature branch is **not** the commit that lands on `main`; the squash commit is. The release tag must point to the squash commit (a `main` commit), not the original feature branch commit. Step 5 below handles this.

### Prerequisites

Expand All @@ -43,7 +45,7 @@ Version assignment happens **in the feature branch**, before the PR is opened. T
node scripts/version.mjs patch # or minor, or major
```

This updates `version` in all four `package.json` files, creates a git commit (`chore: release v<N>`), and creates a local `v<N>` tag. The tag stays local — do not push it yet.
This updates `version` in all four `package.json` files and creates a git commit (`chore: release v<N>`). A local `v<N>` tag is also created — you will re-create it after the squash merge in step 5.

2. **Push only the branch** (not the tag):

Expand All @@ -55,29 +57,36 @@ Version assignment happens **in the feature branch**, before the PR is opened. T

3. **Open a pull request** targeting `main`. The PR must pass the quality gate CI and receive at least one maintainer approval before it can be merged (enforced by branch protection).

4. **After the PR is merged**, fetch and switch to main:
4. **After the PR is squash-merged**, fetch and switch to main:

```bash
git fetch origin main
git checkout main
git pull origin main
```

5. **Push the tag** to trigger the release pipeline:
5. **Re-create the tag** pointing to the squash merge commit (the current `HEAD` of `main`):

```bash
git tag -d v<N> # delete the local tag that points to the feature branch commit
git tag v<N> # create a fresh tag at the squash merge commit (current HEAD)
```

6. **Push the tag** to trigger the release pipeline:

```bash
git push origin v<N>
```

Pushing the tag triggers `.github/workflows/release.yml`. The pipeline first verifies that the tagged commit is on `main`; it will fail immediately if it is not.
Pushing the tag triggers `.github/workflows/release.yml`. The pipeline first verifies that the tagged commit is reachable from `main`; it will fail immediately if it is not.

6. **Approve the deployment** in GitHub Actions:
7. **Approve the deployment** in GitHub Actions:
- Go to **Actions → Release** → the running workflow.
- Under **Environments**, click **Review deployments** → **Approve**.

7. **Monitor the pipeline** — the workflow runs six quality gate stages (audit, lint, typecheck, coverage × 4, build), then publishes all four packages in dependency order, then creates a GitHub Release with notes generated from commit messages.
8. **Monitor the pipeline** — the workflow runs six quality gate stages (audit, lint, typecheck, coverage × 4, build), then publishes all four packages in dependency order, then creates a GitHub Release with notes generated from commit messages.

8. **Verify** the packages appear on npmjs.com under the `@dawmatt` scope and the GitHub Release description lists the changes.
9. **Verify** the packages appear on npmjs.com under the `@dawmatt` scope and the GitHub Release description lists the changes.

---

Expand All @@ -92,6 +101,18 @@ Version assignment happens **in the feature branch**, before the PR is opened. T

## Recovery from a Failed Release

### Tag not on main (failed main-branch verification)

The pipeline failed at "Verify tag is on main branch". This happens when the tag points to a feature branch commit that was not included in the squash merge. Delete the tag and re-create it at the squash merge commit:

```bash
git tag -d v<N> # delete local tag (points to wrong commit)
git push origin --delete v<N> # delete remote tag
git checkout main && git pull origin main
git tag v<N> # create fresh tag at squash merge commit
git push origin v<N> # push to trigger release
```

### Pipeline failed before any packages published

The quality gate caught a problem. No packages were published. Delete the tag, fix the issue in a new or updated branch, and re-release:
Expand All @@ -104,6 +125,7 @@ node scripts/version.mjs patch # re-bump on the branch
git push origin <branch-name> # push branch (not tag)
# after PR is reviewed, approved, and merged:
git checkout main && git pull origin main
git tag -d v<N> 2>/dev/null; git tag v<N> # re-create tag at squash merge commit
git push origin v<N> # push tag to trigger release
```

Expand Down
20 changes: 18 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ The grading algorithm is **error-first**: one error outweighs many warnings. It

---

## The Three Components
## The Four Components

api-grade is built from three components that share the same grading engine:
api-grade is built from four components that share the same grading engine:

### CLI Tool

Expand Down Expand Up @@ -53,13 +53,28 @@ Two Backstage plugin packages that display API grades directly on your Backstage

---

### MCP Server (`@dawmatt/api-grade-mcp`)

An MCP (Model Context Protocol) server that exposes api-grade as six AI tools: `grade-api`, `grade-api-detailed`, `assert-api-grade`, `grade-api-quick-fixes-only`, `set-ruleset-config`, and `get-ruleset-config`. Register it in Claude Code, GitHub Copilot (VS Code Agent mode), or any MCP-compatible AI host and let the AI grade specs directly.

```bash
claude mcp add api-grade -- npx -y @dawmatt/api-grade-mcp
```

Configure a default Spectral ruleset so grading requests automatically use your organisation's standards — at session, workspace, or global scope.

→ [MCP Server quick start](mcp/quick-start.md) | [Configuration reference](mcp/configuration.md) | [Troubleshooting](mcp/troubleshooting.md)

---

## Choose Your Path

| I want to… | Start here |
|------------|-----------|
| Grade a spec from the terminal | [CLI Tool](cli/README.md) |
| Set up a CI/CD grade gate | [CLI Commands → CI/CD example](cli/commands.md) |
| Integrate grading into my own code | [Core Package (`@dawmatt/api-grade-core`)](package/README.md) |
| Grade specs from an AI assistant | [MCP Server quick start](mcp/quick-start.md) |
| Show grades in Backstage | [Backstage Quick Start](backstage-plugins/quick-start.md) |
| Understand the full documentation | [Documentation Index](index.md) |

Expand All @@ -71,3 +86,4 @@ Two Backstage plugin packages that display API grades directly on your Backstage
- [CLI Tool](cli/README.md) — installation and quick-start
- [Core Package](package/README.md) — package overview and installation
- [Backstage Plugins](backstage-plugins/README.md) — plugin architecture and setup
- [MCP Server](mcp/quick-start.md) — grade specs from AI tools
8 changes: 8 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@
| [Core Package](package/README.md) | Embed `api-grade-core` in your own tools and scripts |
| [Package Usage Guide](package/usage-guide.md) | Common integration patterns and worked examples |
| [Package API Reference](package/api-reference.md) | All exported functions, classes, and types |
| [MCP Server](mcp/README.md) | Grade specs from AI tools via MCP |
| [MCP Server Overview](package/api-grade-mcp.md) | All six MCP tools and their inputs/outputs |
| [MCP Quick Start](mcp/quick-start.md) | Install and configure the MCP server in minutes |
| [MCP Configuration Reference](mcp/configuration.md) | Default rulesets, auth, and scope precedence |
| [MCP Entra ID Setup](mcp/entra-id-setup.md) | One-time Azure-side app registration for Entra ID auth |
| [MCP GitHub Token Setup](mcp/github-pat-setup.md) | One-time GitHub PAT creation for `github-pat` ruleset auth |
| [MCP Troubleshooting](mcp/troubleshooting.md) | Auth failures, missing tools, and common errors |
| [Backstage Plugins](backstage-plugins/README.md) | Display grades on Backstage API entity pages |
| [Backstage Quick Start](backstage-plugins/quick-start.md) | Get both plugins running in under 30 minutes |
| [Backstage Configuration](backstage-plugins/configuration.md) | All plugin configuration options |
Expand All @@ -38,6 +45,7 @@ The grading algorithm is error-first: a single error outweighs many warnings. It

- **I want to grade a spec from the command line** → [CLI Tool](cli/README.md)
- **I want to use grading in my own code or tooling** → [Core Package](package/README.md)
- **I want to grade specs from an AI assistant** → [MCP Server](mcp/README.md)
- **I want grades to appear in my Backstage developer portal** → [Backstage Plugins](backstage-plugins/README.md)
- **I'm not sure where to start** → [Getting Started](getting-started.md)

Expand Down
Loading
Loading