Skip to content

feat: derive MCP server evidence from declarations and package registries - #5069

Open
daviddanialy wants to merge 11 commits into
daviddanialy/ais-467-feat-mcp-server-identity-resolutionfrom
daviddanialy/ais-468-feat-assemble-mcp-server-evidence
Open

feat: derive MCP server evidence from declarations and package registries#5069
daviddanialy wants to merge 11 commits into
daviddanialy/ais-467-feat-mcp-server-identity-resolutionfrom
daviddanialy/ais-468-feat-assemble-mcp-server-evidence

Conversation

@daviddanialy

@daviddanialy daviddanialy commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Stacked on #5036, which is stacked on #5035. Review those first; the diff
here is only the final commit.

Note that stacked PRs get no CI in this repo — hygiene.yaml and pr.yaml
are both gated on branches: [main] — so this needs a manual
workflow_dispatch until its base merges.

First slice of AIS-468. Answers the approver's most concrete question — "what
permissions is it requesting, just read-only or taking action on behalf?"

from the two sources available without contacting anything: the MCP annotations
a server publishes, and the shape of each tool's input schema.

Everything here is a declaration, never an observation

The MCP specification is explicit about this:

For trust & safety and security, clients MUST consider tool annotations to
be untrusted unless they come from trusted servers.

A server is free to declare readOnlyHint: true and write anyway. So an
assessment is not evidence of what a server does — its value is that it shows
what authority a server is asking for, which is what lets an admin grant
narrowly. That distinction is load-bearing enough that it is stated in the
package doc, not just here.

Undeclared is its own state

Hint is tri-state: declared true, declared false, or undeclared.

The specification assigns defaults for omitted annotations, and I deliberately
did not encode them. Reading an absent annotation as a positive claim would
manufacture reassurance the server never offered, and this surface has to show
undeclared as undeclared — the same rule the approval page applies to an
unidentified server. Unannotated is surfaced explicitly so a tool that
declared nothing cannot be mistaken for one that declared itself harmless.

For the same reason destructiveHint: false does not add
CapabilityDestructive: it is a claim of non-destructiveness, not a declaration
of destructive capability.

Schema-implied capabilities are heuristics, and labelled as such

Declared and SchemaImplied are separate fields because the two carry
different confidence. Parameter-name and format matching flags a tool
accepting a command, a path, a URL, or a credential — but a server can accept
the same input under any name it likes, so a miss is expected and a hit is a
prompt to look closer rather than a finding. The signal list is deliberately
short and legible instead of exhaustive.

The walk descends through nested objects, array items, and
allOf/anyOf/oneOf so a dangerous parameter cannot hide one level down. It
is depth-bounded: schemas come from the server under review, so a
self-referential one must terminate rather than exhaust the stack.

An unparseable or absent schema yields nothing, which is explicitly not a
statement that the tool takes no dangerous input.

Scope

This is the capability half of AIS-468. Registry provenance and maturity
parsing, the OAuth-scope surfacing, and the ClickHouse "are we already exposed?"
signals land separately. Nothing calls this yet — its consumer is the
mcpapproval API in AIS-470.

Not yet done, and tracked on the ticket: factoring the shared annotation reading
out of scanners/destructivetool, which reads the same destructiveHint but is
call-driven rather than definition-driven.


Summary by cubic

Builds evidence for AIS-468 by deriving MCP server capability, requested authority, registry provenance/popularity, and npm/PyPI package metadata for uncatalogued servers. Unifies destructiveHint handling and fixes correctness and lookup edge cases; no user-facing changes.

  • New Features

    • Evidence packages: Capability (tri-state hints, ActsOnBehalf, safe-depth schema signals for arbitrary_command|filesystem_path|arbitrary_url|credential_input), Authority (mode undeclared|none|api_key|oauth, scope normalization, DCR from registration_endpoint, demanded/optional secrets, unauthenticated tools), Provenance (official/status/isLatest, publish/update times, visitor estimates), Package metadata (npm/PyPI license, publish window, version/maintainer counts, deprecated/yanked; base URLs overridable; responses size-bounded; nil on unknown).
    • Refactor: share destructiveHint interpretation via capability.DeclaresDestructive(...) with scanners/destructivetool.
  • Bug Fixes

    • Deterministic schema capability detection across depths; all same-named parameters examined.
    • OAuth/DCR: a registration_endpoint implies OAuth; DCR derived from the endpoint; fix undeclared vs none.
    • Package lookups: npm recency uses newest per-version release; tolerate time.unpublished; strip only well-formed PyPI extras (treat empty [] as malformed too; malformed specs surface as unknown); reject 200 responses that carry no package name as unrecognized; oversized responses fail with a clear size error.
    • Provenance: empty/unrecognized _meta reads as “not catalogued,” not empty facts.
    • Tests: mutex-guarded registry test helper to remove a race.

Written for commit f3249fe. Summary will update on new commits.

Review in cubic

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daviddanialy
daviddanialy requested a review from a team as a code owner August 7, 2026 17:25
@daviddanialy daviddanialy added enhancement New feature or request go Pull requests that update go code labels Aug 7, 2026
@linear-code

linear-code Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

AIS-468

@changeset-bot

changeset-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f3249fe

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
server Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/internal/mcpapproval/capability/capability.go">

<violation number="1" location="server/internal/mcpapproval/capability/capability.go:202">
P2: `folderId` inputs are now reported as `filesystem_path` even when they are remote resource IDs; constrain the new marker to path-like fields or require an explicit path format before exposing it as a capability. This creates a false approval signal for common cloud-storage schemas.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

},
{
capability: CapabilityFilesystemPath,
substrings: []string{"path", "filename", "filepath", "directory", "dirname", "folder"},

@cubic-dev-ai cubic-dev-ai Bot Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: folderId inputs are now reported as filesystem_path even when they are remote resource IDs; constrain the new marker to path-like fields or require an explicit path format before exposing it as a capability. This creates a false approval signal for common cloud-storage schemas.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/internal/mcpapproval/capability/capability.go, line 202:

<comment>`folderId` inputs are now reported as `filesystem_path` even when they are remote resource IDs; constrain the new marker to path-like fields or require an explicit path format before exposing it as a capability. This creates a false approval signal for common cloud-storage schemas.</comment>

<file context>
@@ -183,7 +199,7 @@ var parameterSignals = []struct {
 	{
 		capability: CapabilityFilesystemPath,
-		substrings: []string{"path", "filename", "filepath", "directory", "dirname"},
+		substrings: []string{"path", "filename", "filepath", "directory", "dirname", "folder"},
 		formats:    []string{"path"},
 	},
</file context>
Fix with cubic

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping this one as-is. The signal table's own contract (comment at the top of parameterSignals) is deliberately loose: a hit is "a prompt to look closer", never an adjudication — and a folder id on a cloud-storage tool is a resource-location parameter an approver should glance at. Constraining to an explicit path format would drop most honest filesystem servers, which declare no format at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/mcpapproval/authority/authority.go
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daviddanialy daviddanialy changed the title feat: summarise the capability an MCP tool declares feat: derive MCP server evidence from declarations and package registries Aug 7, 2026
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daviddanialy
daviddanialy force-pushed the daviddanialy/ais-468-feat-assemble-mcp-server-evidence branch from 80035a2 to 233e6e4 Compare August 7, 2026 17:48
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/mcpapproval/packagemeta/packagemeta_test.go Outdated
Comment thread server/internal/mcpapproval/provenance/provenance.go
Comment thread server/internal/mcpapproval/packagemeta/packagemeta.go
Comment thread server/internal/mcpapproval/packagemeta/packagemeta.go Outdated
daviddanialy and others added 2 commits August 7, 2026 11:50
Four confirmed defects in the evidence packages, each producing actively
wrong evidence on a surface whose whole value is being trustworthy:

- Schema capability detection was nondeterministic: properties were
  collected into a name-keyed map, so two same-named parameters at
  different depths collided and which survived depended on map iteration
  order. Every occurrence is now examined, in a deterministic order.
- npm maintenance recency read the registry's `modified` timestamp,
  which moves on any metadata edit — deprecating an abandoned package
  would have made it look actively maintained. Recency now reads the
  newest per-version release time, matching the PyPI path.
- A declaration carrying only an OAuth registration endpoint summarised
  as "nothing published about authentication" while simultaneously
  advertising dynamic registration. Publishing that endpoint now counts
  as the OAuth declaration it is.
- A registry `_meta` blob with no recognized content — an empty map, or
  the zero struct the registry client hands over when a response carried
  no meta — read as catalogued with every fact zeroed: an empty panel
  presented as a clean bill of health. It now reads as not catalogued.

Also hardened from the same review: npm's `time.unpublished` object no
longer fails the whole lookup, PEP 508 extras are stripped before the
PyPI path lookup, oversized registry responses fail with a clear size
error instead of truncating into a decode error (and the cap now sits
above what long-lived packages actually produce), and doc comments that
overclaimed — "the single reading of destructiveHint", ModeNone as an
explicit declaration — now state what the code does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The handler runs on the server's goroutine and a completed HTTP
round-trip is not a synchronization edge the race detector recognises,
so the recorded path is now read through a mutex-guarded getter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/mcpapproval/provenance/provenance.go
Comment thread server/internal/mcpapproval/packagemeta/packagemeta.go Outdated
A malformed spec like `foo[bar` names nothing pip would install, and
quietly resolving it to `foo` would attribute another package's
evidence to it. Only a terminal, bracket-balanced extras expression is
stripped; anything else is left unchanged, finds no project, and
surfaces as unknown.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/mcpapproval/packagemeta/packagemeta.go Outdated
daviddanialy and others added 2 commits August 7, 2026 13:30
`foo[]` is not a valid spec — extras are a non-empty identifier list —
so it stays unstripped and surfaces as unknown like the other
malformed forms.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A 200 whose document names no package is a response this client does
not recognize, not metadata — returning it as found would put an empty
package in front of an approver as if it were a finding. It now errors,
which the evidence assembler records as a lookup gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant