feat: derive MCP server evidence from declarations and package registries - #5069
Conversation
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: f3249fe The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="server/internal/mcpapproval/capability/capability.go">
<violation number="1" location="server/internal/mcpapproval/capability/capability.go:202">
P2: `folderId` inputs are now reported as `filesystem_path` even when they are remote resource IDs; constrain the new marker to path-like fields or require an explicit path format before exposing it as a capability. This creates a false approval signal for common cloud-storage schemas.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
| }, | ||
| { | ||
| capability: CapabilityFilesystemPath, | ||
| substrings: []string{"path", "filename", "filepath", "directory", "dirname", "folder"}, |
There was a problem hiding this comment.
P2: folderId inputs are now reported as filesystem_path even when they are remote resource IDs; constrain the new marker to path-like fields or require an explicit path format before exposing it as a capability. This creates a false approval signal for common cloud-storage schemas.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/internal/mcpapproval/capability/capability.go, line 202:
<comment>`folderId` inputs are now reported as `filesystem_path` even when they are remote resource IDs; constrain the new marker to path-like fields or require an explicit path format before exposing it as a capability. This creates a false approval signal for common cloud-storage schemas.</comment>
<file context>
@@ -183,7 +199,7 @@ var parameterSignals = []struct {
{
capability: CapabilityFilesystemPath,
- substrings: []string{"path", "filename", "filepath", "directory", "dirname"},
+ substrings: []string{"path", "filename", "filepath", "directory", "dirname", "folder"},
formats: []string{"path"},
},
</file context>
There was a problem hiding this comment.
Keeping this one as-is. The signal table's own contract (comment at the top of parameterSignals) is deliberately loose: a hit is "a prompt to look closer", never an adjudication — and a folder id on a cloud-storage tool is a resource-location parameter an approver should glance at. Constraining to an explicit path format would drop most honest filesystem servers, which declare no format at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
80035a2 to
233e6e4
Compare
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
All reported issues were addressed across 6 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
Four confirmed defects in the evidence packages, each producing actively wrong evidence on a surface whose whole value is being trustworthy: - Schema capability detection was nondeterministic: properties were collected into a name-keyed map, so two same-named parameters at different depths collided and which survived depended on map iteration order. Every occurrence is now examined, in a deterministic order. - npm maintenance recency read the registry's `modified` timestamp, which moves on any metadata edit — deprecating an abandoned package would have made it look actively maintained. Recency now reads the newest per-version release time, matching the PyPI path. - A declaration carrying only an OAuth registration endpoint summarised as "nothing published about authentication" while simultaneously advertising dynamic registration. Publishing that endpoint now counts as the OAuth declaration it is. - A registry `_meta` blob with no recognized content — an empty map, or the zero struct the registry client hands over when a response carried no meta — read as catalogued with every fact zeroed: an empty panel presented as a clean bill of health. It now reads as not catalogued. Also hardened from the same review: npm's `time.unpublished` object no longer fails the whole lookup, PEP 508 extras are stripped before the PyPI path lookup, oversized registry responses fail with a clear size error instead of truncating into a decode error (and the cap now sits above what long-lived packages actually produce), and doc comments that overclaimed — "the single reading of destructiveHint", ModeNone as an explicit declaration — now state what the code does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The handler runs on the server's goroutine and a completed HTTP round-trip is not a synchronization edge the race detector recognises, so the recorded path is now read through a mutex-guarded getter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
All reported issues were addressed across 8 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
A malformed spec like `foo[bar` names nothing pip would install, and quietly resolving it to `foo` would attribute another package's evidence to it. Only a terminal, bracket-balanced extras expression is stripped; anything else is left unchanged, finds no project, and surfaces as unknown. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
`foo[]` is not a valid spec — extras are a non-empty identifier list — so it stays unstripped and surfaces as unknown like the other malformed forms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A 200 whose document names no package is a response this client does not recognize, not metadata — returning it as found would put an empty package in front of an approver as if it were a finding. It now errors, which the evidence assembler records as a lookup gap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
First slice of AIS-468. Answers the approver's most concrete question — "what
permissions is it requesting, just read-only or taking action on behalf?" —
from the two sources available without contacting anything: the MCP annotations
a server publishes, and the shape of each tool's input schema.
Everything here is a declaration, never an observation
The MCP specification is explicit about this:
A server is free to declare
readOnlyHint: trueand write anyway. So anassessment is not evidence of what a server does — its value is that it shows
what authority a server is asking for, which is what lets an admin grant
narrowly. That distinction is load-bearing enough that it is stated in the
package doc, not just here.
Undeclared is its own state
Hintis tri-state: declared true, declared false, or undeclared.The specification assigns defaults for omitted annotations, and I deliberately
did not encode them. Reading an absent annotation as a positive claim would
manufacture reassurance the server never offered, and this surface has to show
undeclared as undeclared — the same rule the approval page applies to an
unidentified server.
Unannotatedis surfaced explicitly so a tool thatdeclared nothing cannot be mistaken for one that declared itself harmless.
For the same reason
destructiveHint: falsedoes not addCapabilityDestructive: it is a claim of non-destructiveness, not a declarationof destructive capability.
Schema-implied capabilities are heuristics, and labelled as such
DeclaredandSchemaImpliedare separate fields because the two carrydifferent confidence. Parameter-name and
formatmatching flags a toolaccepting a command, a path, a URL, or a credential — but a server can accept
the same input under any name it likes, so a miss is expected and a hit is a
prompt to look closer rather than a finding. The signal list is deliberately
short and legible instead of exhaustive.
The walk descends through nested objects, array
items, andallOf/anyOf/oneOfso a dangerous parameter cannot hide one level down. Itis depth-bounded: schemas come from the server under review, so a
self-referential one must terminate rather than exhaust the stack.
An unparseable or absent schema yields nothing, which is explicitly not a
statement that the tool takes no dangerous input.
Scope
This is the capability half of AIS-468. Registry provenance and maturity
parsing, the OAuth-scope surfacing, and the ClickHouse "are we already exposed?"
signals land separately. Nothing calls this yet — its consumer is the
mcpapprovalAPI in AIS-470.Not yet done, and tracked on the ticket: factoring the shared annotation reading
out of
scanners/destructivetool, which reads the samedestructiveHintbut iscall-driven rather than definition-driven.
Summary by cubic
Builds evidence for AIS-468 by deriving MCP server capability, requested authority, registry provenance/popularity, and
npm/PyPIpackage metadata for uncatalogued servers. UnifiesdestructiveHinthandling and fixes correctness and lookup edge cases; no user-facing changes.New Features
arbitrary_command|filesystem_path|arbitrary_url|credential_input), Authority (modeundeclared|none|api_key|oauth, scope normalization, DCR fromregistration_endpoint, demanded/optional secrets, unauthenticated tools), Provenance (official/status/isLatest, publish/update times, visitor estimates), Package metadata (npm/PyPIlicense, publish window, version/maintainer counts, deprecated/yanked; base URLs overridable; responses size-bounded; nil on unknown).destructiveHintinterpretation viacapability.DeclaresDestructive(...)withscanners/destructivetool.Bug Fixes
registration_endpointimplies OAuth; DCR derived from the endpoint; fixundeclaredvsnone.time.unpublished; strip only well-formed PyPI extras (treat empty[]as malformed too; malformed specs surface as unknown); reject 200 responses that carry no package name as unrecognized; oversized responses fail with a clear size error._metareads as “not catalogued,” not empty facts.Written for commit f3249fe. Summary will update on new commits.