Skip to content

[codex] fix MCP output schema contracts - #851

Merged
RhysSullivan merged 8 commits into
mainfrom
codex/mcp-output-schema-regression-tests
May 20, 2026
Merged

[codex] fix MCP output schema contracts#851
RhysSullivan merged 8 commits into
mainfrom
codex/mcp-output-schema-regression-tests

Conversation

@RhysSullivan

@RhysSullivan RhysSullivan commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes the MCP output schema/display mismatch and keeps regression coverage around the full contract.

For MCP tools, upstream outputSchema describes the MCP result's structuredContent. Executor invocation data, however, is the full successful MCP CallToolResult object under ToolResult.ok.data: content, optional structuredContent, optional _meta, and optional isError. The MCP plugin now registers tool output schemas as that CallToolResult shape, nesting the upstream MCP schema under structuredContent.

The CallToolResult wrapper is derived from @modelcontextprotocol/sdk's CallToolResultSchema rather than hand-written. Executor only substitutes structuredContent with the per-tool upstream output schema and narrows successful results to isError: false, so content block details and future MCP schema changes track the SDK. Top-level extra properties remain permitted because MCP's Result type is intentionally extensible.

The PR also adds reusable output contract coverage through typeCheckOutputTypeScript, which compiles a described tool output TypeScript type against an actual runtime output value. MCP tests use it against real invocation data, and the existing core tool-invoker tests now share the helper too.

Root Cause

MCP discovery copied the upstream MCP outputSchema directly into Executor's registered tool schema. That made the schema panel and generated output TypeScript describe data as the upstream structured payload, while the actual invocation returned data as the full MCP call result envelope.

Validation

  • bun run --cwd packages/core/sdk typecheck passes with existing Effect language-service warnings
  • bun run --cwd packages/core/execution typecheck passes with existing src/promise.ts warning
  • bun run --cwd packages/plugins/mcp typecheck passes with existing Effect language-service warnings
  • ./node_modules/.bin/oxlint -c .oxlintrc.jsonc packages/plugins/mcp/src/sdk/plugin.ts packages/plugins/mcp/src/sdk/elicitation.test.ts packages/plugins/mcp/package.json bun.lock --deny-warnings
  • bun run --cwd packages/core/sdk test -- src/testing/tool-output-contract.test.ts
  • bun run --cwd packages/plugins/mcp test -- src/sdk/elicitation.test.ts -t "outputSchema|CallToolResult|refreshSource|structured MCP result"
  • bun run --cwd packages/plugins/mcp test -- src/sdk/elicitation.test.ts

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 20, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 5e7f807 Commit Preview URL

Branch Preview URL
May 20 2026, 07:05 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 20, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 5e7f807 May 20 2026, 07:05 PM

@RhysSullivan RhysSullivan changed the title [codex] add MCP output schema regression tests [codex] add MCP output contract regression tests May 20, 2026
@pkg-pr-new

pkg-pr-new Bot commented May 20, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@851

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@851

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@851

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@851

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@851

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@851

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@851

@executor-js/plugin-google-discovery

npm i https://pkg.pr.new/@executor-js/plugin-google-discovery@851

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@851

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@851

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@851

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@851

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@851

executor

npm i https://pkg.pr.new/executor@851

commit: 5e7f807

@RhysSullivan RhysSullivan changed the title [codex] add MCP output contract regression tests [codex] fix MCP output schema contracts May 20, 2026
@RhysSullivan
RhysSullivan marked this pull request as ready for review May 20, 2026 19:55
@RhysSullivan
RhysSullivan merged commit 44a02ad into main May 20, 2026
11 checks passed
RhysSullivan added a commit that referenced this pull request May 31, 2026
* add MCP output schema regression tests

* extract tool output type contract helper

* fix MCP tool output schema wrapper

* derive MCP output schema from SDK

* format MCP output schema changes

* Project typed MCP results for sandbox agents

* Align MCP invoke results with tool schema

* Keep MCP code mode result envelope consistent
@RhysSullivan
RhysSullivan deleted the codex/mcp-output-schema-regression-tests branch June 11, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant