Route tool defects opaquely; remove legacy envelope shim (#826)

* feat(tool-result): add ToolResult union and dual-shape invoker

Adds `ToolResult<T>` / `ToolError` to @executor-js/sdk and a
`ToolResult.ok` / `ToolResult.fail` constructor pair.

Updates the sandbox tool dispatcher to accept both the new typed union
(passes through unchanged) and the legacy `{ data, error }` envelope.
The legacy branch now walks known upstream shapes (Microsoft Graph,
DealCloud, JSON:API, plain message-bearing bodies) before falling back
to a clamped JSON.stringify of the body, so structured 4xx payloads no
longer collapse to 'Tool execution failed'. The cause/defect branch
keeps its strict .message-only discipline.

* feat(plugins): emit ToolResult from openapi, graphql, mcp, google-discovery

Migrates each dynamic plugin's invokeTool to wrap its result in the
typed ToolResult<T> union. OpenAPI and Google Discovery map non-2xx
responses to ToolResult.fail with an extracted upstream message and
the raw body in error.details; 2xx responses to ToolResult.ok with
{ status, headers, data }. GraphQL maps 200-with-errors bodies to
ToolResult.fail({ code: 'graphql_errors', ... }) and bare data to
ToolResult.ok. MCP maps { isError: true } to ToolResult.fail and
content arrays to ToolResult.ok.

Static plugin tools (executor.openapi.previewSpec / addSource,
executor.graphql.addSource) now also wrap their plain values in
ToolResult.ok for shape consistency.

Updates plugin tests that asserted on the old { status, headers, data,
error } envelope or the bare MCP { content } shape. Tests that consume
the OpenAPI plugin's invocation result use the new unwrapInvocation
helper from @executor-js/plugin-openapi/testing.

* feat(tool-result): make ToolResult mandatory and route defects opaquely

Removes the legacy { data, error } envelope shim from the sandbox tool
dispatcher. The invoker now passes ToolResult<T> through unchanged and
wraps any other plain-value plugin return in ToolResult.ok so the
sandbox surface is uniform.

Plugin/infra defects no longer pass their message into the sandbox.
The dispatcher generates a short hex correlation id, logs the full
cause with that id under executor.correlation_id, and rejects with
`Internal tool error [<corrId>]`. The QuickJS bridge and the
dynamic-worker module template defensively re-stamp the same opaque
shape; the MCP host server's top-level execute failure path does the
same.

ExecutionToolError in-band messages from the execution package's
built-in validators (tools.search arg checks, etc.) are still passed
through at the QuickJS bridge so model-facing input errors keep their
useful diagnostic.

Tests:
- ToolResult.ok / fail / isToolResult constructor unit tests.
- repro tests assert structured upstream payloads now reach the
  sandbox through ToolResult.error.details (not through .message).
- leak tests pin the new invariant: plugin defects only escape as the
  opaque generic + correlation id; no token / connection string / file
  path leaks into Error.message.
- QuickJS end-to-end defect test confirms the same shape at the
  sandbox boundary.
- Cloud HTTP integration tests, MCP host tests, dynamic-worker
  invocation tests, and plugin tests updated for the new ToolResult
  wire shape and the opaque-generic defect contract.

* Fix tool result contracts

Author: RhysSullivan

apps/cloud/src/services/sources-api.node.test.ts

@@ -207,9 +207,15 @@ describe("sources api (HTTP)", () => {
       expect(execution.structured).toMatchObject({
         status: "completed",
         result: {
-          message: "hello",
-          suffix: "world",
-          path: "/echo/hello",
+          ok: true,
+          data: {
+            status: 200,
+            data: {
+              message: "hello",
+              suffix: "world",
+              path: "/echo/hello",
+            },
+          },
         },
         logs: [],
       });
@@ -315,7 +321,7 @@ describe("sources api (HTTP)", () => {
       expect(execution.isError).toBe(false);
       expect(execution.structured).toMatchObject({
         status: "completed",
-        result: { hello: "Hello Ada" },
+        result: { ok: true, data: { hello: "Hello Ada" } },
       });
       const requests = yield* server.requests;
       expect(requests.some((request) => request.payload.query?.includes("__schema"))).toBe(true);
@@ -391,7 +397,8 @@ describe("sources api (HTTP)", () => {
       expect(execution.structured).toMatchObject({
         status: "completed",
         result: {
-          content: [{ type: "text", text: "cloud-mcp-ok" }],
+          ok: true,
+          data: { content: [{ type: "text", text: "cloud-mcp-ok" }] },
         },
       });
       expect((yield* server.requests).length).toBeGreaterThanOrEqual(2);

apps/local/src/server/migrate-oauth-connections.test.ts

@@ -159,7 +159,7 @@ describe("0009_repair_openapi_oauth_cutover_residue", () => {
     `);
 
     db.prepare("INSERT INTO `openapi_source` (id, scope_id, oauth2) VALUES (?, ?, ?)").run(
-      "dealcloud_api",
+      "example_api",
       "org-1",
       JSON.stringify({
         kind: "oauth2",
@@ -174,27 +174,27 @@ describe("0009_repair_openapi_oauth_cutover_residue", () => {
       "INSERT INTO `connection` (id, scope_id, provider, provider_state, scope, updated_at) VALUES (?, ?, ?, ?, ?, ?)",
     );
     insertConnection.run(
-      "openapi-oauth2-app-dealcloud_api",
+      "openapi-oauth2-app-example_api",
       "org-1",
       "oauth2",
       JSON.stringify({
         kind: "client-credentials",
-        tokenEndpoint: "https://resolve.dealcloud.com/oauth/token",
-        clientIdSecretId: "dealcloud-client-id",
-        clientSecretSecretId: "dealcloud-client-secret",
+        tokenEndpoint: "https://auth.example.test/oauth/token",
+        clientIdSecretId: "example-client-id",
+        clientSecretSecretId: "example-client-secret",
       }),
       null,
       now,
     );
     insertConnection.run(
-      "openapi-oauth2-app-dealcloud_api",
+      "openapi-oauth2-app-example_api",
       "user-org:user-jd:org-1",
       "openapi:oauth2",
       JSON.stringify({
         kind: "client-credentials",
-        tokenEndpoint: "https://resolve.dealcloud.com/oauth/token",
-        clientIdSecretId: "dealcloud-client-id-jd",
-        clientSecretSecretId: "dealcloud-client-secret-jd",
+        tokenEndpoint: "https://auth.example.test/oauth/token",
+        clientIdSecretId: "example-client-id-jd",
+        clientSecretSecretId: "example-client-secret-jd",
       }),
       null,
       now,
@@ -207,12 +207,12 @@ describe("0009_repair_openapi_oauth_cutover_residue", () => {
       "org-client-id",
       "org-1",
       "openapi",
-      "dealcloud_api",
+      "example_api",
       "org-1",
       "oauth2:oauth2:client-id",
       "secret",
       null,
-      "dealcloud-client-id-jd",
+      "example-client-id-jd",
       null,
       now,
       now,
@@ -221,12 +221,12 @@ describe("0009_repair_openapi_oauth_cutover_residue", () => {
       "org-client-secret",
       "org-1",
       "openapi",
-      "dealcloud_api",
+      "example_api",
       "org-1",
       "oauth2:oauth2:client-secret",
       "secret",
       null,
-      "dealcloud-client-secret-jd",
+      "example-client-secret-jd",
       null,
       now,
       now,
@@ -235,27 +235,27 @@ describe("0009_repair_openapi_oauth_cutover_residue", () => {
       "org-connection",
       "org-1",
       "openapi",
-      "dealcloud_api",
+      "example_api",
       "org-1",
       "oauth2:oauth2:connection",
       "connection",
       null,
       null,
-      "openapi-oauth2-app-dealcloud_api",
+      "openapi-oauth2-app-example_api",
       now,
       now,
     );
     insertBinding.run(
       "jd-connection",
       "user-org:user-jd:org-1",
       "openapi",
-      "dealcloud_api",
+      "example_api",
       "org-1",
       "oauth2:oauth2:connection",
       "connection",
       null,
       null,
-      "openapi-oauth2-app-dealcloud_api",
+      "openapi-oauth2-app-example_api",
       now,
       now,
     );
@@ -271,49 +271,49 @@ describe("0009_repair_openapi_oauth_cutover_residue", () => {
       .prepare(
         "SELECT scope_id, slot_key, kind, secret_id, connection_id FROM `credential_binding` WHERE source_id = ? ORDER BY scope_id, slot_key",
       )
-      .all("dealcloud_api");
+      .all("example_api");
     expect(bindings).toEqual([
       {
         scope_id: "org-1",
         slot_key: "oauth2:oauth2:client-id",
         kind: "secret",
-        secret_id: "dealcloud-client-id",
+        secret_id: "example-client-id",
         connection_id: null,
       },
       {
         scope_id: "org-1",
         slot_key: "oauth2:oauth2:client-secret",
         kind: "secret",
-        secret_id: "dealcloud-client-secret",
+        secret_id: "example-client-secret",
         connection_id: null,
       },
       {
         scope_id: "org-1",
         slot_key: "oauth2:oauth2:connection",
         kind: "connection",
         secret_id: null,
-        connection_id: "openapi-oauth2-app-dealcloud_api",
+        connection_id: "openapi-oauth2-app-example_api",
       },
       {
         scope_id: "user-org:user-jd:org-1",
         slot_key: "oauth2:oauth2:client-id",
         kind: "secret",
-        secret_id: "dealcloud-client-id-jd",
+        secret_id: "example-client-id-jd",
         connection_id: null,
       },
       {
         scope_id: "user-org:user-jd:org-1",
         slot_key: "oauth2:oauth2:client-secret",
         kind: "secret",
-        secret_id: "dealcloud-client-secret-jd",
+        secret_id: "example-client-secret-jd",
         connection_id: null,
       },
       {
         scope_id: "user-org:user-jd:org-1",
         slot_key: "oauth2:oauth2:connection",
         kind: "connection",
         secret_id: null,
-        connection_id: "openapi-oauth2-app-dealcloud_api",
+        connection_id: "openapi-oauth2-app-example_api",
       },
     ]);
   });

packages/core/execution/src/description.ts

@@ -48,6 +48,7 @@ const formatDescription = (sources: readonly Source[]): string => {
     "- `tools.search()` returns paginated, ranked matches: `{ items, total, hasMore, nextOffset }`. Best-first. Use short intent phrases like `github issues`, `repo details`, or `create calendar event`.",
     '- When you already know the namespace, narrow with `tools.search({ namespace: "github", query: "issues" })`.',
     "- `tools.executor.sources.list()` returns the same paged shape: `{ items: [{ id, toolCount, ... }], total, hasMore, nextOffset }`.",
+    "- Tool calls return a value union: `{ ok: true, data }` for success or `{ ok: false, error: { code, message, status?, details?, retryable? } }` for expected tool/domain failures. Branch on `result.ok`.",
     "- If `hasMore` is true and you didn't find what you need, fetch the next page: `tools.search({ query, offset: nextOffset, limit })`. Same `offset` parameter on `tools.executor.sources.list({ offset, limit })`.",
     "- Always use the namespace prefix when calling tools: `tools.<namespace>.<tool>(args)`. Example: `tools.home_assistant_rest_api.states.getState(...)` — not `tools.states.getState(...)`.",
     "- The `tools` object is a lazy proxy — `Object.keys(tools)` won't work. Use `tools.search()` or `tools.executor.sources.list()` instead.",

packages/core/execution/src/tool-invoker.leak.test.ts

@@ -0,0 +1,113 @@
+import { describe, expect, it } from "@effect/vitest";
+import { Data, Effect, Schema } from "effect";
+
+import { ElicitationResponse, createExecutor, definePlugin } from "@executor-js/sdk";
+import { makeTestConfig } from "@executor-js/sdk/testing";
+import { ExecutionToolError } from "./errors";
+import { makeExecutorToolInvoker } from "./tool-invoker";
+
+const EmptyInputSchema = Schema.toStandardSchemaV1(
+  Schema.toStandardJSONSchemaV1(Schema.Struct({})),
+);
+
+const acceptAll = () => Effect.succeed(ElicitationResponse.make({ action: "accept" }));
+
+// Plugin-internal tagged error whose `cause` carries internal diagnostics.
+// The dispatcher must route this through the opaque-generic path so none of
+// that context reaches the sandbox via Error.message.
+class FakePluginInvocationError extends Data.TaggedError("PluginInvocationError")<{
+  readonly message: string;
+  readonly cause: unknown;
+}> {}
+
+const leakyPlugin = definePlugin(() => ({
+  id: "leaky-test" as const,
+  storage: () => ({}),
+  staticSources: () => [
+    {
+      id: "leaky",
+      kind: "in-memory",
+      name: "Leaky",
+      tools: [
+        {
+          name: "failsWithCause",
+          description: "",
+          inputSchema: EmptyInputSchema,
+          handler: () =>
+            Effect.fail(
+              new FakePluginInvocationError({
+                message: "Upstream request failed",
+                cause: {
+                  _tag: "InternalTransportError",
+                  request: {
+                    method: "GET",
+                    url: "https://internal.service.local/v1/resources?trace=trace-123",
+                    headers: { "x-internal-routing": "private-cluster" },
+                  },
+                  stack: "Error: connect failed\n    at plugin-transport.ts:42:11",
+                  note: "internal diagnostic detail",
+                },
+              }),
+            ),
+        },
+        {
+          name: "throwsRawError",
+          description: "",
+          inputSchema: EmptyInputSchema,
+          handler: () =>
+            Effect.fail(
+              Object.assign(
+                // oxlint-disable-next-line executor/no-error-constructor -- boundary: leak test deliberately fails with a raw Error + crafted stack to assert the dispatcher's opaque-generic redaction
+                new Error("Internal: secret 'sk_live_abcd' rotation failed"),
+                {
+                  stack:
+                    "Error: Internal: secret 'sk_live_abcd' rotation failed\n    at /home/svc/.../secret-store.ts:88",
+                },
+              ),
+            ),
+        },
+      ],
+    },
+  ],
+}));
+
+describe("internal-error leak audit (opaque defects)", () => {
+  it.effect("plugin tagged error: defect surfaces only as opaque generic + correlation id", () =>
+    Effect.gen(function* () {
+      const executor = yield* createExecutor(makeTestConfig({ plugins: [leakyPlugin()] as const }));
+      const invoker = makeExecutorToolInvoker(executor, {
+        invokeOptions: { onElicitation: acceptAll },
+      });
+
+      const err = yield* Effect.flip(invoker.invoke({ path: "leaky.failsWithCause", args: {} }));
+      expect(err).toBeInstanceOf(ExecutionToolError);
+      // oxlint-disable-next-line executor/no-unknown-error-message -- boundary: leak test inspects the rendered message to assert it is the opaque generic
+      const msg = (err as { message: string }).message;
+      // Must be the canonical opaque shape: "Internal tool error [<hex>]"
+      expect(msg).toMatch(/^Internal tool error \[[0-9a-f]{8}\]$/);
+      // Crucially, no internal context leaks
+      expect(msg).not.toContain("trace-123");
+      expect(msg).not.toContain("private-cluster");
+      expect(msg).not.toContain("internal.service.local");
+      expect(msg).not.toContain("InternalTransportError");
+      expect(msg).not.toContain("Upstream request failed");
+    }),
+  );
+
+  it.effect("plain Error with stack: stack and message do NOT escape", () =>
+    Effect.gen(function* () {
+      const executor = yield* createExecutor(makeTestConfig({ plugins: [leakyPlugin()] as const }));
+      const invoker = makeExecutorToolInvoker(executor, {
+        invokeOptions: { onElicitation: acceptAll },
+      });
+
+      const err = yield* Effect.flip(invoker.invoke({ path: "leaky.throwsRawError", args: {} }));
+      // oxlint-disable-next-line executor/no-unknown-error-message -- boundary: leak test inspects the rendered message to assert it is the opaque generic
+      const msg = (err as { message: string }).message;
+      expect(msg).toMatch(/^Internal tool error \[[0-9a-f]{8}\]$/);
+      expect(msg).not.toContain("secret-store.ts");
+      expect(msg).not.toContain("at /home/");
+      expect(msg).not.toContain("sk_live_abcd");
+    }),
+  );
+});