Skip to content

Fix cua-ai ESM packaging, error-handling docs, API consistency - #18

Merged
rgarcia merged 9 commits into
mainfrom
hypeship/cua-ai-dx-fixes
Jun 10, 2026
Merged

Fix cua-ai ESM packaging, error-handling docs, API consistency#18
rgarcia merged 9 commits into
mainfrom
hypeship/cua-ai-dx-fixes

Conversation

@rgarcia

@rgarcia rgarcia commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • ESM packaging fix + release smoke: packages/ai dist/ is now bundled with tsdown (single ESM file, deps external, source keeps extensionless imports; tsc -b stays for typechecking via a gitignored dist-tsc/), so the published tarball imports under plain Node ESM (0.1.0 fails with ERR_UNSUPPORTED_DIR_IMPORT). CI and release-cua-ai.yml gain a pack + install-into-temp-dir + ESM import smoke step that would have caught the 0.1.0 regression.
  • Silent-error docs fix: complete()/stream() resolve with stopReason: "error" instead of throwing; the README Quick Start, a new Error Handling section, and examples/quickstart.ts now check stopReason, so a bad API key fails loudly instead of exiting 0 with no output. A new Continuing the Loop section documents the ToolResultMessage shape and a two-turn example.
  • API-key docs: new Prerequisites section with the per-provider env-var table, all five api-keys helpers documented, and the quickstart resolves its key via requireCuaEnvApiKeyForModel. The shipped example now imports @onkernel/cua-ai (resolvable from the tarball), switches providers via CUA_MODEL, and docs/ ships in the npm files.
  • Version 0.2.0 + changelog: covers the unreleased post-0.1.0 breaking changes (coordinateSystem, computerTools, Yutori native tool_set, Anthropic explicit tools) plus this PR's renames; lockfile re-synced.
  • API surface consistency: all five provider namespaces follow one convention (<PROVIDER>_CUA_ACTION_TYPES, <PROVIDER>_COMPUTER_INSTRUCTIONS, computerTools/computerToolExecutors/createActionSchema/coordinateSystem/providerModule, <Provider>Action, ComputerToolsOptions), enforced by a parity test. CUA_BATCH_TOOL_NAME is now "computer_batch" (matching the only shipped batch tool) with anthropic.ANTHROPIC_BATCH_TOOL_NAME exported. yutori.computerTools honors and validates { actions }. registerCuaProviders() is exported and re-registers after pi-ai registry mutators. resolveCuaRuntimeSpec(input, options?) threads tool options through. Mouse buttons are typed as closed unions (CuaMouseButton/CuaDragMouseButton; wire schemas unchanged). Family annotations match only root + numeric revision/dated-snapshot suffixes, so gpt-5.4-mini/-nano/-pro no longer list as CUA-capable. gemini-2.5-computer-use-preview-10-2025 is removed from the catalog (rejects the function declarations this package sends); gemini: refs are accepted as an alias for google:. YutoriOptions/TzafonResponsesOptions are exported and aligned on keepToolNames; malformed Yutori tool calls degrade per-call instead of erroring the turn. JSDoc added across models.ts and api-keys.ts.
  • README trim: the Continuing the Loop section now leads with the cua-agent pointer (loop + browser execution live there); the executor/Yutori wire-format internals, the pi-ai versioning + registry side-effect notes, and the google-vs-gemini naming aside are removed. The root README marks the legacy cua-* packages deprecated — cua-ai/cua-agent are the canonical source — and the "start here" pointer leads with cua-agent.
  • CI test-list fix: the unit job runs the full suite (12 files / 83 tests, previously 2 hard-coded files) with integration/live tests split into vitest.integration.config.ts + npm run test:integration; release-cua-ai.yml drops its stale 6-file list.
  • Release-skill refresh: .agents/skills/release/SKILL.md uses the plain full-suite invocations and adds a post-publish install + import verification step.

Merge coupling: this PR also bumps packages/agent's @onkernel/cua-ai pin to 0.2.0 so the workspace package resolves locally (an exact 0.1.0 pin plus the 0.2.0 workspace version made npm ci nest the stale, ESM-broken published 0.1.0 under packages/agent, breaking the agent-e2e build). The de-vendor PR (#17) sets the same line to the same value, so the branches merge cleanly. Publishing order still applies: cua-ai/v0.2.0 before cua-agent.

Test plan

  • npm ci from scratch, then npm run build --workspace @onkernel/cua-ai (tsdown) — green
  • Full unit suite: 12 files / 83 tests pass; dist/ is a bundled ESM entry (no relative runtime imports beyond the emitted chunk)
  • npm run test:integration --workspace @onkernel/cua-ai collects only the integration file; ran live with available keys (2 passed, 3 skipped)
  • Monorepo npm run typecheck (tsc -b, includes packages/agent via project reference) — green
  • Packed tarball installed into a clean temp dir; plain-Node ESM import passes a full assertion set (getCuaModel, registerCuaProviders, CUA_BATCH_TOOL_NAME === "computer_batch", namespace parity, action-subset counts, gemini: alias)
  • Shipped example resolves from the installed tarball (package self-reference) and in-repo via --conditions=source; both fail loudly with the missing-key error before any request; example typechecks standalone
  • README claims cross-checked against source: pi-ai StopReason/ToolResultMessage shapes, pi-ai google env fallback (GEMINI_API_KEY only), per-provider action subsets (anthropic missing back/forward/url; yutori missing screenshot/url/cursor_position), env-var table order, button-coercion semantics
  • CHANGELOG breaking-change claims verified against the cua-ai/v0.1.0 tag (e.g. maxOutputTokens was accepted in 0.1.0)

🤖 Generated with Claude Code


Note

Medium Risk
0.2.0 introduces multiple documented breaking API and catalog changes; CI/release now gate on ESM smoke tests, but consumers and any code still on 0.1.0 naming must migrate.

Overview
@onkernel/cua-ai 0.2.0 fixes plain Node ESM consumption by bundling dist/ with tsdown (replacing extensionless tsc output that broke import "@onkernel/cua-ai"), ships docs/ in the tarball, and bumps @onkernel/cua-agent to depend on cua-ai@0.2.0. Root build/typecheck now build cua-ai first via tsdown.

CI and release move to Node 22, run the full cua-ai unit suite (integration via test:integration), and add pack + ESM import smoke in CI and release-cua-ai.yml; the release skill documents full-suite tests and post-publish import checks.

Public API and docs (many 0.2.0 breaking changes): unified provider namespace exports, CUA_BATCH_TOOL_NAME"computer_batch", stricter model family matching, removal of gemini-2.5-computer-use-preview from the catalog, gemini: alias for google:, exported registerCuaProviders() and resolveCuaRuntimeSpec(..., options), Yutori/Tzafon stream hardening, and expanded README (API keys, stopReason, tool loop). Root README positions cua-ai/cua-agent as canonical and marks legacy cua-* packages deprecated.

Reviewed by Cursor Bugbot for commit 7880147. Bugbot is set up for automated code reviews on this repo. Configure here.

rgarcia and others added 5 commits June 10, 2026 15:16
Switch packages/ai to NodeNext module resolution and add explicit .js
extensions to relative imports so the published dist loads in plain
Node ESM (previously failed with ERR_UNSUPPORTED_DIR_IMPORT). Run the
full unit suite in CI and release instead of hard-coded file lists by
excluding integration/live tests from the default vitest config, and
add a post-pack ESM import smoke test to CI and the release workflow.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Family annotations now cover only the root id plus numeric revision or
dated-snapshot suffixes, so named sibling variants like gpt-5.4-mini no
longer list as CUA-capable. Drop gemini-2.5-computer-use-preview-10-2025
from the catalog: it rejects the function-declaration tools this package
sends. Accept "gemini:" refs as an alias for the canonical "google:"
prefix, name the valid providers in the unsupported-provider error, and
add JSDoc to the models and api-keys entry points.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Standardize every provider namespace on <PROVIDER>_CUA_ACTION_TYPES and
<PROVIDER>_COMPUTER_INSTRUCTIONS, restore anthropic's action-type export,
and enforce the convention in tests. Align CUA_BATCH_TOOL_NAME with the
computer_batch tool Anthropic actually ships and export
ANTHROPIC_BATCH_TOOL_NAME. Make yutori.computerTools honor and validate
its options instead of silently ignoring them, and document that the
definitions are local mirrors stripped from the wire payload.

Export and align the Yutori/Tzafon stream option interfaces (both now
carry keepToolNames; redundant temperature/maxOutputTokens dropped),
type mouse buttons as closed unions, export registerCuaProviders and
make it re-register after pi-ai registry mutators clobber it, and thread
tool options through resolveCuaRuntimeSpec. Harden the Yutori stream so
one malformed tool call degrades to empty args instead of erroring the
whole turn, restore api-literal typing on the Tzafon stream, and add
unit coverage for Tzafon action normalization and argument unwrapping
plus mocked-stream tests for both custom providers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Document API key prerequisites and helpers, stopReason error handling,
a multi-turn tool-result loop, the complete export surface, and
per-provider canonical action subsets. Align the family-matcher docs
with the tightened semantics and drop the removed Gemini 2.5 CUA
preview model. Make the shipped quickstart import the package name
(resolvable from the npm tarball), resolve its API key loudly, check
stopReason, and switch providers via CUA_MODEL. Add packages/ai and
packages/agent to the root README.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a 0.2.0 changelog entry covering the unreleased breaking changes
since the 0.1.0 publish plus this release's fixes. Ship docs/ in the
tarball and run the in-repo example through the source export
condition. Update the release skill to full-suite test invocations and
add a post-publish step that installs and imports the published
package.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@rgarcia
rgarcia marked this pull request as ready for review June 10, 2026 18:26
@firetiger-agent

Copy link
Copy Markdown

Firetiger deploy monitoring skipped

This PR didn't match the auto-monitor filter configured on your GitHub connection:

PRs in the kernel, infra, hypeman, and hypeship repos. kernel is a ~mono repo with many logical services underneath, ensure to focus on the implicated service for the PR

Reason: PR targets packages/ai (a service package in the codebase), but the filter requires changes to the kernel, infra, hypeman, or hypeship repositories, which are not mentioned in this PR.

To monitor this PR anyway, reply with @firetiger monitor this.

rgarcia and others added 2 commits June 10, 2026 18:45
Replace the NodeNext + .js-specifier approach with a tsdown bundle:
dist/ is a single ESM file with bundled-in relative modules, deps stay
external, and src/test go back to extensionless imports. tsc -b remains
for typechecking only, emitting declarations to a gitignored dist-tsc/
so project references keep working. Root build now builds cua-ai before
tsc -b since dependents resolve its types from dist/.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lead the loop section with the cua-agent pointer, drop the executor/
Yutori wire-format internals, the pi-ai versioning and registry
side-effect notes, and the google-vs-gemini naming aside. Mark the
legacy cua-* packages deprecated in the root README and make the
"start here" pointer lead with cua-agent.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Comment thread packages/ai/tsconfig.build.json
tsdown evaluates its TypeScript config via Node's native type stripping
(node >=22.18); on Node 20 it falls back to the optional unrun loader,
which is not installed, so every job failed at the cua-ai build step.
Node 22 also matches the >=22.19.0 engines floor the de-vendor PR
introduces. Release workflows already run Node 24.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Comment thread packages/ai/src/runtime-spec.ts
The exact 0.1.0 pin plus the workspace version bump made npm ci nest
the published 0.1.0 (stale API, ESM-broken) under packages/agent, which
broke the agent-e2e job at the build step. Pinning 0.2.0 resolves the
workspace package again. The de-vendor PR sets the same value, so the
branches still merge cleanly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Node 20 engines mismatch
    • Updated the root Node engine range to ^22.18.0 || >=24.0.0 so advertised support matches the new tsdown build requirement.
  • ✅ Fixed: ESM smoke omits rejection handler
    • Added explicit .catch(...) handlers to both ESM smoke-test import checks so import failures always exit non-zero.

Create PR

Or push these changes by commenting:

@cursor push 6260abd7ef
Preview (6260abd7ef)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -31,7 +31,7 @@
           cd "$RUNNER_TEMP/esm-smoke"
           npm init -y
           npm install "$RUNNER_TEMP"/onkernel-cua-ai-*.tgz
-          node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof m.getCuaModel !== 'function') process.exit(1); })"
+          node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof m.getCuaModel !== 'function') process.exit(1); }).catch((err) => { console.error(err); process.exit(1); })"
 
   integration:
     runs-on: ubuntu-latest

diff --git a/.github/workflows/release-cua-ai.yml b/.github/workflows/release-cua-ai.yml
--- a/.github/workflows/release-cua-ai.yml
+++ b/.github/workflows/release-cua-ai.yml
@@ -71,7 +71,7 @@
           cd "$RUNNER_TEMP/esm-smoke"
           npm init -y
           npm install "$RUNNER_TEMP"/onkernel-cua-ai-*.tgz
-          node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof m.getCuaModel !== 'function') process.exit(1); })"
+          node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof m.getCuaModel !== 'function') process.exit(1); }).catch((err) => { console.error(err); process.exit(1); })"
 
       - name: Publish to npm
         run: npm publish --workspace @onkernel/cua-ai --access public

diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -24,7 +24,7 @@
     "clean": "tsc -b --clean && npm run clean:native --workspace @onkernel/ptywright --if-present"
   },
   "engines": {
-    "node": ">=20"
+    "node": "^22.18.0 || >=24.0.0"
   },
   "devDependencies": {
     "@types/node": "22.18.4",

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 7880147. Configure here.

Comment thread package.json
"build:cli": "npm run build --workspace @onkernel/cua-cli",
"dev": "tsc -b --watch",
"typecheck": "tsc -b",
"typecheck": "npm run build --workspace @onkernel/cua-ai && tsc -b",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node 20 engines mismatch

Medium Severity

The root engines.node field still allows Node 20, but @onkernel/cua-ai now builds with tsdown, which requires Node ^22.18.0 or ≥24. CI was bumped to 22 while the repo still advertises ≥20, so a normal npm run build or root typecheck can fail on Node 20 even though the project claims support.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7880147. Configure here.

Comment thread .github/workflows/ci.yml
cd "$RUNNER_TEMP/esm-smoke"
npm init -y
npm install "$RUNNER_TEMP"/onkernel-cua-ai-*.tgz
node --input-type=module -e "import('@onkernel/cua-ai').then((m) => { if (typeof m.getCuaModel !== 'function') process.exit(1); })"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ESM smoke omits rejection handler

Low Severity

The new post-pack ESM smoke step only chains .then() on import('@onkernel/cua-ai') and never handles rejection. If the tarball fails to load, the check may not exit with a non-zero code reliably, so CI could pass without proving the artifact is importable.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7880147. Configure here.

@cursor

This comment has been minimized.

@rgarcia
rgarcia merged commit bd8043f into main Jun 10, 2026
5 checks passed
@cursor

cursor Bot commented Jun 10, 2026

Copy link
Copy Markdown

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Yutori/Tzafon executor narrowing mismatch
    • resolveCuaRuntimeSpec now threads narrowed actions into payload context, and Yutori/Tzafon payload adapters skip native full-tool exposure (while filtering local tools to allowed actions) when actions are narrowed so emitted tool calls stay executable.

Create PR

Or push these changes by commenting:

@cursor push ebccaacc97
Preview (ebccaacc97)
diff --git a/packages/ai/src/providers/common.ts b/packages/ai/src/providers/common.ts
--- a/packages/ai/src/providers/common.ts
+++ b/packages/ai/src/providers/common.ts
@@ -442,6 +442,8 @@
 export interface CuaPayloadContext {
 	/** Tool names that should remain in the outbound provider payload even if the provider strips local CUA executors. */
 	keepToolNames?: readonly string[];
+	/** Canonical CUA actions enabled for this runtime, if narrowed by options. */
+	actions?: readonly CuaActionType[];
 }
 
 export type CuaPayloadHook = (payload: unknown, model: Model<Api>, context?: CuaPayloadContext) => unknown | Promise<unknown>;

diff --git a/packages/ai/src/providers/tzafon/provider.ts b/packages/ai/src/providers/tzafon/provider.ts
--- a/packages/ai/src/providers/tzafon/provider.ts
+++ b/packages/ai/src/providers/tzafon/provider.ts
@@ -117,14 +117,20 @@
 	if (!payload || typeof payload !== "object") return undefined;
 	const current = payload as { tools?: unknown };
 	const keepToolNames = new Set(context?.keepToolNames ?? []);
+	const useNativeComputerUse = shouldUseTzafonNativeComputerUse(context);
+	const allowedActions = new Set(context?.actions ?? []);
 	const existingTools = Array.isArray(current.tools) ? current.tools : [];
-	const shouldAddComputerUse = existingTools.some((tool) => {
-		const name = readToolName(tool);
-		return Boolean(name && TZAFON_LOCAL_ACTION_TOOL_NAMES.has(name) && !keepToolNames.has(name));
-	});
+	const shouldAddComputerUse = useNativeComputerUse
+		? existingTools.some((tool) => {
+				const name = readToolName(tool);
+				return Boolean(name && TZAFON_LOCAL_ACTION_TOOL_NAMES.has(name) && !keepToolNames.has(name));
+			})
+		: false;
 	const tools = existingTools.filter((tool) => {
 		const name = readToolName(tool);
-		return !name || keepToolNames.has(name) || !TZAFON_LOCAL_ACTION_TOOL_NAMES.has(name);
+		if (!name || keepToolNames.has(name) || !TZAFON_LOCAL_ACTION_TOOL_NAMES.has(name)) return true;
+		if (!useNativeComputerUse && allowedActions.size > 0) return allowedActions.has(name as (typeof CUA_ACTION_TYPES)[number]);
+		return !useNativeComputerUse;
 	});
 	return {
 		...(payload as Record<string, unknown>),
@@ -132,6 +138,12 @@
 	};
 }
 
+function shouldUseTzafonNativeComputerUse(context?: CuaPayloadContext): boolean {
+	if (!context?.actions) return true;
+	const allowed = new Set(context.actions);
+	return CUA_ACTION_TYPES.every((action) => allowed.has(action));
+}
+
 /** Derive a unique canonical tool-call id for a Tzafon computer action. */
 export function tzafonToolCallId(callId: string, actionIndex: number): string {
 	return actionIndex === 0 ? callId : `${callId}:${actionIndex}`;

diff --git a/packages/ai/src/providers/yutori/provider.ts b/packages/ai/src/providers/yutori/provider.ts
--- a/packages/ai/src/providers/yutori/provider.ts
+++ b/packages/ai/src/providers/yutori/provider.ts
@@ -17,6 +17,7 @@
 	isYutoriLocalActionToolName,
 	toCanonicalActions,
 	yutoriToolSetForModel,
+	YUTORI_CUA_ACTION_TYPES,
 	YUTORI_N15_EXPANDED_ACTION_TYPES,
 } from "./actions";
 import { canonicalToolCallArguments, canonicalToolCallName, type CuaPayloadContext } from "../common";
@@ -45,20 +46,31 @@
 	if (!payload || typeof payload !== "object") return undefined;
 	const current = payload as { tools?: unknown };
 	const keepToolNames = new Set(context?.keepToolNames ?? []);
+	const useNativeToolSet = shouldUseYutoriNativeToolSet(context);
+	const allowedActions = new Set<string>(context?.actions ?? []);
 	const tools = Array.isArray(current.tools)
 		? current.tools.filter((tool) => {
 				const name = readToolName(tool);
-				return !name || keepToolNames.has(name) || !isYutoriLocalActionToolName(name);
+				if (!name || keepToolNames.has(name) || !isYutoriLocalActionToolName(name)) return true;
+				if (!useNativeToolSet && allowedActions.size > 0) return allowedActions.has(name);
+				return !useNativeToolSet;
 			})
 		: undefined;
-	const toolSet = model ? yutoriToolSetForModel(model.id) : undefined;
+	const toolSet = useNativeToolSet && model ? yutoriToolSetForModel(model.id) : undefined;
 	return {
 		...(payload as Record<string, unknown>),
+		...(useNativeToolSet ? {} : { tool_set: undefined, disable_tools: undefined }),
 		...(toolSet ? { tool_set: toolSet, disable_tools: [...YUTORI_N15_EXPANDED_ACTION_TYPES] } : {}),
 		...(tools && tools.length > 0 ? { tools } : { tools: undefined }),
 	};
 }
 
+function shouldUseYutoriNativeToolSet(context?: CuaPayloadContext): boolean {
+	if (!context?.actions) return true;
+	const allowed = new Set(context.actions);
+	return YUTORI_CUA_ACTION_TYPES.every((action) => allowed.has(action));
+}
+
 async function runYutoriStream(
 	stream: ReturnType<typeof createAssistantMessageEventStream>,
 	model: Model<Api>,

diff --git a/packages/ai/src/runtime-spec.ts b/packages/ai/src/runtime-spec.ts
--- a/packages/ai/src/runtime-spec.ts
+++ b/packages/ai/src/runtime-spec.ts
@@ -32,6 +32,13 @@
 	const model = typeof input === "string" ? getCuaModel(input) : input;
 	const provider = providerForModel(model);
 	const mod: CuaProviderModule = PROVIDERS[provider];
+	const onPayload = mod.onPayload
+		? (payload: unknown, requestModel: typeof model, context?: Parameters<NonNullable<CuaProviderModule["onPayload"]>>[2]) =>
+				mod.onPayload?.(payload, requestModel, {
+					...context,
+					...(options?.actions ? { actions: options.actions } : {}),
+				})
+		: undefined;
 	return {
 		model,
 		provider,
@@ -40,6 +47,6 @@
 		defaultSystemPrompt: mod.buildSystemPrompt(),
 		coordinateSystem: mod.coordinateSystem(),
 		screenshot: mod.screenshot,
-		onPayload: mod.onPayload,
+		onPayload,
 	};
 }

diff --git a/packages/ai/test/runtime-spec.test.ts b/packages/ai/test/runtime-spec.test.ts
--- a/packages/ai/test/runtime-spec.test.ts
+++ b/packages/ai/test/runtime-spec.test.ts
@@ -53,5 +53,9 @@
 		const yutoriSpec = resolveCuaRuntimeSpec("yutori:n1.5-latest", { actions: ["click"] });
 		expect(yutoriSpec.toolDefinitions).toEqual([]);
 		expect(yutoriSpec.toolExecutors.map((executor) => executor.definition.name)).toEqual(["click"]);
+
+		const tzafonSpec = resolveCuaRuntimeSpec("tzafon:tzafon.northstar-cua-fast", { actions: ["click"] });
+		expect(tzafonSpec.toolDefinitions.map((tool) => tool.name)).toEqual(["click"]);
+		expect(tzafonSpec.toolExecutors.map((executor) => executor.definition.name)).toEqual(["click"]);
 	});
 });

diff --git a/packages/ai/test/tzafon-payload.test.ts b/packages/ai/test/tzafon-payload.test.ts
--- a/packages/ai/test/tzafon-payload.test.ts
+++ b/packages/ai/test/tzafon-payload.test.ts
@@ -42,6 +42,25 @@
 		]);
 	});
 
+	it("keeps narrowed local action tools and skips native computer_use", () => {
+		const payload = {
+			tools: [
+				{ type: "function", name: "click" },
+				{ type: "function", name: "move" },
+				{ type: "function", name: "custom_tool" },
+			],
+		};
+
+		const next = tzafon.tzafonComputerUseOnPayload(payload, undefined, {
+			actions: ["click"],
+		}) as { tools?: Array<{ type?: string; name?: string }> };
+
+		expect(next.tools).toEqual([
+			{ type: "function", name: "click" },
+			{ type: "function", name: "custom_tool" },
+		]);
+	});
+
 	it("returns undefined for non-object payloads", () => {
 		expect(tzafon.tzafonComputerUseOnPayload(undefined)).toBeUndefined();
 		expect(tzafon.tzafonComputerUseOnPayload("x")).toBeUndefined();

diff --git a/packages/ai/test/yutori-payload.test.ts b/packages/ai/test/yutori-payload.test.ts
--- a/packages/ai/test/yutori-payload.test.ts
+++ b/packages/ai/test/yutori-payload.test.ts
@@ -39,6 +39,26 @@
 		expect(next.tools?.map((tool) => tool.function?.name)).toEqual(["batch_computer_actions"]);
 	});
 
+	it("keeps narrowed local action tools and skips the native tool set", () => {
+		const payload = {
+			tools: [
+				{ type: "function", function: { name: "click" } },
+				{ type: "function", function: { name: "move" } },
+				{ type: "function", function: { name: "custom_tool" } },
+			],
+		};
+		const next = yutori.yutoriNativeToolSetOnPayload(payload, { id: "n1.5-latest" } as never, {
+			actions: ["click"],
+		}) as {
+			tool_set?: string;
+			disable_tools?: string[];
+			tools?: Array<{ function?: { name?: string } }>;
+		};
+		expect(next.tool_set).toBeUndefined();
+		expect(next.disable_tools).toBeUndefined();
+		expect(next.tools?.map((tool) => tool.function?.name)).toEqual(["click", "custom_tool"]);
+	});
+
 	it("returns undefined for non-object payloads", () => {
 		expect(yutori.yutoriNativeToolSetOnPayload(undefined)).toBeUndefined();
 		expect(yutori.yutoriNativeToolSetOnPayload("x")).toBeUndefined();

You can send follow-ups to the cloud agent here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant