Add Grok 4.5 computer-use provider

[rgarcia]

Summary

• upgrade the aligned pi packages to 0.80.6 and use pi-ai's built-in xai:grok-4.5 catalog entry instead of maintaining a local model copy
• route Grok 4.5 through the OpenAI-compatible Responses transport with normalized 0-1000 computer tools
• preserve xAI encrypted reasoning, map unsupported thinking levels to low, apply xAI's long-context pricing tier, keep browser actions serial, and continue tool loops through previous_response_id
• preserve pi-ai's builtin xAI provider/auth/catalog while exposing Grok through the agent, harness, CLI, and documentation
• extend the repo's update-models skill with xAI model discovery, function-tool smokes, action probes, official SDK evidence, and docs drift checks; fix its pi-ai registry lookup to use getBuiltinModel

Testing

• npm ci
• npm run typecheck
• npm test --workspace @onkernel/cua-ai (158 tests)
• npm test --workspace @onkernel/cua-agent (160 tests; live matrix skipped in unit run)
• npm test --workspace @onkernel/cua-cli (88 tests; TUI fixtures skipped)
• npm run test:integration --workspace @onkernel/cua-ai -- test/computer-tool.integration.test.ts (8 live provider tests)
• live Grok CuaAgent and CuaAgentHarness browser smokes
• live CLI browser navigation, screenshot, normalized click, and final-URL verification
• live xai:grok-4.5 quickstart
• xAI model discovery, action probe, official SDK audit, and docs drift checks
• npm run build --workspace @onkernel/cua-cli

---

Note

Medium Risk
Introduces a new live API provider path (auth, Responses threading, reasoning, and tool loops) across ai/agent/cli, though it largely mirrors the existing Meta Responses pattern and is covered by unit and integration tests.

Overview
Adds xAI Grok 4.5 as a first-class CUA provider using screenshot input and developer-supplied function tools (normalized 0–1000 coordinates), not a native computer tool—same pattern as Meta Muse.

Runtime: New xai provider module and xai-cua-responses stream wraps pi-ai’s OpenAI Responses transport with store: true, parallel_tool_calls: false, encrypted reasoning replay, and previous_response_id threading (agent/harness responseThreading now includes xAI). getCuaModel("xai:grok-4.5") uses pi-ai 0.80.6’s built-in catalog entry with routing overrides (API id, thinking-level map, 200k long-context pricing tier).

Surface area: CuaProvider / annotations / XAI_API_KEY, CLI and README examples, integration and live e2e matrices, and CI secrets for XAI_API_KEY.

Tooling: The update-models skill gains xAI discovery, smokes, action probes, and doc drift checks; registry lookup switches to getBuiltinModel.

^{Reviewed by Cursor Bugbot for commit 4be7168. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Off thinking maps to null
  • Updated the xAI Grok 4.5 thinking map so off now maps to low (instead of omitting reasoning), and adjusted the corresponding model test expectation.

Or push these changes by commenting:

@cursor push 5bfae1d7ba

Preview (5bfae1d7ba)

diff --git a/packages/ai/src/models.ts b/packages/ai/src/models.ts
--- a/packages/ai/src/models.ts
+++ b/packages/ai/src/models.ts
@@ -301,7 +301,7 @@
 				...base,
 				api: XAI_CUA_RESPONSES_API,
 				baseUrl: "https://api.x.ai/v1",
-				thinkingLevelMap: { off: null, minimal: "low", xhigh: "high" },
+				thinkingLevelMap: { off: "low", minimal: "low", xhigh: "high" },
 				cost: { input: 2, output: 6, cacheRead: 0.5, cacheWrite: 0 },
 				contextWindow: 500_000,
 				maxTokens: 500_000,

diff --git a/packages/ai/test/models.test.ts b/packages/ai/test/models.test.ts
--- a/packages/ai/test/models.test.ts
+++ b/packages/ai/test/models.test.ts
@@ -67,7 +67,7 @@
 		expect(grok.baseUrl).toBe("https://api.x.ai/v1");
 		expect(grok.contextWindow).toBe(500_000);
 		expect(grok.maxTokens).toBe(500_000);
-		expect(grok.thinkingLevelMap?.off).toBeNull();
+		expect(grok.thinkingLevelMap?.off).toBe("low");
 		expect(grok.thinkingLevelMap?.minimal).toBe("low");
 		expect(grok.thinkingLevelMap?.xhigh).toBe("high");
 	});

_{You can send follow-ups to the cloud agent here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Missing xAI provider docs links
  • Added TSDoc source links and controlling comments for both XAI_COMPUTER_INSTRUCTIONS and coordinateSystem in the xAI provider module.

Or push these changes by commenting:

@cursor push 035a5c0c9b

Preview (035a5c0c9b)

diff --git a/packages/ai/src/providers/xai/index.ts b/packages/ai/src/providers/xai/index.ts
--- a/packages/ai/src/providers/xai/index.ts
+++ b/packages/ai/src/providers/xai/index.ts
@@ -19,6 +19,13 @@
 } from "./provider";
 export type { XaiResponsesOptions } from "./provider";
 
+/**
+ * Base instructions for xAI computer-use runs, including CUA's normalized 0-1000 coordinate contract.
+ * Sources:
+ * - https://docs.x.ai/developers/grok-4-5
+ * - https://docs.x.ai/developers/tools/function-calling
+ * - https://docs.x.ai/developers/model-capabilities/images/understanding
+ */
 export const XAI_COMPUTER_INSTRUCTIONS = `You control a Kernel cloud browser through individual browser tools. Coordinates are normalized from 0 to 1000 relative to the most recent screenshot, with (0, 0) at the top left and (1000, 1000) at the bottom right. Base each action on the latest observed state and request a screenshot when you need a fresh view.`;
 
 /**
@@ -30,6 +37,10 @@
 
 /**
  * Return the normalized coordinate contract CUA supplies to Grok function tools.
+ * Sources:
+ * - https://docs.x.ai/developers/grok-4-5
+ * - https://docs.x.ai/developers/tools/function-calling
+ * - https://docs.x.ai/developers/model-capabilities/images/understanding
  */
 export function coordinateSystem(): ComputerToolCoordinateSystem {
 	return { type: "normalized", range: [0, 1000] };

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 42b057b. Configure here.}