diff --git a/.cursor/plans/cua-ai-agent.md b/.cursor/plans/cua-ai-agent.md deleted file mode 100644 index c5f571c9..00000000 --- a/.cursor/plans/cua-ai-agent.md +++ /dev/null @@ -1,47 +0,0 @@ -# CUA AI And Agent SDK Plan - -Implement additive packages under `packages/ai` and `packages/agent`, plus -`docs/DESIGN.md`. Do not delete existing `packages/cua-` packages, -do not delete `packages/cua-translator`, and do not migrate `packages/cua-cli` -in this pass. - -This pass should make the follow-up work easy: - -- Migrate `@onkernel/cua-cli` to use `@onkernel/cua-ai` and `@onkernel/cua-agent`. -- Delete the old `@onkernel/cua-` packages. -- Delete the old `@onkernel/cua-translator` package if the port proves complete. - -## Workspace - -- Add `packages/ai` and `packages/agent` to workspaces and TypeScript project references. -- The new packages must not depend on existing provider packages or `@onkernel/cua-translator`. -- Use top-level package files: `README.md`, `CHANGELOG.md`, `package.json`, - `tsconfig.build.json`, `vitest.config.ts`, `src/`, and `test/`. - -## `@onkernel/cua-ai` - -- Re-export pi-ai primitives. -- Expose only provider-qualified CUA model refs through `getCuaModel(ref)`. -- Do not expose a default model helper or default model constant. -- Port provider schemas, prompt constants, model metadata, and direct provider - API logic from existing provider packages into the new package. -- Register custom Tzafon and Yutori providers locally. -- Keep Kernel browser execution out of this package. - -## `@onkernel/cua-agent` - -- Re-export pi-agent-core primitives. -- Keep translator/session internals private. -- Expose `createCuaComputerTools({ provider, browser, client })`. -- Expose `createCuaAgent()` returning the pi-agent-core `Agent` directly. -- Keep model and tools in `initialState`, matching pi-agent-core ergonomics. -- If `initialState.tools` is omitted, install provider-specific CUA computer tools. -- If `initialState.tools` is provided, use it exactly. -- Do not bundle coding/file tools. - -## Docs And Validation - -- Add package READMEs analogous in structure to pi's README files, but focused - on what Kernel adds. -- Add `docs/DESIGN.md` with the product principles. -- Validate with build, typecheck, and package tests. diff --git a/README.md b/README.md index 5bae352c..dbbec9bf 100644 --- a/README.md +++ b/README.md @@ -85,12 +85,16 @@ flowchart LR git clone https://github.com/kernel/cua cd cua npm install -npm run build -# put `cua` on PATH (creates ~/.local/bin/cua → bin/cua): -mkdir -p ~/.local/bin -ln -s "$(pwd)/bin/cua" ~/.local/bin/cua -# make sure ~/.local/bin is on $PATH (most distros already do) +# run the CLI directly from source (no global install required): +npx tsx packages/cli/src/cli.ts --help + +# if you want `cua` on $PATH from any directory, add a shell function to +# your rc that pins the repo location while preserving the caller's cwd +# (so `--out`, transcript bucketing, and `.agents/skills` discovery use +# the directory you invoked from), e.g. in ~/.bashrc: +# CUA_REPO=/absolute/path/to/cua +# cua() { "$CUA_REPO/node_modules/.bin/tsx" "$CUA_REPO/packages/cli/src/cli.ts" "$@"; } # set API keys via env vars export OPENAI_API_KEY=sk-... # for gpt-5.5 @@ -318,8 +322,6 @@ ln -s "$(pwd)/skills/cua-cli" ~/.agents/skills/cua-cli ## Project layout ``` -bin/ -└── cua # POSIX wrapper script (symlink into your $PATH) skills/ └── cua-cli/SKILL.md # skill aimed at OTHER agents driving cua via shell packages/ diff --git a/bin/cua b/bin/cua deleted file mode 100755 index 0b6b56d9..00000000 --- a/bin/cua +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/sh -# Wrapper that runs the built cua CLI from this checkout. -# Symlink this file into ~/.local/bin/cua (or anywhere on $PATH) to use as `cua`. -DIR="$(cd "$(dirname "$0")/.." && pwd)" -exec node "$DIR/packages/cli/dist/cli.js" "$@" diff --git a/docs/DESIGN.md b/docs/DESIGN.md deleted file mode 100644 index 9f722c12..00000000 --- a/docs/DESIGN.md +++ /dev/null @@ -1,73 +0,0 @@ -# CUA SDK Design - -This document captures the evergreen product principles for Kernel's -computer-use SDK packages. - -## Package The Boring Plumbing - -Kernel packages should make the common browser-control work disappear: - -- Kernel browser session wiring -- screenshots and screenshot reinjection -- coordinate normalization -- provider-specific computer tool schemas -- tool execution against Kernel browser APIs -- provider registration -- context and payload quirks -- sensible default prompts - -These details are common to most CUA agents and are easy to get subtly wrong. - -## Do Not Over-Own The Agent - -Kernel should not hide the agent architecture from users. Builders should keep -control over: - -- system prompts -- context and memory strategy -- custom tools -- streaming UI -- orchestration policy -- transport hooks and payload inspection - -The SDK should make the default path pleasant while keeping pi's primitives -visible and replaceable. - -## Layering - -```mermaid -flowchart TD - piAi["pi-ai: Model, Context, Message, Tool, transport"] - piAgent["pi-agent-core: Agent, AgentOptions, AgentTool, events"] - cuaAi["@onkernel/cua-ai: CUA models and providers"] - cuaAgent["@onkernel/cua-agent: Kernel browser tool execution"] - app["User CUA agent"] - - piAi --> cuaAi - piAgent --> cuaAgent - cuaAi --> cuaAgent - cuaAgent --> app -``` - -`@onkernel/cua-ai` is the direct model-call layer. It is for calling -CUA-capable providers and working with pi-ai contexts, messages, and tools. - -`@onkernel/cua-agent` is the optional stateful loop layer. It adds Kernel -browser execution and returns a pi-agent-core `Agent`. - -## Keep Model Refs Explicit - -CUA model refs should be provider-qualified, for example -`openai:gpt-5.5` or `yutori:n1.5-latest`. This keeps examples, logs, -persisted config, and transcripts unambiguous. The SDK should not export a -default CUA model. - -## Current Surface - -`@onkernel/cua-ai`, `@onkernel/cua-agent`, and `@onkernel/cua-cli` are the -public SDK surface. `@onkernel/cua-cli` consumes both `@onkernel/cua-ai` (for -the CUA model catalog and API-key helpers) and `@onkernel/cua-agent` (for the -`CuaAgentHarness` and the re-exported pi-agent-core primitives). The legacy -provider-specific packages and the public translator package have been removed -from the workspace; provider quirks now live inside `@onkernel/cua-ai`'s -internal providers and `@onkernel/cua-agent`'s tool translator. diff --git a/docs/architecture.md b/docs/architecture.md index ec1f089b..568ea16c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -3,6 +3,47 @@ This document explains how `cua` is wired together. It's aimed at someone who wants to read the code, contribute, or fork. +## Product principles + +These are the evergreen product principles for Kernel's computer-use SDK +packages. They explain the *why* behind the technical invariants below. + +### Package the boring plumbing + +Kernel's SDK packages should make the common browser-control work disappear: + +- Kernel browser session wiring +- screenshots and screenshot reinjection +- coordinate normalization +- provider-specific computer-tool schemas +- tool execution against Kernel browser APIs +- provider registration +- context and payload quirks +- sensible default prompts + +These details are common to most CUA agents and are easy to get subtly wrong. + +### Do not over-own the agent + +The SDK should not hide the agent architecture from users. Builders keep +control over: + +- system prompts +- context and memory strategy +- custom tools +- streaming UI +- orchestration policy +- transport hooks and payload inspection + +The default path stays pleasant, but pi's primitives remain visible and +replaceable. + +### Keep model refs explicit + +CUA model refs are provider-qualified — for example `openai:gpt-5.5` or +`yutori:n1.5-latest`. This keeps examples, logs, persisted config, and +transcripts unambiguous. The SDK does not export a default CUA model. + ## Design goals and invariants - `@onkernel/cua-ai` owns provider-specific policy: the curated diff --git a/docs/cua-cli-harness-migration.md b/docs/cua-cli-harness-migration.md index 8c2e542a..5cefdcdb 100644 --- a/docs/cua-cli-harness-migration.md +++ b/docs/cua-cli-harness-migration.md @@ -1,6 +1,6 @@ # cua-cli → CuaAgentHarness migration plan -Status: approved, in progress. +Status: completed. `@onkernel/cua-cli` predates the public SDK packages. It hand-assembles a pi 0.67 (`@mariozechner/*`) `Agent` from the deprecated provider packages @@ -202,12 +202,9 @@ Strictly ordered; each PR merges before the next starts. update root `package.json` workspaces, root tsconfig references, CI, README workspace table + mermaid diagram, and any references under `docs/` and `skills/`. -- Out of scope (manual follow-up, requires npm publish rights): - `npm deprecate` of the published `@onkernel/cua-*` provider packages. - Acceptance: clean `npm install` + build + tests from a fresh checkout. ## Manual follow-ups after the migration -- `npm deprecate` the published deprecated packages. - Decide on auto-compaction trigger and proper context-file injection. - Release a new `@onkernel/cua-cli` version per `docs/npm-releases.md`. diff --git a/packages/cli/README.md b/packages/cli/README.md index 5176972c..cd2f682b 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -10,10 +10,17 @@ coding tools for workspace access. ## Install (from the monorepo) ```bash +# from the repo root: npm install -npm run build -ln -s "$(pwd)/bin/cua" ~/.local/bin/cua # put `cua` on your $PATH -cua --help +# run directly from source via tsx (no global install required): +npx tsx packages/cli/src/cli.ts --help + +# optional: pin a shell function in your rc so `cua` works from any cwd +# while preserving the caller's directory (so `--out`, transcript +# bucketing, and `.agents/skills` discovery use the directory you +# invoked from): +# CUA_REPO=/absolute/path/to/cua +# cua() { "$CUA_REPO/node_modules/.bin/tsx" "$CUA_REPO/packages/cli/src/cli.ts" "$@"; } ``` ## Usage