diff --git a/.vitepress/sidebars/concepts.ts b/.vitepress/sidebars/concepts.ts index 0a690e1..e7bbe26 100644 --- a/.vitepress/sidebars/concepts.ts +++ b/.vitepress/sidebars/concepts.ts @@ -32,6 +32,37 @@ export const conceptsSidebar: DefaultTheme.SidebarItem[] = [ }, ], }, + { + text: "Client", + items: [ + { + text: "Authentication", + items: [ + { + text: "Methods", + link: "/concepts/client/auth_methods", + }, + { + text: "Caching of Tokens", + link: "/concepts/client/auth_caching", + }, + ], + }, + { + text: "Community implementations", + items: [ + { + text: "Golang SDK", + link: "/concepts/client/community_golang", + }, + { + text: "Python SDK", + link: "/concepts/client/community_python", + }, + ], + }, + ], + }, { text: "Essentials", items: [ diff --git a/.vitepress/sidebars/tutorials.ts b/.vitepress/sidebars/tutorials.ts index ec70ded..0038e3a 100644 --- a/.vitepress/sidebars/tutorials.ts +++ b/.vitepress/sidebars/tutorials.ts @@ -19,4 +19,13 @@ export const tutorialsSidebar: DefaultTheme.SidebarItem[] = [ }, ], }, + { + text: "Client SDK", + items: [ + { + text: "Using the Scanner API", + link: "/tutorials/client/scanner", + }, + ], + }, ]; diff --git a/src/concepts/client/auth_caching.md b/src/concepts/client/auth_caching.md new file mode 100644 index 0000000..b74155c --- /dev/null +++ b/src/concepts/client/auth_caching.md @@ -0,0 +1,38 @@ +# Caching of Tokens + +After a script authenticates against a Caido instance, the SDK keeps the resulting access token in memory for the rest of the process. To preserve that state across script runs without repeating the [authentication flow](./auth_methods.md) every time, the SDK can serialize it to a cache and reload it on the next connection. + +The SDK ships with two built-in caches (file-based for Node and localStorage-based for browsers) and accepts a custom implementation for everything else. + +## File + +The file cache writes the token state to a JSON file on disk. It is the natural choice for Node.js scripts and CI/CD environments where local disk is available and acceptable. + +The file is written with `0o600` permissions (read and write for the owner only) to limit accidental exposure to other users on the same machine. Beyond that, the file content is plain JSON: anyone with file system access can read the tokens. + +The file cache is not suitable for shared, networked, or untrusted storage. For those scenarios, see [Custom Implementation](#custom-implementation). + +## Local Storage + +The localStorage cache writes the token state to the browser's `localStorage` under a key. It is the built-in choice for SDK use inside a browser environment. + +Like the file cache, the stored value is plain JSON. Browser sandboxing scopes the storage to the origin, but anything running in the same origin (including third-party scripts loaded into the page) can read it. + +## Custom Implementation + +For everything that does not fit the file or localStorage models, scripts can implement the `TokenCache` interface (`load`, `save`, `clear`) and pass an instance to the `Client` constructor. Common motivations include: + +- Encrypted storage on disk +- Tokens stored in a remote secret manager (Vault, AWS Secrets Manager, etc.) +- Shared cache across multiple workers (Redis, database) +- In-memory cache for tests + +A custom cache is opaque to the SDK: it only calls the three interface methods, and the implementation owns where and how the tokens are stored. + +## Refresh + +Access tokens have a limited lifetime. When an authenticated request fails because the access token has expired or been revoked, the SDK uses the cached refresh token to mint a new access token through the instance's `refreshAuthenticationToken` mutation. The new tokens replace the old ones in memory and are written back to the cache. + +The renewal happens reactively rather than on a timer: the SDK does not consult the cached `expiresAt` to refresh proactively. It waits until the server rejects a request, then refreshes and retries. From the script's point of view, the failure is invisible. + +A cache holding only an access token (no refresh token) can serve a single short run but cannot recover when the access token expires. Pairing access tokens with refresh tokens is what keeps cached state usable over time. diff --git a/src/concepts/client/auth_methods.md b/src/concepts/client/auth_methods.md new file mode 100644 index 0000000..4dc6227 --- /dev/null +++ b/src/concepts/client/auth_methods.md @@ -0,0 +1,29 @@ +# Authentication Methods + +The Client SDK supports three methods for authenticating a script against a Caido instance: Personal Access Token, Browser Login, and Direct Token. They all produce the same kind of access token, but they differ in how that token is obtained. + +Two of them use the same OAuth 2.0 Device Authorization grant that the Caido desktop and web applications use, described in [Instance Authentication](https://docs.caido.io/app/concepts/instance_authentication.html). The third skips the flow entirely. + +## Personal Access Token + +A Personal Access Token (PAT) is a long-lived credential created in the Caido Dashboard. When the SDK authenticates with a PAT, it starts the device authorization flow on the instance and then automatically approves it by calling the Caido Cloud API with the PAT as a Bearer credential. No human interaction is required. + +This is the natural choice for headless environments: scripts, CI/CD pipelines, scheduled jobs, and anything that runs without a person nearby. A PAT is tied to the user that created it and inherits that user's permissions on the resources it targets, whether that's a personal account or a Team. + +The PAT itself is not used as the credential for API calls. Once the Cloud approves the device flow, the instance delivers an access token and a refresh token to the script, and those are what subsequent API calls carry as Bearer tokens. + +The full mechanics of PATs as a credential, where to create one, and how they relate to accounts and Teams are covered in [Personal Access Token](https://docs.caido.io/app/concepts/pat.html) in the user docs. + +## Browser Login + +Browser Login uses the same device authorization flow as PAT, but with a person in the loop instead of the Cloud. The SDK starts the flow on the instance, receives a verification URL and a short user code, and surfaces them to the script through an `onRequest` callback. A person opens the URL in a browser, enters the code, and approves the request on the [Caido Dashboard](https://dashboard.caido.io). The SDK waits on a WebSocket subscription until the instance delivers the resulting token pair. + +This suits interactive scripts, developer tools, and one-off automations where the person running the script can complete the approval step at the time of authentication. Outside of who approves the request, the resulting token pair is identical to one obtained through a PAT. + +## Direct Token + +Direct Token skips the device authorization flow entirely. Instead of negotiating with the instance and the Cloud, the script provides an access token, and optionally a refresh token, that it has already obtained through some other channel. The SDK uses the token as-is on every authenticated request. + +This pattern fits when the surrounding system already manages tokens for the script. Examples include a parent process that ran the device flow itself and passed the result down, a service that mints tokens out-of-band, or tests that need deterministic credentials. + +If only an access token is provided, the SDK has no way to obtain a new one when it expires. The script must rotate the token externally or switch to one of the other two methods. Providing both an access token and a refresh token restores the automatic renewal behavior covered in [Caching of Tokens](./auth_caching.md). diff --git a/src/concepts/client/community_golang.md b/src/concepts/client/community_golang.md new file mode 100644 index 0000000..00f0029 --- /dev/null +++ b/src/concepts/client/community_golang.md @@ -0,0 +1,22 @@ +# Golang SDK + +A community-maintained Go port of the official [JavaScript Client SDK](https://github.com/caido/sdk-js), hosted at [`caido-community/sdk-go`](https://github.com/caido-community/sdk-go) under the MIT license. It mirrors the API surface of `@caido/sdk-client` and uses [`genqlient`](https://github.com/Khan/genqlient) for type-safe GraphQL code generation, tracking the same schema the official SDK consumes. + +The Go SDK exposes the same domain-specific clients as the JavaScript SDK (requests, intercept, replay, findings, scopes, projects, environments, hosted files, workflows, tasks, filters, plugins, and more), plus low-level GraphQL access for operations not covered by domain methods. + +Initialization mirrors the JavaScript SDK's `Client` constructor: + +```go +client, _ := caido.NewClient(caido.Options{ + URL: "http://localhost:8080", + Auth: caido.PATAuth("caido_xxxxx"), +}) +client.Connect(context.Background()) +``` + +## Differences from the JavaScript SDK + +Coverage of the authentication model is intentional rather than exhaustive: + +- **Browser Login** is not exposed in the public API. Only [Personal Access Token](./auth_methods.md#personal-access-token) (`caido.PATAuth`) and [Direct Token](./auth_methods.md#direct-token) (`caido.TokenAuth(accessToken, refreshToken)`) are supported. +- **[Token caching](./auth_caching.md)** is not built in. Tokens are held in memory for the lifetime of the `Client`, and refresh is driven by an optional `TokenRefreshFunc` callback that the script provides. Persisting tokens across runs is the script's responsibility. diff --git a/src/concepts/client/community_python.md b/src/concepts/client/community_python.md new file mode 100644 index 0000000..9b26fac --- /dev/null +++ b/src/concepts/client/community_python.md @@ -0,0 +1,27 @@ +# Python SDK + +A community-maintained Python port of the official [JavaScript Client SDK](https://github.com/caido/sdk-js), hosted at [`caido-community/sdk-py`](https://github.com/caido-community/sdk-py) under the MIT license. The client is published to PyPI as [`caido-sdk-client`](https://pypi.org/project/caido-sdk-client/). + +The repository is a monorepo with two packages that mirror the JavaScript split: + +- `caido-sdk-client` is the high-level client that scripts use to interact with a Caido instance. +- `caido-server-auth` is the lower-level authentication library that the client builds on. + +The Python SDK is `asyncio`-based, and its type names match the JavaScript SDK closely (`Client`, `PATAuthOptions`, `AuthCacheFile`, and so on). + +A minimal usage looks like: + +```python +client = Client( + "http://localhost:8080", + auth=PATAuthOptions( + pat="caido_xxxxxx", + cache=AuthCacheFile(file=".secrets.json"), + ), +) +await client.connect() +``` + +## Differences from the JavaScript SDK + +The Python SDK supports all three [authentication methods](./auth_methods.md) and the file and custom variants of [token caching](./auth_caching.md). The `localStorage` cache variant has no equivalent in Python, since the language does not have a browser storage model. diff --git a/src/tutorials/client/scanner.md b/src/tutorials/client/scanner.md new file mode 100644 index 0000000..8b66a80 --- /dev/null +++ b/src/tutorials/client/scanner.md @@ -0,0 +1,378 @@ +# Using the Scanner API + +The goal of this tutorial is to drive the [Caido Scanner](https://github.com/caido-community/scanner) plugin from an external script using the [Client SDK](https://github.com/caido/sdk-js). By the end, you will be able to install Scanner against a Caido instance, run an active scan against a request already captured in your proxy history, and read the resulting findings, all without opening the Caido UI. + +This is useful for security automation (running scans on a schedule, scanning batches of requests collected by another tool), CI/CD pipelines (checking new endpoints against a baseline), and any workflow where opening the Caido UI is not an option. + +## 1. Prerequisites + +::: info Requirements +- [Node.js](https://nodejs.org/en/) 18 or higher +- A running Caido instance with an open project +- A [Personal Access Token](https://docs.caido.io/dashboard/concepts/pat.html) (PAT) for your account +- At least one request to your target host in the project's HTTP history. This tutorial uses `caido.local` as the example target, so send any request through your Caido proxy to that host before starting (or substitute your own host throughout). +::: + +## 2. Setting up the script + +### Initializing the project + +Create a working directory for the script and initialize it: + +```bash +mkdir caido-scanner-tutorial +cd caido-scanner-tutorial +pnpm init +``` + +Add `"type": "module"` to `package.json` so Node treats the `.ts` file as an ES module, which the `import` statements below require. + +### Installing dependencies + +Install the Client SDK and the Scanner spec package. The spec package is what makes the Scanner functions and events typed when you call them through the SDK: + +```bash +pnpm add @caido/sdk-client @caido-community/scanner +``` + +::: info +The [`@caido-community/scanner`](https://www.npmjs.com/package/@caido-community/scanner) package is the spec for the Scanner plugin. The Client SDK uses it to type the calls in this tutorial. See [Use a Plugin's NPM Spec Package](/guides/client/spec_typing.md) for the broader concept. +::: + +### Setting environment variables + +Export your PAT and (optionally) the instance URL: + +```bash +export CAIDO_PAT=caido_xxxxx +export CAIDO_INSTANCE_URL=http://localhost:8080 +``` + +::: warning +Never commit the PAT to source control. Treat it like a password and store it in your shell's secret manager or a `.env` file that is gitignored. +::: + +## 3. Connecting and installing Scanner + +Create `index.ts`. The first thing the script does is connect to the Caido instance using the PAT from the environment, then either look up or install the Scanner plugin: + +### index.ts + +```ts +import { Client } from "@caido/sdk-client"; +import type { Spec as ScannerSpec } from "@caido-community/scanner"; + +async function main() { + const client = new Client({ + url: process.env["CAIDO_INSTANCE_URL"] ?? "http://localhost:8080", + auth: { + pat: process.env["CAIDO_PAT"]!, + cache: { file: ".caido-token.json" }, + }, + }); + + await client.connect(); + console.log("Connected to Caido"); + + // Look up the installed plugin, install if missing + let pkg = await client.plugin.pluginPackage("scanner"); + if (pkg === undefined) { + console.log("Installing Scanner..."); + pkg = await client.plugin.install({ manifestId: "scanner" }); + } + console.log("Scanner ready"); +} + +main().catch((error) => { + console.error(error); + process.exit(1); +}); +``` + +Two things to note. The `ScannerSpec` generic on `pluginPackage` and `install` is what makes the rest of the script's calls typed: `pkg.startActiveScan(...)`, `pkg.getChecks()`, and so on, all autocomplete with the correct argument and return types. The lookup-then-install pattern is idiomatic: `pluginPackage()` returns `undefined` when the plugin is not present, and `install()` returns a fresh handle when it is invoked. + +For more on installing plugins from a script, see the [Install a Plugin](/guides/client/install_plugin.md) guide. + +## 4. Discovering available checks + +Before running a scan, you can list the checks Scanner ships with. This is useful for picking which checks to include or exclude from your scan, and for understanding what Scanner can detect: + +```ts +const checks = await pkg.getChecks(); +if (checks.kind === "Error") { + throw new Error(checks.error); +} + +console.log(`Scanner ships with ${checks.value.length} checks`); +for (const check of checks.value.slice(0, 5)) { + console.log(` ${check.id} [${check.type}] - ${check.name}`); +} +``` + +Each check exposes its `id`, `name`, `description`, `type` (`"passive"` or `"active"`), `tags`, and `severities`. You can filter for specific types or IDs by passing options: + +```ts +const activeOnly = await pkg.getChecks({ type: "active" }); +const specific = await pkg.getChecks({ include: ["reflected-xss", "sql-injection"] }); +``` + +::: info +Scanner functions return a `Result` envelope of the form `{ kind: "Ok"; value }` or `{ kind: "Error"; error }`. Always branch on `result.kind` before using the value. This is a Scanner convention; other plugins may use different shapes. +::: + +## 5. Finding a target request + +Active scans run against existing requests in the project's HTTP history. To get the request ID for a target, use [`client.request.list()`](/guides/client/extract_requests.md) with an [HTTPQL](https://docs.caido.io/reference/httpql.html) filter on the host: + +```ts +const page = await client.request + .list() + .filter('req.host.eq:"caido.local"') + .first(1); + +const target = page.edges[0]?.node.request; +if (target === undefined) { + throw new Error( + "No requests to caido.local found in this project. " + + "Send a request through the Caido proxy to that host first.", + ); +} + +console.log(`Target: ${target.method} ${target.host}${target.path} (id=${target.id})`); +``` + +The filter syntax is the same one you use in the Caido HTTP History UI. Adjust it for your own target host. + +## 6. Configuring and starting the scan + +The active scan is configured with a `ScanConfig` object. Each field controls a different aspect of the scan: + +```ts +const start = await pkg.startActiveScan({ + requestIDs: [target.id], + title: `Scan of ${target.host}${target.path}`, + scanConfig: { + aggressivity: "low", + scopeIDs: [], + concurrentChecks: 2, + concurrentRequests: 3, + concurrentTargets: 1, + requestsDelayMs: 0, + scanTimeout: 60000, + checkTimeout: 30000, + severities: ["info", "low", "medium", "high", "critical"], + }, +}); + +if (start.kind === "Error") { + throw new Error(start.error); +} + +const sessionId = start.value.id; +console.log(`Scan started: ${sessionId} (kind=${start.value.kind})`); +``` + +The fields: + +- `aggressivity`: `"low"`, `"medium"`, or `"high"`. Higher aggressivity sends more probe requests per check. +- `scopeIDs`: limit the scan to requests within specific Caido scopes. An empty array means no scope restriction. +- `concurrentChecks`, `concurrentRequests`, `concurrentTargets`: parallelism knobs that trade speed for load on the target. +- `requestsDelayMs`: delay between requests, useful for rate-limited targets. +- `scanTimeout` and `checkTimeout`: timeouts in milliseconds. +- `severities`: which severities to surface in the results. + +`startActiveScan` returns a `Session` in the `Pending` state. The scan starts asynchronously. + +## 7. Tracking progress + +The session transitions through states as the scan runs: `Pending → Running → Done` (or `Interrupted` / `Error`). The simplest way to track it is to poll `getScanSession` until the kind is no longer `Pending` or `Running`: + +```ts +let session = start.value; +while (session.kind === "Pending" || session.kind === "Running") { + await new Promise((resolve) => setTimeout(resolve, 2000)); + const got = await pkg.getScanSession(sessionId); + if (got.kind === "Error") { + throw new Error(got.error); + } + session = got.value; + const done = session.kind === "Running" ? session.progress.checksHistory.length : 0; + const total = session.kind === "Running" ? session.progress.checksTotal : 0; + console.log(` status=${session.kind} progress=${done}/${total}`); +} + +console.log(`Scan finished with status: ${session.kind}`); +``` + +::: tip +For long-running scans, subscribe to the `session:progress` event instead of polling. See [Receive Plugin Events](/guides/client/receive_events.md). Polling is shown here because it keeps the example self-contained. +::: + +## 8. Reading the findings + +When the session reaches `Done`, the `progress.checksHistory` array contains one `CheckExecution` for every check that ran. Each execution carries the findings it produced: + +```ts +if (session.kind !== "Done") { + console.log("Scan did not complete successfully:", session.kind); + return; +} + +const findings = []; +for (const execution of session.progress.checksHistory) { + for (const finding of execution.findings) { + findings.push({ checkID: execution.checkID, finding }); + } +} + +console.log(`\nTotal findings: ${findings.length}`); +for (const { checkID, finding } of findings) { + console.log(` [${finding.severity}] ${finding.name} (${checkID})`); + console.log(` ${finding.description}`); +} +``` + +Each `Finding` has a `name`, `description`, `severity`, and a `correlation` block that pins the finding to a request and optionally a byte range within it. + +## 9. Cleaning up + +Scan sessions persist in the project until they are deleted. Remove the session at the end of the script to keep the project tidy: + +```ts +await pkg.deleteScanSession(sessionId); +console.log("Scan session deleted"); +``` + +## Examples + +The script below combines every step into a single file. It connects, ensures Scanner is installed, finds the first `caido.local` request, runs an active scan, prints the findings, and cleans up. + +### index.ts + +```ts +import { Client } from "@caido/sdk-client"; +import type { Spec as ScannerSpec } from "@caido-community/scanner"; + +async function main() { + const client = new Client({ + url: process.env["CAIDO_INSTANCE_URL"] ?? "http://localhost:8080", + auth: { + pat: process.env["CAIDO_PAT"]!, + cache: { file: ".caido-token.json" }, + }, + }); + + await client.connect(); + + // 1. Look up or install Scanner + let pkg = await client.plugin.pluginPackage("scanner"); + if (pkg === undefined) { + pkg = await client.plugin.install({ manifestId: "scanner" }); + } + + // 2. Find a target request + const page = await client.request + .list() + .filter('req.host.eq:"caido.local"') + .first(1); + const target = page.edges[0]?.node.request; + if (target === undefined) { + throw new Error("Send a request to caido.local through the Caido proxy first"); + } + + // 3. Start the scan + const start = await pkg.startActiveScan({ + requestIDs: [target.id], + title: `Scan of ${target.host}${target.path}`, + scanConfig: { + aggressivity: "low", + scopeIDs: [], + concurrentChecks: 2, + concurrentRequests: 3, + concurrentTargets: 1, + requestsDelayMs: 0, + scanTimeout: 60000, + checkTimeout: 30000, + severities: ["info", "low", "medium", "high", "critical"], + }, + }); + if (start.kind === "Error") { + throw new Error(start.error); + } + const sessionId = start.value.id; + console.log(`Scan started: ${sessionId}`); + + // 4. Wait for the scan to finish + let session = start.value; + while (session.kind === "Pending" || session.kind === "Running") { + await new Promise((resolve) => setTimeout(resolve, 2000)); + const got = await pkg.getScanSession(sessionId); + if (got.kind === "Error") { + throw new Error(got.error); + } + session = got.value; + } + + // 5. Print findings + if (session.kind === "Done") { + const findings = session.progress.checksHistory.flatMap((execution) => + execution.findings.map((finding) => ({ + checkID: execution.checkID, + finding, + })), + ); + console.log(`Scan finished with ${findings.length} finding(s):`); + for (const { checkID, finding } of findings) { + console.log(` [${finding.severity}] ${finding.name} (${checkID})`); + } + } else { + console.log(`Scan ended with status: ${session.kind}`); + } + + // 6. Cleanup + await pkg.deleteScanSession(sessionId); +} + +main().catch((error) => { + console.error(error); + process.exit(1); +}); +``` + +Run it with: + +```bash +export CAIDO_PAT=caido_xxxxx +npx tsx ./index.ts +``` + +A successful run against a `caido.local` target prints something like: + +```txt +[caido] Attempting to load cached token +[caido] Loaded token from cache +Scan started: ascan-xxxxxxxxxxx +Scan finished with 1 finding(s): + [medium] Missing X-Frame-Options Header (anti-clickjacking) +``` + +Exact findings will vary based on what the target's response contains. + +## Script Breakdown + +The script performs the following operations: + +1. **Connect**: authenticates against the Caido instance using a PAT and caches the resulting tokens on disk so subsequent runs skip the auth flow. See [Base Setup](/guides/client/base_setup.md) for details. +2. **Plugin handle**: looks up Scanner by manifest ID, installing it via the SDK if it is not yet present. See [Install a Plugin](/guides/client/install_plugin.md). +3. **Target lookup**: queries the HTTP history with an HTTPQL filter to find a request to scan. See [Extract Requests](/guides/client/extract_requests.md) and the [HTTPQL reference](https://docs.caido.io/reference/httpql.html). +4. **Active scan**: builds a `ScanConfig`, calls `startActiveScan`, and polls the resulting `Session` until it transitions out of `Running`. +5. **Findings**: flattens `progress.checksHistory[*].findings` into a single list and prints each by severity and check ID. +6. **Cleanup**: deletes the session so it does not accumulate in the project. + +## Next Steps + +You can extend this tutorial in several directions: + +- Subscribe to the `session:progress` event instead of polling, as described in [Receive Plugin Events](/guides/client/receive_events.md). +- Scan multiple requests in one session by passing more IDs to `requestIDs`. +- Persist findings to Caido as native [findings](/guides/client/manage_findings.md) tied to the original request.