Skip to content

feat: add content throttle and dashboard APIs#36

Merged
isuttell merged 21 commits into
mainfrom
agents/next-three-status-items
May 24, 2026
Merged

feat: add content throttle and dashboard APIs#36
isuttell merged 21 commits into
mainfrom
agents/next-three-status-items

Conversation

@isuttell

@isuttell isuttell commented May 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add artifact-level read throttling to the content worker with a native Cloudflare Rate Limit binding and rate_limited_artifact contracts.
  • Add WorkOS member dashboard API key create/revoke endpoints with member audit/idempotency actor support and migration 0005.
  • Add cursor pagination for dashboard audit reads and update project status docs.

Changes

  • Content GET/HEAD now verifies token/path, checks denylist, rate-limits by artifact_id, then reads R2; 429 responses include Retry-After: 60.
  • Web key lifecycle adds idempotent create/revoke routes scoped to admin workspace members and one-time secret replay semantics.
  • Web audit reads accept limit and opaque cursor parameters, ordered by occurred_at desc, id desc.
  • Contracts/OpenAPI, DB migration/snapshot, worker typegen, and local Vitest discovery hygiene were updated.

Risk

High: touches auth-scoped web API endpoints, DB actor constraints, OpenAPI contracts, and Cloudflare preview worker bindings. Mitigated with focused unit/integration tests, DB snapshot checks, local smoke, preview migration/deploy, and local CodeRabbit review.

Test plan

  • pnpm --filter @agent-paste/content test
  • pnpm --filter @agent-paste/content typecheck
  • pnpm --filter @agent-paste/api test
  • pnpm --filter @agent-paste/api typecheck
  • pnpm --filter @agent-paste/db test
  • pnpm --filter @agent-paste/db typecheck
  • pnpm --filter @agent-paste/db db:check
  • pnpm --filter @agent-paste/contracts openapi:check
  • pnpm smoke:local
  • pnpm migrate:preview
  • pnpm deploy:preview
  • pnpm verify

Preview hosted smoke was not run locally because AGENT_PASTE_PREVIEW_ADMIN_TOKEN / AGENT_PASTE_ADMIN_TOKEN is not configured in this shell.

CodeRabbit notes

  • Local CodeRabbit pass 1 found a missing rate_limited_artifact entry in the shared OpenAPI rate-limit envelope; fixed and verified.
  • Local CodeRabbit pass 2 found web audit cursor helper consistency and a query simplification; fixed and verified.
  • Local CodeRabbit pass 3 requested a negative DB actor-type constraint test; fixed and verified.
  • No fourth local CodeRabbit pass was started because the requested local review loop was capped at three iterations.

Summary by CodeRabbit

  • New Features

    • Added web API key management endpoints for creating and revoking keys
    • Implemented paginated web audit event listing with cursor-based navigation
    • Added per-artifact read rate limiting with configurable thresholds
  • Updates

    • Extended actor type support to include "member" authentication
    • Simplified API key creation request format
  • Documentation

    • Updated API contracts to document new endpoints and rate limiting behavior

Review Change Stack

@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 00:13 — with GitHub Actions Inactive
@coderabbitai

coderabbitai Bot commented May 24, 2026

Copy link
Copy Markdown

Important

Review skipped

This PR was authored by the user configured for CodeRabbit reviews. CodeRabbit does not review PRs authored by this user. It's recommended to use a dedicated user account to post CodeRabbit review feedback.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 39dfdc14-87e8-42c5-9e7e-bc4f1cc973f0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

This PR adds member-scoped web API key management (POST /v1/web/keys and POST /v1/web/keys/:id/revoke), cursor-based pagination for web audit events (GET /v1/web/audit with limit/cursor), and per-artifact read rate limiting (ARTIFACT_RATE_LIMIT binding). Changes span contracts/OpenAPI, API routing and validation, DB schema/migrations/queries/repositories, tests across API/content/db, and tooling/config (wrangler, vitest).

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant API_Server
  participant DB
  participant RateLimiter
  Client->>API_Server: POST /v1/web/keys (idempotency-key, body)
  API_Server->>DB: createWebApiKey(member, idempotencyKey, name)
  DB->>DB: persist key, emit api_key.created event
  DB-->>API_Server: {api_key_id, public, secret}
  API_Server-->>Client: 201 Created

  Client->>API_Server: GET /v1/web/audit?limit=...&cursor=...
  API_Server->>DB: listWebAuditEvents(member, {limit,cursor})
  DB->>DB: decode cursor, query page, encode next_cursor
  DB-->>API_Server: {items[], page_info}
  API_Server-->>Client: 200 {items,page_info}

  Client->>API_Server: GET /v/{token}/{path}
  API_Server->>RateLimiter: rateLimitArtifactRead(key=artifact)
  alt allowed
    RateLimiter-->>API_Server: allowed
    API_Server-->>Client: 200 Content
  else denied
    RateLimiter-->>API_Server: limited
    API_Server-->>Client: 429 rate_limited_artifact (Retry-After)
  end
Loading

Possibly related PRs

Poem

🐰 I stitched a cursor, hid a key in moss,
I hopped through audits tallying each toss.
I watched a rate-limited petal hold the gate,
and helped the web keep order — neat and straight.
Hooray — a rabbit paged and sealed the vault with zest.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat: add content throttle and dashboard APIs' accurately summarizes the main changes: artifact-level rate limiting for content reads and new web API key/audit dashboard endpoints.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/api/src/index.ts`:
- Around line 426-431: Wrap the call to readJsonObject(...) in a try/catch and
convert JSON parse errors into a 400 invalid_request response instead of letting
them bubble to a 500; specifically, in the handler around readJsonObject and
CreateApiKeyRequest.safeParse, catch any error thrown by readJsonObject and call
errorResponse(context, "invalid_request", 400), otherwise proceed to validate
with CreateApiKeyRequest.safeParse and call runIdempotent(..., () =>
createWebApiKey(...), 201) as before.

In `@packages/db/snapshot/schema.sql`:
- Around line 69-70: The snapshot is missing a composite index that matches the
audit cursor ordering; add a new index on (workspace_id, occurred_at, id) for
the operation_events table so keyset pagination using (workspace_id,
occurred_at, id) is efficient. Create the index with a clear name (e.g.
operation_events_workspace_occurred_id_idx) and ensure it uses the same sort
directions as your queries (default ascending unless you need DESC on
occurred_at), keeping the existing CONSTRAINT
"operation_events_actor_type_check" unchanged. Run the snapshot migration after
adding the index so the schema includes the new composite key for audit reads.

In `@packages/db/src/schema.ts`:
- Around line 170-171: The composite index
operation_events_workspace_occurred_idx currently indexes table.workspaceId and
table.occurredAt but the web audit query paginates and orders by occurred_at
DESC, id DESC; update the index definition
(operation_events_workspace_occurred_idx) to include table.id as the final
column (i.e., index on workspace_id, occurred_at, id) so the DB can satisfy
ordering and cursor scans without an extra sort/scan.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 54adb2ea-0524-4e14-9804-dbea43b6a848

📥 Commits

Reviewing files that changed from the base of the PR and between ba78ff5 and 7acf7e9.

📒 Files selected for processing (33)
  • apps/api/src/index.test.ts
  • apps/api/src/index.ts
  • apps/cli/src/index.ts
  • apps/content/src/index.test.ts
  • apps/content/src/index.ts
  • apps/content/src/worker-configuration.d.ts
  • apps/content/wrangler.jsonc
  • docs/ops/project-status.md
  • docs/ops/web-app-todo.md
  • packages/auth/src/request-id.ts
  • packages/commands/src/index.ts
  • packages/contracts/openapi/api.json
  • packages/contracts/openapi/content.json
  • packages/contracts/openapi/upload.json
  • packages/contracts/package.json
  • packages/contracts/src/apiKeys.ts
  • packages/contracts/src/common.ts
  • packages/contracts/src/enums.ts
  • packages/contracts/src/mvp-contracts.test.ts
  • packages/contracts/src/openapi/api.ts
  • packages/contracts/src/openapi/content.ts
  • packages/contracts/src/openapi/responses.ts
  • packages/contracts/src/routes.ts
  • packages/db/migrations/0005_member_actor_events.sql
  • packages/db/snapshot/schema.sql
  • packages/db/src/index.test.ts
  • packages/db/src/local-repository.ts
  • packages/db/src/postgres/repository.ts
  • packages/db/src/postgres/rls.test.ts
  • packages/db/src/queries/operation-events.ts
  • packages/db/src/schema.ts
  • packages/db/src/types.ts
  • vitest.config.ts
💤 Files with no reviewable changes (1)
  • packages/contracts/src/apiKeys.ts

Comment thread apps/api/src/index.ts Outdated
Comment thread packages/db/snapshot/schema.sql
Comment thread packages/db/src/schema.ts Outdated
@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 00:24 — with GitHub Actions Inactive

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/db/migrations/0006_operation_events_web_audit_cursor_index.sql`:
- Around line 3-4: The migration creates the index
operation_events_workspace_occurred_id_idx with "occurred_at desc, id desc" but
the schema sources define it without DESC; pick one canonical ordering and make
all three definitions identical. Update the migration SQL
(operation_events_workspace_occurred_id_idx), the in-code schema definition in
schema.ts, and the snapshot/schema.sql so the index columns and ordering (either
plain ascending/no DESC or explicit DESC for occurred_at and id) match exactly
to avoid drift.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0bcbb7ba-8694-4fe5-ad31-214d1c1f6121

📥 Commits

Reviewing files that changed from the base of the PR and between 7acf7e9 and 194b895.

📒 Files selected for processing (5)
  • apps/api/src/index.test.ts
  • apps/api/src/index.ts
  • packages/db/migrations/0006_operation_events_web_audit_cursor_index.sql
  • packages/db/snapshot/schema.sql
  • packages/db/src/schema.ts

Comment on lines +3 to +4
create index if not exists operation_events_workspace_occurred_id_idx
on operation_events(workspace_id, occurred_at desc, id desc);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Keep the index definition consistent across migration and schema sources.

packages/db/src/schema.ts and packages/db/snapshot/schema.sql define operation_events_workspace_occurred_id_idx without DESC, while this migration creates (workspace_id, occurred_at desc, id desc). Reusing the same index name for different definitions leaves fresh schemas out of sync with migrated databases and will create drift in future schema checks. Pick one ordering and align all three files.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/db/migrations/0006_operation_events_web_audit_cursor_index.sql`
around lines 3 - 4, The migration creates the index
operation_events_workspace_occurred_id_idx with "occurred_at desc, id desc" but
the schema sources define it without DESC; pick one canonical ordering and make
all three definitions identical. Update the migration SQL
(operation_events_workspace_occurred_id_idx), the in-code schema definition in
schema.ts, and the snapshot/schema.sql so the index columns and ordering (either
plain ascending/no DESC or explicit DESC for occurred_at and id) match exactly
to avoid drift.

@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 00:38 — with GitHub Actions Inactive
@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 00:56 — with GitHub Actions Inactive
@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 01:08 — with GitHub Actions Inactive

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/contracts/openapi/upload.json`:
- Line 649: Update the 429 response descriptions for the upload endpoints to
reference the correct schema: replace mentions of ArtifactRateLimitErrorEnvelope
with RateLimitErrorEnvelope in the response "description" fields (the 429
responses for upload routes), so the documentation and SDKs reflect the actual
response schema; search for occurrences of ArtifactRateLimitErrorEnvelope in
upload-related 429 descriptions and change them to RateLimitErrorEnvelope.

In `@packages/contracts/src/mvp-contracts.test.ts`:
- Around line 47-86: The test only asserts OpenAPI responses for content.get;
add equivalent assertions for content.head: find the routeContracts entry with
id "content.head" and assert it's defined and its .errors includes
"rate_limited_artifact"; read the HEAD responses from
buildContentOpenApiDocument() via
contentOpenApi.paths?.["/v/{token}/{path}"]?.head?.responses? and assert the 429
response exists, has a "Retry-After" header and uses the
ArtifactRateLimitErrorEnvelope $ref, and assert the 404 response uses
ContentNotFoundErrorEnvelope; mirror the same component schema enum checks you
already have for GET so HEAD is validated against the same envelopes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5b7e00c7-6249-45d1-a06e-58558053067a

📥 Commits

Reviewing files that changed from the base of the PR and between 194b895 and 148c592.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (15)
  • apps/api/src/index.test.ts
  • apps/upload/src/index.test.ts
  • docs/ops/project-status.md
  • packages/contracts/openapi/api.json
  • packages/contracts/openapi/content.json
  • packages/contracts/openapi/upload.json
  • packages/contracts/package.json
  • packages/contracts/src/mvp-contracts.test.ts
  • packages/contracts/src/openapi/api.ts
  • packages/contracts/src/openapi/content.ts
  • packages/contracts/src/openapi/responses.ts
  • packages/db/migrations/0006_operation_events_web_audit_cursor_index.sql
  • packages/db/snapshot/schema.sql
  • packages/db/src/index.test.ts
  • packages/db/src/schema.ts

Comment thread packages/contracts/openapi/upload.json Outdated
Comment thread packages/contracts/src/mvp-contracts.test.ts
@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 01:18 — with GitHub Actions Inactive
@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 01:22 — with GitHub Actions Inactive
# Conflicts:
#	apps/api/src/index.ts
#	docs/ops/project-status.md
#	packages/db/src/local-repository.ts
#	packages/db/src/postgres/repository.ts
@isuttell isuttell temporarily deployed to pr-preview-36 May 24, 2026 01:40 — with GitHub Actions Inactive

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/api/src/index.ts (1)

504-507: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Handle malformed JSON as invalid_request in admin API key creation.

readJsonObject(request) can throw here; right now malformed JSON becomes a 500 instead of a 400 client error.

Proposed fix
-  const body = await readJsonObject(request);
+  let body: Record<string, unknown>;
+  try {
+    body = await readJsonObject(request);
+  } catch {
+    return errorResponse(context, "invalid_request", 400);
+  }
   const parsed = CreateApiKeyRequest.safeParse(body);
   if (!parsed.success) {
     return errorResponse(context, "invalid_request", 400);
   }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/api/src/index.ts` around lines 504 - 507, Wrap the call to
readJsonObject(request) in a try/catch in the admin API key creation flow so
malformed JSON is caught and returned as a 400: call readJsonObject inside the
try, assign to body used by CreateApiKeyRequest.safeParse, and in the catch call
errorResponse(context, "invalid_request", 400) (same signature already used) to
ensure malformed JSON doesn't result in a 500; update the logic around
readJsonObject, CreateApiKeyRequest.safeParse, and errorResponse accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@apps/api/src/index.ts`:
- Around line 504-507: Wrap the call to readJsonObject(request) in a try/catch
in the admin API key creation flow so malformed JSON is caught and returned as a
400: call readJsonObject inside the try, assign to body used by
CreateApiKeyRequest.safeParse, and in the catch call errorResponse(context,
"invalid_request", 400) (same signature already used) to ensure malformed JSON
doesn't result in a 500; update the logic around readJsonObject,
CreateApiKeyRequest.safeParse, and errorResponse accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2445611a-124a-4f83-b39c-d4c14e2e0b8d

📥 Commits

Reviewing files that changed from the base of the PR and between fa0725a and ddbde2e.

📒 Files selected for processing (15)
  • apps/api/src/index.ts
  • apps/upload/src/index.test.ts
  • docs/ops/project-status.md
  • packages/contracts/openapi/api.json
  • packages/contracts/openapi/upload.json
  • packages/contracts/src/mvp-contracts.test.ts
  • packages/contracts/src/openapi/responses.ts
  • packages/contracts/src/routes.ts
  • packages/db/src/index.test.ts
  • packages/db/src/repository/core.ts
  • packages/db/src/repository/interface.ts
  • packages/db/src/repository/local-entities.ts
  • packages/db/src/repository/ports.ts
  • packages/db/src/repository/postgres-entities.ts
  • packages/db/src/repository/web-transforms.ts

@isuttell isuttell merged commit 88d9bf3 into main May 24, 2026
4 checks passed
@github-actions

Copy link
Copy Markdown

agent-paste PR preview resources were cleaned up. The pr-preview-${context.issue.number} environment is left in place; remove it from the GitHub UI if desired.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant