From 0778854097b876e3eda40897871d0d66394dc620 Mon Sep 17 00:00:00 2001 From: Chris Mucciolo Date: Sun, 26 Jul 2026 15:20:06 -0400 Subject: [PATCH] Document shared kickstart progress and drop host-local runtime. Denoise no longer runs kickstart on the app host; docs now cover the HTTP bootstrap for GHA/Cursor/exe.dev, NDJSON device runners, and todo_loop dispatch. Co-authored-by: Cursor --- astro.config.mjs | 4 ++ src/content/docs/denoise/device-runners.md | 5 ++ .../docs/denoise/github-integration.md | 14 ++-- .../docs/denoise/kickstart-runtimes.md | 49 +++++++++++++ src/content/docs/denoise/milestone-details.md | 21 ++++-- .../docs/denoise/tips-troubleshooting.md | 8 ++- .../docs/dn/completing-github-issues.md | 5 +- src/content/docs/dn/headless-use.md | 34 +++++++-- src/content/docs/dn/progress-reporting.md | 71 ++++++++++++++----- src/content/docs/dn/sandbox.md | 4 ++ src/content/docs/dn/scheduled-workflows.md | 33 ++++++--- 11 files changed, 205 insertions(+), 43 deletions(-) create mode 100644 src/content/docs/denoise/kickstart-runtimes.md diff --git a/astro.config.mjs b/astro.config.mjs index 9559ff9..f81e094 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -107,6 +107,10 @@ export default defineConfig({ label: "Milestone details", slug: "denoise/milestone-details", }, + { + label: "Kickstart runtimes", + slug: "denoise/kickstart-runtimes", + }, { label: "Subscription & Pro", slug: "denoise/subscription-and-pro", diff --git a/src/content/docs/denoise/device-runners.md b/src/content/docs/denoise/device-runners.md index 7402988..31a9fd7 100644 --- a/src/content/docs/denoise/device-runners.md +++ b/src/content/docs/denoise/device-runners.md @@ -48,6 +48,11 @@ one job at a time. An offline device can retain a queued job for up to 24 hours and claim it after reconnecting. Denoise never silently moves a device job to hosted compute. +Device runners report progress with **NDJSON** over the device job API (not the +shared HTTP bootstrap used by GitHub Actions, Cursor Cloud, and exe.dev). See +[Kickstart runtimes](/denoise/kickstart-runtimes/) and +[Progress reporting](/dn/progress-reporting/). + From the device, scripts can also queue work: ```bash diff --git a/src/content/docs/denoise/github-integration.md b/src/content/docs/denoise/github-integration.md index b5c600d..17a7320 100644 --- a/src/content/docs/denoise/github-integration.md +++ b/src/content/docs/denoise/github-integration.md @@ -88,7 +88,9 @@ On a GitHub-linked milestone, Pro users can: - Install or update dn workflow templates and configure the agent harness - Dispatch `dn.init_stack` for milestone stack context - Sort tasks by kickstart plan priority -- Dispatch `dn.kickstart_issue` per task from the task detail dialog +- Dispatch `dn.kickstart_issue` per task from the task detail dialog (pick a + runtime — see [Kickstart runtimes](/denoise/kickstart-runtimes/)) +- Start `dn.todo_loop` when that workflow is installed ![DN setup action row on the milestone view](../../../assets/screenshots/milestone-view-second-btn-row.png) @@ -97,11 +99,15 @@ Denoise dispatches the same workflow events exposed by `dn workflows dispatch`: - `dn.init_stack` — Generate milestone stack markdown and JSON files. - `dn.meld_issue_plan` — Produce a plan for an issue (CLI and Actions; not exposed as a separate milestone-page button today). -- `dn.kickstart_issue` — Run plan plus implementation with AWP from - **Kickstart!** in the task detail dialog. +- `dn.kickstart_issue` — Run plan plus implementation from **Kickstart!** in the + task detail dialog. +- `dn.todo_loop` — Advance the repository todo plan on a stable automation + branch (GitHub Actions). See [Milestone details](/denoise/milestone-details/) for the full UI workflow, -setup states, dispatch feedback, and kickstart blockers. +setup states, dispatch feedback, and kickstart blockers. Live progress uses the +shared HTTP bootstrap described in +[Progress reporting](/dn/progress-reporting/). The legacy `dn.prep_issue_plan` event remains compatible with the installed workflow, but new integrations use `dn.meld_issue_plan`. diff --git a/src/content/docs/denoise/kickstart-runtimes.md b/src/content/docs/denoise/kickstart-runtimes.md new file mode 100644 index 0000000..0e5ae13 --- /dev/null +++ b/src/content/docs/denoise/kickstart-runtimes.md @@ -0,0 +1,49 @@ +--- +title: Kickstart runtimes +description: Where denoise runs kickstart — GitHub Actions, Cursor Cloud, exe.dev, device runners, and CLI Docker. +--- + +When you click **Kickstart!** on a task, denoise asks where the run should +execute. Choose a runtime that matches your isolation and credential needs. +Denoise does **not** run kickstart on the denoise application host. + +## Supported matrix + +| Runtime | Where it runs | Progress | Prerequisites | +| ---------------- | ------------------------------------- | -------------------------------- | -------------------------------------------------- | +| `github_actions` | Target repo GitHub Actions | HTTP if base URL set; else coarse | Agent secrets; `KICKSTART_PROGRESS_BASE_URL` for detailed | +| `cursor_cloud` | Cursor-managed VM | HTTP | `CURSOR_API_KEY` + progress base URL on denoise | +| `cloud_vm` | exe.dev sandbox VM | HTTP | `EXE_TOKEN` + progress base URL (+ managed checkout for launcher) | +| `device_runner` | Paired developer laptop | NDJSON via device job API | Pairing enabled; registered checkout | +| Docker | Your machine via CLI only | N/A on hosted denoise | `dn kickstart --sandbox docker` locally | + +Preflight availability is listed at `GET /api/kickstart/runtimes?owner=&repo=` +and shown in the confirm dialog. Unavailable options stay visible with a short +reason. + +## Progress fidelity + +- **Detailed** — Phase and step events stream into the task progress panel + (Resolve → Implement → Lint → Publish timeline). +- **Coarse** — Queued / running / succeeded / failed only. Common for GitHub + Actions when the denoise deploy has no public `KICKSTART_PROGRESS_BASE_URL`. + +Shared HTTP bootstrap details: +[Progress reporting](/dn/progress-reporting/). + +## Notes + +- Managed Cursor Cloud and exe.dev launches still use a server-side checkout + under `KICKSTART_RUNNER_WORKSPACE_ROOT` only to **start** `dn` with + `--cursor-cloud` or `--sandbox exe.dev`. Agent work runs in the remote + environment, not as untrusted code on the denoise app process. +- Device jobs never fall back silently to hosted compute. See + [Developer device runners](/denoise/device-runners/). +- Historical runs may still show a legacy `local` source label in progress + history; new dispatches reject that source. + +## Related + +- [Milestone details — Kickstart a task](/denoise/milestone-details/#kickstart-a-task) +- [Sandbox execution](/dn/sandbox/) +- [Headless Use](/dn/headless-use/) diff --git a/src/content/docs/denoise/milestone-details.md b/src/content/docs/denoise/milestone-details.md index 17f42e1..25c4ca2 100644 --- a/src/content/docs/denoise/milestone-details.md +++ b/src/content/docs/denoise/milestone-details.md @@ -188,14 +188,23 @@ requirement. ## Kickstart a task Per-task kickstart plans and implements the issue. Choose an available runtime -in the confirmation dialog: GitHub Actions, Cursor Cloud, a managed cloud VM, -local managed execution, or a paired developer device. Availability depends on -repository setup and the account. A device job never falls back silently to -hosted compute. +in the confirmation dialog: **GitHub Actions**, **Cursor Cloud**, **exe.dev** +(`cloud_vm`), or a **paired device runner**. Availability depends on repository +setup and the account. A device job never falls back silently to hosted compute. +Docker sandbox is CLI-only on hosted denoise. See +[Kickstart runtimes](/denoise/kickstart-runtimes/). + +The progress panel shows **detailed** phase/step events when the chosen runtime +streams HTTP progress (or NDJSON on a device). Without a public progress base +URL, GitHub Actions stays available with **coarse** queued/running/terminal +status only. See [Progress reporting](/dn/progress-reporting/). + +From the milestone view you can also start a **Todo loop** (`dn.todo_loop`) when +the repository has that workflow installed; it uses GitHub Actions today. 1. Open a task in a GitHub-linked milestone (click the task row). 2. In the task detail dialog, click **Kickstart!** -3. Confirm **Run kickstart for this task?** in the dialog. +3. Confirm **Run kickstart for this task?** and pick a runtime in the dialog. 4. Follow queued, running, failed, and completed states in the dialog or task status chips. GitHub Actions runs also provide **Watch on GitHub**. Completed published runs show a PR link when one was reported. @@ -221,6 +230,8 @@ For CLI-oriented planning and implementation depth, see ## Next steps +- [Kickstart runtimes](/denoise/kickstart-runtimes/) — Where kickstart runs and + how progress fidelity works - [GitHub integration](/denoise/github-integration/) — Link milestones, sync issues, convert tasks to GitHub issues - [Tips & troubleshooting](/denoise/tips-troubleshooting/) — When DN setup or diff --git a/src/content/docs/denoise/tips-troubleshooting.md b/src/content/docs/denoise/tips-troubleshooting.md index 0a48fce..21b7694 100644 --- a/src/content/docs/denoise/tips-troubleshooting.md +++ b/src/content/docs/denoise/tips-troubleshooting.md @@ -26,7 +26,13 @@ issues, and collaborator visibility predictable. 6. **Initialize the repo before kickstart** — Install workflows, configure secrets, and run `dn.init_stack` from the milestone view before using **Kickstart!** on tasks. See - [Milestone details](/denoise/milestone-details/). + [Milestone details](/denoise/milestone-details/). Choose where the run + executes in + [Kickstart runtimes](/denoise/kickstart-runtimes/). +7. **Expect coarse progress without a public base URL** — If live phase/step + updates are missing for GitHub Actions, the denoise deploy may lack + `KICKSTART_PROGRESS_BASE_URL`. See + [Progress reporting](/dn/progress-reporting/). ### General diff --git a/src/content/docs/dn/completing-github-issues.md b/src/content/docs/dn/completing-github-issues.md index bdf2f65..a6d0ff3 100644 --- a/src/content/docs/dn/completing-github-issues.md +++ b/src/content/docs/dn/completing-github-issues.md @@ -217,7 +217,10 @@ dn kickstart --cursor-cloud --publish pr --ref main 123 Cursor Cloud uses a remote clone on a Cursor-managed VM and does not modify the local workspace. Without progress configuration, the run is durable fire-and-forget. With `DN_DISPATCH_ID` and `DN_PROGRESS` set, `dn` waits, -reports progress, and returns the pull request URL or terminal failure. See +reports progress, and returns the pull request URL or terminal failure. In +denoise, that bootstrap is issued per invocation — see +[Progress reporting](/dn/progress-reporting/) and +[Kickstart runtimes](/denoise/kickstart-runtimes/). Also see [Headless Use](/dn/headless-use/) for CI dispatch and [Sandbox execution](/dn/sandbox/) for Docker and exe.dev. diff --git a/src/content/docs/dn/headless-use.md b/src/content/docs/dn/headless-use.md index c68c8a0..478868b 100644 --- a/src/content/docs/dn/headless-use.md +++ b/src/content/docs/dn/headless-use.md @@ -38,6 +38,7 @@ This writes: | `.github/workflows/dn-prep-issue-plan.yml` | Meld plan phase; filename retained temporarily for compatibility | | `.github/workflows/dn-kickstart-issue.yml` | Full kickstart (plan + implement) | | `.github/workflows/dn-daily-kickstart.yml` | Scheduled milestone queue runner — see [Scheduled Workflows](/dn/scheduled-workflows/) | +| `.github/workflows/dn-todo-loop.yml` | Todo plan loop (`schedule`, `workflow_dispatch`, `repository_dispatch`) | Set the agent once in `.github/dn/config.json`: @@ -195,7 +196,7 @@ modes, Docker image notes, and exe.dev troubleshooting. | `dn.meld_issue_plan` | `dn-prep-issue-plan.yml` | `repository_dispatch` → `dn.meld_issue_plan` | `dn meld` | | `dn.kickstart_issue` | `dn-kickstart-issue.yml` | `repository_dispatch` → `dn.kickstart_issue` | `dn kickstart` | | `dn.daily_kickstart` | `dn-daily-kickstart.yml` | `schedule`, `workflow_dispatch` | `dn kickstart --publish pr --milestone --once` | -| `dn.todo_loop` | `dn-todo-loop.yml` | `schedule`, `workflow_dispatch` | Run the repository todo loop | +| `dn.todo_loop` | `dn-todo-loop.yml` | `schedule`, `workflow_dispatch`, `repository_dispatch` → `dn.todo_loop` | `dn loop` on the repo todo plan | See [Scheduled Workflows](/dn/scheduled-workflows/) for setup, the `DN_DAILY_KICKSTART_MILESTONE` variable, and manual runs. @@ -259,6 +260,26 @@ echo '{"schema_version":"1.0","dispatch_id":"'"$(uuidgen)"'","issue_url":"https: `gh run list --repo owner/repo --event repository_dispatch` or use `dn workflows dispatch --wait`. +### `dn.todo_loop` + +Required: `schema_version`, `dispatch_id`. + +Optional: nested `progress` (`mode`, `url`, `token`) for live phase/step ingest +when denoise (or another orchestrator) supplies a per-invocation bootstrap; +`validate_only`. + +```bash +echo '{"schema_version":"1.0","dispatch_id":"'"$(uuidgen)"'}' \ + | dn workflows dispatch dn.todo_loop --repo owner/repo --json --wait +``` + +The workflow runs the repository todo loop on a stable automation branch and +opens or advances one recurring pull request for that plan. Requires +`plans/todo.plan.md` (or the path configured for the repo). Denoise can start +todo loop from the web with the same `repository_dispatch` event. Progress +bootstrap details: +[Progress reporting](/dn/progress-reporting/). + ## Dispatch and manage workflows from the CLI ```bash @@ -301,6 +322,7 @@ dn workflows install --agent cursor | `dn.meld_issue_plan` | `contents: write`, `issues: write` | | `dn.kickstart_issue` | `contents: write`, `pull-requests: write`, `issues: write` | | `dn.daily_kickstart` | `contents: write`, `pull-requests: write`, `issues: write` | +| `dn.todo_loop` | `contents: write`, `pull-requests: write`, `issues: write` | For kickstart with `publish: pr`, also enable **Allow GitHub Actions to create and approve pull requests** under **Settings → Actions → General → Workflow @@ -309,14 +331,16 @@ permissions**. ## denoise and other integrators denoise dispatches the same payload shapes through its backend GitHub App. The -milestone dashboard can trigger `dn.init_stack`, `dn.meld_issue_plan`, and -`dn.kickstart_issue` on linked repositories that have installed templates. +milestone dashboard can trigger `dn.init_stack`, `dn.meld_issue_plan`, +`dn.kickstart_issue`, and `dn.todo_loop` on linked repositories that have +installed templates. Choose a kickstart runtime in the task dialog — see +[Kickstart runtimes](/denoise/kickstart-runtimes/). Every dispatch ID is copied into `DN_DISPATCH_ID` and the correlated workflow run name. Use `--wait` or match that run name exactly; do not associate overlapping runs by creation time. See -[Progress reporting](/dn/progress-reporting/) for NDJSON, HTTP delivery, -redaction, and PR URL fields. +[Progress reporting](/dn/progress-reporting/) for the shared HTTP bootstrap, +NDJSON on device runners, redaction, and PR URL fields. Compatibility paths (still supported, separate from dispatch): diff --git a/src/content/docs/dn/progress-reporting.md b/src/content/docs/dn/progress-reporting.md index 566d325..00bf4fb 100644 --- a/src/content/docs/dn/progress-reporting.md +++ b/src/content/docs/dn/progress-reporting.md @@ -1,22 +1,49 @@ --- title: Progress reporting -description: Correlate workflow dispatches and consume versioned dn progress events safely. +description: Shared HTTP progress bootstrap for denoise kickstart runners, plus NDJSON for device runners. --- -`dn` can report structured kickstart progress to an orchestrator. Callers must -generate a unique dispatch ID before dispatch so events and the resulting GitHub -Actions run can be correlated exactly. +`dn` reports structured kickstart progress to denoise (and other +orchestrators) when correlation and a delivery mode are configured. Denoise +issues a **per-invocation** progress token and delivers the same HTTP bootstrap +to GitHub Actions, Cursor Cloud, and exe.dev (`cloud_vm`). Device runners use +NDJSON over the device job API instead. -## Configure reporting +Do **not** mint a shared `DN_PROGRESS_TOKEN` repository secret for every target +repo. Tokens are short-lived and scoped to one invocation. -| Variable | Purpose | -| ----------------------- | --------------------------------------- | -| `DN_DISPATCH_ID` | Invocation correlation ID | -| `DN_PROGRESS=ndjson` | Write one JSON event per line to stderr | -| `DN_PROGRESS=http` | POST events to the configured URL | -| `DN_PROGRESS_URL` | HTTP ingest endpoint | -| `DN_PROGRESS_TOKEN` | Bearer token for HTTP delivery | -| `DN_PROGRESS_VERBOSE=1` | Include redacted `agent.line` events | +## Shared HTTP bootstrap + +When denoise has `KICKSTART_PROGRESS_BASE_URL` configured, primary web runners +receive: + +| Field / env | Meaning | +| ----------------------- | ---------------------------------------------------- | +| `DN_DISPATCH_ID` | Invocation correlation ID | +| `DN_PROGRESS=http` | POST events to the ingest URL | +| `DN_PROGRESS_URL` | Denoise `/api/kickstart/invocations//events` URL | +| `DN_PROGRESS_TOKEN` | Bearer token for that invocation only | +| `DN_PROGRESS_VERBOSE=1` | Include redacted `agent.line` events (optional) | + +**GitHub Actions** receive the same values nested under +`client_payload.progress` (`mode`, `url`, `token`). `dn workflows exec` exports +them into the job environment so kickstart reports phases and steps without a +repo-wide progress secret. + +**Cursor Cloud** and **exe.dev** managed runners get the same variables in the +child `dn` environment when denoise starts the managed launch. + +Without `KICKSTART_PROGRESS_BASE_URL`, GitHub Actions remains available but the +denoise panel uses **coarse** status (queued / running / terminal) instead of +the phase timeline. Cursor Cloud and exe.dev require the public base URL for +detailed progress and are unavailable in the picker until it is set. + +## Delivery modes + +| Variable | Purpose | +| -------------------- | ---------------------------------------------------- | +| `DN_PROGRESS=http` | POST events to `DN_PROGRESS_URL` with the bearer token | +| `DN_PROGRESS=ndjson` | Write one JSON event per line to stderr (device runners) | Events use schema version `1.0` and include `invocation_id`, increasing `seq`, ISO-8601 `ts`, `type`, and `message`. Phase events can identify `plan`, @@ -52,10 +79,15 @@ creation-time window. Overlapping runs make time-based matching unsafe. ## Render progress in denoise -Render queued and running events as active states, `invocation.failed` as a -failure with its safe message, and `invocation.succeeded` as complete. Prefer -`publish.completed.data.pr_url` for the PR action. A successful run without that -field may have used local or direct publication; do not invent a PR link. +With **detailed** fidelity, render queued and running events as active states, +`invocation.failed` as a failure with its safe message, and +`invocation.succeeded` as complete. Prefer `publish.completed.data.pr_url` for +the PR action. A successful run without that field may have used direct +publication; do not invent a PR link. + +With **coarse** fidelity (typically GitHub Actions without a public progress +base URL), show only queued / running / terminal status — not a fake phase +timeline. If events are missing, confirm both `DN_DISPATCH_ID` and a reporting mode are set. For HTTP, check the URL and token without logging the token. Reject or @@ -65,3 +97,8 @@ by `invocation_id`. Cursor Cloud normally dispatches and exits. When both correlation and progress reporting are configured, `dn` waits for completion, reports the PR URL when available, and emits failure or timeout as terminal events. + +## Where each runtime reports + +See [Kickstart runtimes](/denoise/kickstart-runtimes/) for the supported matrix. +Denoise does **not** run kickstart on the application host. diff --git a/src/content/docs/dn/sandbox.md b/src/content/docs/dn/sandbox.md index 4ba762e..8e1d555 100644 --- a/src/content/docs/dn/sandbox.md +++ b/src/content/docs/dn/sandbox.md @@ -8,6 +8,10 @@ the default; Docker provides a reproducible local container and exe.dev provides a remote VM. Cursor Cloud and GitHub Actions are separate execution runtimes, not `dn` sandbox providers. +On **hosted denoise**, kickstart runtimes are GitHub Actions, Cursor Cloud, +exe.dev, and paired device runners — not Docker and not the denoise application +host. See [Kickstart runtimes](/denoise/kickstart-runtimes/). + ## Choose a runtime | Runtime | Best for | Workspace behavior | diff --git a/src/content/docs/dn/scheduled-workflows.md b/src/content/docs/dn/scheduled-workflows.md index ad2386e..d080139 100644 --- a/src/content/docs/dn/scheduled-workflows.md +++ b/src/content/docs/dn/scheduled-workflows.md @@ -1,21 +1,23 @@ --- title: Scheduled Workflows -description: Run dn kickstart daily against a milestone queue with dn-daily-kickstart.yml. +description: Run dn kickstart daily against a milestone queue, and schedule or dispatch the todo loop. --- -`dn init workflows` installs four canonical workflow files. The fourth, -**`dn-daily-kickstart.yml`**, runs kickstart on a schedule against a committed -milestone stack — one unchecked queue item per run, opening a PR for each. +`dn init workflows` installs the canonical workflow files, including +**`dn-daily-kickstart.yml`** and **`dn-todo-loop.yml`**. Daily kickstart runs +against a committed milestone stack — one unchecked queue item per run, opening +a PR for each. The todo loop can run on a schedule or via +`repository_dispatch` from denoise. -Use this when you want steady, automated progress through a prioritized backlog -without dispatching `dn.kickstart_issue` by hand every day. +Use daily kickstart when you want steady, automated progress through a +prioritized backlog without dispatching `dn.kickstart_issue` by hand every day. Complete [Headless Use — Configure a repository](/dn/headless-use/#configure-a-repository) -first. Scheduled kickstart reuses the same `.github/dn/config.json`, agent +first. Scheduled workflows reuse the same `.github/dn/config.json`, agent secret, and OpenCode config as the other canonical workflows. -## What each run does +## What each daily kickstart run does On schedule (or manual trigger), `chesapeakedev/dn-action` runs: @@ -67,7 +69,7 @@ Set this in **Settings → Secrets and variables → Actions → Variables**, or ## Schedule and triggers -The installed template defines: +The installed daily kickstart template defines: | Trigger | Behavior | | ------------------- | --------------------------------------- | @@ -97,7 +99,7 @@ gh workflow run dn-daily-kickstart.yml --repo owner/repo -f validate_only=true ## Setup checklist ```bash -# 1. Install all four workflows (if not already done) +# 1. Install canonical workflows (if not already done) dn init workflows --agent opencode gh secret set OPENAI_API_KEY @@ -139,6 +141,17 @@ denoise and other tools dispatch `dn.kickstart_issue` for per-task automation. Scheduled kickstart is for repositories that maintain a committed milestone queue and want one PR per day without external dispatch. +## Todo loop (`dn.todo_loop`) + +`dn-todo-loop.yml` supports `schedule`, `workflow_dispatch`, and +`repository_dispatch` (`dn.todo_loop`). Denoise can start a todo loop from the +web using the same event shape as other tracked dispatches, including optional +nested `progress` for live phase/step reporting when the progress base URL is +configured. + +See [Headless Use — `dn.todo_loop`](/dn/headless-use/#dntodo_loop) and +[Progress reporting](/dn/progress-reporting/). + ## Troubleshooting | Symptom | Check |