Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 19 additions & 9 deletions docs/architecture/context-compaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,20 +30,29 @@ Provider usage updates the state after a response; preflight always recomputes
the estimate for the next request.

An unknown model context window produces unknown pressure, not an invented
limit. The provider may still reject the request; normal generation fallback and
error handling then apply.
limit. If the provider reports that the assembled request exceeds its context
window, the Process applies the same history policy regardless of the estimate.
Context overflow does not advance the main generation fallback chain.

## Overflow policy

Each process has an `auto-compact` or `fail` policy, a pressure threshold, and a
`keepLast` value. The default auto-compacts at `0.9` pressure
`keepLast` value. The threshold governs proactive preflight compaction. The
default auto-compacts at `0.9` pressure
while retaining the newest 80 stored messages. The policy is exposed through
`proc.history.policy.get` and `proc.history.policy.set`.

- `auto-compact` generates a summary and compacts the old prefix before the
model call.
- `fail` ends the run with a visible system error and leaves the process
available for explicit compaction or reset.
- `auto-compact` generates a summary and compacts the old prefix during
preflight or after the first provider-confirmed overflow. It rebuilds the
context and retries the same active model configuration once.
- `fail` ends the run with a visible system error during preflight or after a
provider-confirmed overflow, and leaves the process available for explicit
compaction or reset.

One generation cycle installs at most one automatic compaction. A later tool
round may compact again if newly stored results grow the next assembled context.
If the rebuilt request still overflows, or no older prefix can be archived, the
current run stops explicitly rather than looping or switching models.

Explicit compaction remains available as an operation; `manual` is not an
overflow policy.
Expand All @@ -56,8 +65,9 @@ overflow policy.
- `throughMessageId` selects a prefix through a stored message id.

The caller must also provide a summary or set `generateSummary: true`. Explicit
compaction rejects an active process. Automatic compaction runs in the
owning run's preflight and stops if that run is superseded or aborted.
compaction rejects an active process. Automatic compaction runs inside the
owning run's lifecycle, from preflight or provider-overflow recovery, and stops
if that run is superseded or aborted.

A successful compaction:

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ The AI runtime resolves per-user values first, then falls back to system default
|---|---|---|---|
| `config/ai/provider` | `users/{uid}/ai/provider` | `workers-ai` | Provider adapter. |
| `config/ai/model` | `users/{uid}/ai/model` | `@cf/zai-org/glm-5.2` | Provider model identifier. |
| `config/ai/fallback_model_profile` | `users/{uid}/ai/fallback_model_profile` | `workers-ai-kimi-k2-6` | Saved model profile to try if the selected model fails. |
| `config/ai/fallback_model_profile` | `users/{uid}/ai/fallback_model_profile` | `workers-ai-kimi-k2-6` | Saved model profile to try after eligible generation failures. Context overflow is handled by the process history policy instead. |
| `config/ai/api_key` | `users/{uid}/ai/api_key` | empty | Provider credential. Sensitive. |
| `config/ai/reasoning` | `users/{uid}/ai/reasoning` | `medium` | Reasoning mode hint: `off`, `minimal`, `low`, `medium`, `high`, or `xhigh`. Unsupported values are clamped to the nearest model-supported level at generation time. |
| `config/ai/max_tokens` | `users/{uid}/ai/max_tokens` | `8192` | Maximum output tokens. |
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/syscalls.md
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ Runtime behavior:
| `proc.media.write` | Process DO | Streams one request body directly into process-scoped R2 storage. The body descriptor must declare its exact length so R2 receives a fixed-length stream. An internal caller may supply `mediaId` as an idempotency key: an exact repeated descriptor drains the repeated body and returns the original reference, while conflicting metadata is rejected. Returns a stable media reference for `proc.send`, including its read-only `/var/media/{uid}/{pid}/{id}` filesystem path. |
| `proc.media.delete` | Process DO | Idempotently deletes one unreferenced process-scoped media object. Keys outside the target process or already referenced by process history are rejected. Used to roll back uploads that are not admitted by `proc.send`. |
| `proc.history.policy.get` | Process DO | Returns the process context-overflow policy. The default is `auto-compact` at 90% pressure while retaining the newest 80 stored messages. |
| `proc.history.policy.set` | Process DO | Sets the process context-overflow policy. Supported `overflow` values are `auto-compact` and `fail`; the policy is applied during run preflight. |
| `proc.history.policy.set` | Process DO | Sets the process context-overflow policy. Supported `overflow` values are `auto-compact` and `fail`; the policy is applied during run preflight and after a provider-confirmed overflow. Provider overflow does not advance the main generation fallback chain. |
| `proc.history.compact` | Process DO | Archives an old history prefix, inserts a visible system summary marker, and records a `compaction` segment. Requires a supplied or generated summary and exactly one of `keepLast` or `throughMessageId`. |
| `proc.history.segment.read` | Process DO | Reads paged messages from a compacted segment without restoring them into active history. |
| `proc.history.segments` | Process DO | Lists compacted segments, including archive paths and summary marker ids. |
Expand Down
3 changes: 3 additions & 0 deletions docs/reference/websocket-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,9 @@ Current role defaults from `buildSignalList()`:
- `proc.run.started`
- `proc.run.stream`
- `proc.run.retrying`
- Carries `attempt`, `nextAttempt`, `maxAttempts`, and a sanitized `reason`.
A retry after context compaction stays on the active model and has no
`fallback` field; model fallback transitions include their source and target.
- `proc.run.output`
- Carries assembled assistant text/thinking and, when present, process-owned
`media` references registered for the automatic final reply.
Expand Down
Loading
Loading