parallel-tools.md

Parallel Tool Execution & Compaction

Parallel tool execution

When the LLM returns multiple tool calls in a single response, qcr dispatches independent calls concurrently. This happens automatically with no configuration required.

How it works

1. The LLM response contains multiple tool_use blocks.
2. Each tool is classified into a concurrency group.
3. Tools in different groups run in parallel via tokio::spawn + semaphore.
4. Results are collected and returned to the LLM in the original order.

Concurrency groups

Every tool declares a concurrency group that controls how it interleaves with other tools:

Group	Behavior	Examples
Independent	Runs in parallel with everything	ReadFile, Glob, Grep, Ls, WebFetch, ToolSearch
Named("fs")	Sequential with other fs tools, parallel with non-fs	WriteFile, EditFile, FormatFile, FindAndEdit
Exclusive	Runs alone, no other tools execute concurrently	Shell, Browser

Tools in the same named group execute sequentially. Tools in different groups run in parallel.

For example, if the LLM requests ReadFile, Grep, and Shell in one turn:

• ReadFile and Grep start immediately (both Independent)
• Shell waits until both finish (Exclusive)

Max parallel tools

The concurrency limit defaults to 10. Configure it in settings.json:

{
  "max_parallel_tools": 10
}

This caps the number of simultaneously executing tool calls. Independent tools beyond this limit queue until a slot opens.

---

Multi-level context compaction

Qwen Code Rust uses a 3-tier compaction system to keep conversations within the LLM's context window. Each level is progressively more aggressive.

Level 1: Micro-compact (every turn, free)

Runs after every agent turn with no LLM call. Performs lightweight pruning:

• Truncate verbose tool results — tool outputs exceeding max_tool_result_lines (default: 100) are cut to that limit
• Deduplicate reads — when the same file is read multiple times, only the most recent read is kept
• Collapse error cycles — repeated error-fix-error patterns are collapsed into a single summary

Configure in settings.json:

{
  "compaction": {
    "micro": {
      "enabled": true,
      "max_tool_result_lines": 100,
      "deduplicate_reads": true,
      "collapse_error_cycles": true
    }
  }
}

Level 2: Auto-compact (at threshold, LLM call)

Triggers when the conversation approaches the model's context limit. Uses an LLM call to summarize older messages while preserving recent context.

• Threshold: fires when token usage reaches context_window - reserve_tokens
• Reserve tokens: default 33,000 (space for response + tool schemas)
• Keeps recent: the last N messages stay uncompressed (default: 10)

Everything older than the recent window is summarized into a compact context block.

{
  "compaction": {
    "auto": {
      "enabled": true,
      "reserve_tokens": 33000,
      "keep_recent": 10
    }
  }
}

The TokenBudget tracker uses provider-reported token counts when available, with a chars/4 fallback estimate. It knows the context window size for common models (Claude, GPT, Gemini, Qwen, DeepSeek).

Level 3: Reactive-compact (on error, emergency)

Triggers when the API returns a prompt_too_long error. This is the last resort:

• More aggressive than auto-compact (keeps fewer recent messages, default: 5)
• Retries the failed request after compaction (default: 1 retry)

{
  "compaction": {
    "reactive": {
      "enabled": true,
      "keep_recent": 5,
      "max_retries": 1
    }
  }
}

Manual compaction

Use the /compact slash command to trigger compaction manually:

/compact

This runs auto-compact immediately regardless of the current token usage. Useful when you know the conversation has accumulated a lot of verbose tool output and you want to reclaim context space.

UI notification

When compaction runs, qcr emits a ContextCompacted event that the TUI displays:

[compacted] auto: removed 42 messages, saved ~18.5K tokens

Full compaction config example

{
  "compaction": {
    "micro": {
      "enabled": true,
      "max_tool_result_lines": 150,
      "deduplicate_reads": true,
      "collapse_error_cycles": true
    },
    "auto": {
      "enabled": true,
      "reserve_tokens": 40000,
      "keep_recent": 15
    },
    "reactive": {
      "enabled": true,
      "keep_recent": 3,
      "max_retries": 2
    }
  }
}

Disabling compaction

To disable a specific level, set enabled: false:

{
  "compaction": {
    "micro": { "enabled": false },
    "auto": { "enabled": true },
    "reactive": { "enabled": true }
  }
}

Disabling all three levels is not recommended -- the conversation will eventually exceed the context window and API calls will fail.