Skip to content

feat: add provider-aware prompt caching - #28

Merged
devproje merged 1 commit into
masterfrom
feat/provider-prompt-caching
Aug 18, 2026
Merged

feat: add provider-aware prompt caching#28
devproje merged 1 commit into
masterfrom
feat/provider-prompt-caching

Conversation

@devproje

Copy link
Copy Markdown
Owner

What this changes

Provider configuration previously assumed an OpenAI-compatible API and only recorded cache reads exposed through cached_tokens. This meant native Anthropic endpoints could not be used, Claude prompt caching could not be explicitly configured, and cache writes were invisible.

This change:

  • adds openai and anthropic provider kinds
  • adds auto, off, ephemeral, and ephemeral_1h prompt-cache policies
  • supports the native Anthropic Messages API, including streaming, reasoning, images, tool calls, and cache usage
  • enables automatic prompt caching for native Anthropic and Claude models routed through OpenRouter
  • adds opt-in OpenRouter whole-response caching with a configurable TTL
  • records cache reads and writes separately
  • adds the cache_write_tokens database migration
  • exposes cache-read and cache-write totals through CLI and TUI usage output
  • preserves compatibility with existing provider configurations
  • documents provider and cache configuration, including response-cache retention caveats

How it was verified

  • make test-race passes
  • New behaviour has a test, or there is nothing to test

Added coverage for:

  • OpenAI-compatible top-level cache control
  • automatic OpenRouter Claude prompt caching
  • native Anthropic streaming and cache usage normalization
  • rejection of response caching on non-OpenRouter endpoints
  • cache-write token persistence and aggregation
  • migration of existing databases

Checklist

  • Follows docs/CONVENTION.md: no comments, no :=, one var block per function in first-use order with err last, callees before callers, and main unconditionally last
  • New .go files carry the two-line SPDX header
  • Documentation updated if behaviour a user can see has changed
  • My contribution is licensed GPL-3.0-or-later, matching the project

@devproje
devproje merged commit 1481ab0 into master Aug 18, 2026
2 checks passed
@devproje
devproje deleted the feat/provider-prompt-caching branch August 18, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant