Skip to content

Add Workers AI gateway for direct API integration - #1

Merged
meirdick merged 1 commit into
0.xfrom
feature/workersai-gateway
Apr 14, 2026
Merged

Add Workers AI gateway for direct API integration#1
meirdick merged 1 commit into
0.xfrom
feature/workersai-gateway

Conversation

@meirdick

Copy link
Copy Markdown
Owner

Summary

Adds a native WorkersAiGateway that calls the Cloudflare Workers AI API directly via the Chat Completions format, following the same trait-based composition pattern as the Groq and Mistral gateway migrations.

  • Provider: WorkersAiProvider with text generation + embeddings
  • Gateway: WorkersAiGateway implements EmbeddingGateway, TextGateway
  • API format: Chat Completions (like Groq, Mistral) — not Responses API
  • URL construction: Auto-builds from account_id (direct) or gateway config (AI Gateway)
  • Reasoning models: ReasoningStart/Delta/End events for Kimi K2.5, Gemma 4, QwQ
  • Session affinity: x-session-affinity header for prefix caching
  • Content coercion: String content for Workers AI compatibility
  • Validation: Helpful error when /compat URL used without workers-ai/ model prefix

User setup

# Direct (minimal)
CLOUDFLARE_AI_API_KEY=your-token
CLOUDFLARE_ACCOUNT_ID=abc123

# With AI Gateway (add one line)
WORKERSAI_GATEWAY=my-gateway-name

Test plan

  • 39 Workers AI provider tests (BaseUrl, ErrorHandling, MessageMapping, RequestMapping, Streaming, ToolCallLoop, ToolMapping, ProviderOptions, AgentFake)
  • 527 total tests pass (zero regressions)
  • Verified against real Cloudflare API (direct, AI Gateway, /compat paths)
  • Browser tested on 4 projects: career-forge, nex, endo-baddie, paws
  • Streaming with reasoning events confirmed working (Kimi K2.5)
  • Embeddings confirmed working
  • Tool calling with multi-step loops confirmed working

🤖 Generated with Claude Code

Adds a native WorkersAiGateway that calls the Cloudflare Workers AI API
directly via the Chat Completions format, removing the Prism dependency
for Workers AI text generation, streaming, and embeddings.

Key features:
- Direct and AI Gateway URL construction from account_id/gateway config
- Reasoning model support (ReasoningStart/Delta/End events) for Kimi K2.5,
  Gemma 4, and QwQ models
- Session affinity header for prefix caching
- Content string coercion for Workers AI compatibility
- /compat endpoint model name validation with helpful error messages
- Embeddings via /embeddings endpoint

Follows the same trait-based composition pattern as the Groq and Mistral
gateway migrations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@meirdick
meirdick force-pushed the feature/workersai-gateway branch from efe4147 to 743c18c Compare April 14, 2026 20:08
@meirdick
meirdick merged commit 577d2a8 into 0.x Apr 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant