Tools, libraries, papers, and patterns for reducing the cost of running large language models in production.
High-performance LLM query cache with semantic search. Reduce API costs 80% and latency from 8.5s to 1ms using Redis + Qdrant vector DB. Multi-provider support (OpenAI, Anthropic).
Biological nervous systems don't recompute known workflows from scratch. Mnemon gives LLM agents the same primitive — execution memory that caches plans, not responses. 93% token reduction, 2.66ms vs 20s, zero tokens on repeat runs. LangChain, CrewAI, AutoGen.
Read once. Brief anywhere. Reusable document briefings from fingerprinted cache.
Self-hosted SemanticGuard deployment modules — bring the AI gateway with self-validating semantic caching into your own GCP / AWS / Azure account. https://www.semanticguard.dev
Pre-indexed code knowledge graph for Claude Code, Codex, Gemini, Cursor, OpenCode, AntiGravity, Kiro, and Hermes Agent — fewer tokens, fewer tool calls, 100% local
High-performance Reverse Proxy for LLM Caching and Deduplication (OpenAI, Anthropic, Gemini, Copilot). Saves API costs, encrypts cache at rest, and multicasts concurrent streams.
Add a description, image, and links to the llm-caching topic page so that developers can more easily learn about it.
To associate your repository with the llm-caching topic, visit your repo's landing page and select "manage topics."