A distributed-by-design AI API gateway with a separated control-plane (master) / data-plane (agent) architecture. Provides OpenAI/Claude-compatible /v1/* relay endpoints, built-in management APIs, Web UI, and single-binary distributed deployment.
- Control Plane Management — Users (groups), tokens, channels, models, and agents
- Data Plane Relay — OpenAI/Claude-compatible API endpoints (
/v1/chat/completions,/v1/responses,/v1/messages, etc.) with automatic cross-protocol conversion - Real-Time Config Sync — Master/agent incremental sync over WebSocket; lightweight distributed deployment with zero external dependencies
- Multi-Region Routing — Route requests from region A to agents in region B, enabling cross-region load balancing and bypassing regional restrictions
- Quota & Billing — Usage-based settlement and quota enforcement
- Model Routing — Aggregate multiple upstream models under one name with priority/weight load balancing and error retries
- BYOK (Bring Your Own Key) — End-users can self-serve upload their own provider API keys (AES-GCM encrypted at rest); private channels are merged into the candidate pool with priority over shared admin channels, with optional service-fee billing mode
- Single Binary — Frontend static assets embedded; no separate web server needed
 |
|
 Channels — upstream provider configuration |
 Models — per-model pricing |
 Model Routings — priority/weight aggregation |
 Usage Logs — per-request audit trail |
 Billing — daily rollups by token and channel |
 Playground — in-browser chat tester |
┌─────────────────────────────────────────────────────┐
│ master (control plane) │
│ ┌──────────┐ ┌──────────┐ ┌───────────────────┐ │
│ │ Admin API│ │ Web UI │ │ Agent Sync Hub │ │
│ │ & Auth │ │ (embed) │ │ (WebSocket) │ │
│ └──────────┘ └──────────┘ └───────────────────┘ │
│ ┌──────────────────────────────────────────────┐ │
│ │ Billing & Quota Settlement │ │
│ └──────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
│ WebSocket sync
▼
┌─────────────────────────────────────────────────────┐
│ agent (data plane) │
│ ┌──────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ /v1/* Relay │ │ Token/Chan │ │ Usage │ │
│ │ Endpoints │ │ Cache │ │ Reporter │ │
│ └──────────────┘ └────────────┘ └────────────┘ │
└─────────────────────────────────────────────────────┘
| Topology | Pros | Cons | Use Case |
|---|---|---|---|
| Single node (master + embedded agent) | Simplest setup; one container | Shared resources; single point of failure | PoC, testing, small production |
| Multi-node (master + external agents) | Horizontal scaling; fault isolation; geo-distribution | Higher ops complexity; enrollment lifecycle | Medium/large production, multi-region |
# 1. Prepare config
mkdir -p deploy data
cp config.example.yaml deploy/config.yaml
# Edit deploy/config.yaml — set jwt_secret and admin_password
# 2. Run with Docker Compose
export AI_GATEWAY_IMAGE=vaalacat/ai-gateway:latest
docker compose up -d
# 3. Access
# Web UI: http://localhost:8140
# Health: http://localhost:8140/pingThe configuration file accepts these top-level keys:
log_level— Logging verbosity (debug, info, warn, error)master— Control plane settings (listen address, DB, JWT, admin credentials)agent— Data plane settings (listen address, master URL, enrollment)runtime— Optional advanced tuning (timeouts, heartbeat, retry)
See config.example.yaml for a complete template.
The master stores application and billing-critical state in core.db, request,
trace, and analytics history in log.db, and uses master.db as the legacy
single-database source during an upgrade. These paths are configured with
master.core_db_path, master.log_db_path, and master.legacy_db_path and
must resolve to different SQLite files. The deprecated master.db_path key is
treated as the legacy source so existing deployments can upgrade without a
configuration rewrite; sibling core.db and log.db paths inherit its SQLite
DSN options.
An unavailable log database degrades log-backed pages and metrics without stopping quota settlement or billing facts. New non-critical logs enter a simple bounded delivery queue. When its configured entry or byte limit is reached, the oldest pending log may be dropped; billing data is never put in that queue. The System page reports the database and queue state and provides explicit retry and clear-backlog actions.
There is no automatic log retention. Administrators can manually clear the six derived analytics tables from the System page:
usage_hourly_bucketsusage_duration_histogramsusage_ttft_histogramsusage_tps_histogramsusage_user_ttft_histogramsusage_user_tps_histograms
During an online upgrade, the existing /data/master.db keeps its name and is
opened as the read-only legacy source. The new targets are core.db and
log.db. Background backfill may reach caught_up, but migration becomes
completed only after an administrator runs the final pass. Do not delete the
legacy source before completion.
Older v5 installations may also leave a *.pre-split.bak. This is an
independent legacy artifact: it is not an input to the new backfill and must not
overwrite or be restored as core.db. After validation and migration
completion, an administrator may retain or clean it up separately.
Rolling back to an older image does not automatically merge new state written
to core.db or log.db back into the legacy database. Evaluate that data
window before a rollback.
See the Quick Start section above. Full details in docker-compose.yml.
- Generate an enrollment token from master
- Configure agent with
master_urlandenrollment_token - Start with
docker compose -f docker-compose.yml -f docker-compose.agent.yml up -d
See docker-compose.agent.yml for the overlay template.
See docs/k8s-deployment.md for Kubernetes deployment guidance.
# Prerequisites: Go 1.25+, Node.js 20+, pnpm
# Build (frontend + backend)
CGO_ENABLED=0 bash ./build.sh
# Run the same checks and resource limits as GitHub CI
./scripts/ci.sh
# Frontend dev server (port 8141, proxies to :8140)
cd web && pnpm install && pnpm devReleases are cut by pushing a v* git tag. GitHub Actions builds a multi-arch
image (linux/amd64 + linux/arm64) and pushes it to
Dockerhub.
# Stable release — also updates :latest
git tag v1.2.3
git push origin v1.2.3
# Pre-release — pushes :v1.2.3-rc1 only, does NOT update :latest
git tag v1.2.3-rc1
git push origin v1.2.3-rc1The git tag is injected into the binary as internal/version.Version.
See CONTRIBUTING.md for development setup, code style, and PR process.
This project supports native code (purely self-developed, supporting chat, response, and messages protocols), while other protocols are supported by the new-api channel.
It builds upon the work of the following:
-
new-api by @QuantumNous — the legacy channel adaptor, 50+ upstream provider constants, model-fetch protocols, and token-counting utilities are reused via
github.com/QuantumNous/new-api. Without this prior work, out-of-the-box support for 50+ providers would not be feasible. Sincere thanks to the new-api maintainers and contributors. -
datatype by @franktisellano — variable OpenType font (SIL OFL 1.1) used for inline sparklines in the UI. See
web/public/fonts/OFL.txt.
test/contract/ 包含跨语言一致性测试,默认不跑,需要时手动:
# 1) 启动 master
./ai-gateway --config config.yaml &
# 2) 取 admin token
TOKEN=$(curl -s -X POST http://localhost:8140/api/login \
-H 'Content-Type: application/json' \
-d '{"username":"admin","password":"change-this-password"}' \
| jq -r '.data.token')
# 3) 跑测试
AIGW_ADMIN_TOKEN=$TOKEN go test -tags=contract ./test/contract/测试内容: 扫 web/src/lib/api/*.ts 中所有 /... 路径字面量,逐个发请求,验证后端没有返回 404 (即不存在路由漂移)。