AICore

A production-grade, multi-provider LLM gateway — edge proxy, unified SDK, cost observability, and a delivery operating system to match.

---

AICore sits between an application (or an AI agent) and any LLM provider — OpenAI, Anthropic, Gemini, Groq — and returns one normalized response with per-call cost, token, and latency metadata, regardless of which provider served it. One import, full visibility, from the first call.

This repository is also a deliberate demonstration of how I build production systems: a modular monorepo with explicit module contracts, architecture decision records, a testing pyramid wired into a blocking CI gate, and an honest production-readiness assessment. The engineering rigor is the point as much as the product.

Why it exists: every team shipping AI features pays four invisible taxes — SDK fragmentation across providers, zero visibility into AI cost, manual integration-doc hunting, and no shared knowledge of what works. AICore attacks the first two directly and lays the schema for the rest.

Architecture

flowchart TD
    app["App / AI agent"] -->|"@aicore/sdk<br/>chat · stream · complete"| worker

    subgraph worker["packages/worker — Cloudflare Worker (edge proxy)"]
        chain["middleware chain:<br/>logging → auth → validation →<br/>circuit-breaker → token-audit → executeProxy"]
        chain --> route["pickProvider() → registry.getAdapter()"]
    end

    route -->|"one ProviderAdapter contract"| providers["OpenAI · Anthropic · Gemini · Groq"]
    worker -->|"ctx.waitUntil(emitTelemetry)"| sink[("Supabase Postgres<br/>usagelogs — partitioned + RLS")]
    sink --> dash["apps/dashboard — Next.js<br/>cost · usage · logs · keys"]
    sink --> cli["packages/cli — npx aicore<br/>init · logs · keys"]

    worker -.->|"meta: model, provider,<br/>tokens, cost_usd, latency_ms"| app

Loading

Every unary response carries a meta envelope built once from a uniform ProviderCallUsage, so cost/token/latency are reported identically across all four providers. The same wide telemetry row lands in a monthly-partitioned, RLS-isolated usagelogs table that powers the dashboard and CLI.

Module	Path	Responsibility
types	packages/types	The contract spine — request/response, provider, error, telemetry, DB types. Every package depends on it.
worker	packages/worker	Edge proxy: composed middleware chain, four provider adapters, routing, telemetry.
sdk	packages/sdk	chat / stream / complete, stream normalization, shadow mode.
cli	packages/cli	npx aicore — stack detection, .aicore/context/ generation, logs, keys.
logger / telemetry-gateway	packages/*	Local JSONL + Redis-queued usage logging; Fastify ingest.
dashboard	apps/dashboard	Next.js 15 app — cost, usage, logs, key management (better-auth).

Full system map and module ownership: docs/architecture/architecture-overview.md.

Engineering highlights

What I'd point a reviewer at:

• One provider contract, four implementations. Every adapter returns the discriminated union { ok: true; data; usage } | { ok: false; error }, so the response envelope and telemetry row are built once for all providers (ADR 0003).
• Composed middleware on the edge. logging → auth → validation → circuit-breaker → token-audit → executeProxy, with all logging/telemetry off the hot path via ctx.waitUntil.
• Reliability primitives. Exponential-backoff retries with jitter (all four adapters) and a per-provider circuit breaker; provider errors normalized to a single AICoreError shape with correct HTTP status propagation.
• Telemetry as a first-class contract. A monthly-partitioned, RLS-isolated usagelogs table whose full Phase 1–5 schema is reserved from day one (ADR 0006), so later features turn on without migrations.
• Decisions are written down. Eight ADRs capture the architecturally significant choices and their rejected alternatives.
• A real quality gate. A testing pyramid wired into a blocking CI workflow: repo-wide type-check + every hermetic suite (types · sdk · worker · cli) must pass to merge.
• Honest self-assessment. A production-readiness assessment and a known-defects register that say what's solid and what isn't — including a market analysis concluding where this is and isn't a viable business.

Tech stack

Layer	Technology
Language	TypeScript (strict, no any), Node ≥ 22
Monorepo	pnpm workspaces + Turborepo
Edge proxy	Cloudflare Workers
Database	Supabase Postgres (partitioned, RLS)
Dashboard	Next.js 15, better-auth
Queue / ingest	Redis, Fastify
Testing / CI	Jest, GitHub Actions

Quickstart

pnpm install
pnpm type-check          # strict, all packages
pnpm test                # hermetic suites (types · sdk · worker · cli)

# Generate project context in any repo:
npx aicore init          # detects stack, writes .aicore/context/

# Run a package locally:
pnpm dev:dashboard       # Next.js dashboard
pnpm --filter @aicore/worker dev   # edge proxy (wrangler)

Using the SDK:

import { AICore } from "aicore-sdk";

const ai = new AICore({ endpoint: process.env.AICORE_URL!, apiKey: process.env.AICORE_KEY! });

const res = await ai.chat(
  [{ role: "user", content: "Explain edge proxies in one sentence." }],
  { feature: "docs", taskType: "generation", model: "gpt-4o-mini" }
);

console.log(res.content);
console.log(res.usage); // { inputTokens, outputTokens, costCents, latencyMs }

Status

Pre-alpha — built as a portfolio-grade reference, not a launched product.

• ✅ Working & tested: unified SDK, edge proxy + middleware chain, four provider adapters, response meta envelope, token-audit + spend-ceiling enforcement, telemetry → usagelogs, npx aicore init stack detection, repo-wide type-check + green test suites in CI.
• 🔧 Needs deploy to demo live: apply DB migrations (supabase db push — includes the usage_logs compatibility view), deploy worker (Cloudflare) + dashboard (Vercel).
• ⬜ Deferred by design: community context registry, semantic caching, agent-native routing, enterprise/SSO — schema reserved, not built. See roadmap and scope.

Documentation

This repo carries its own delivery operating system under docs/:

• Engineering case study — the decisions, trade-offs, and what I'd do next.
• Product brief · Scope & non-goals · Roadmap
• Architecture overview · ADRs
• Testing strategy · Production-readiness assessment · Known defects

Repository layout

packages/
  types/             # shared contracts (the spine)
  worker/            # Cloudflare Worker edge proxy
  sdk/               # TypeScript client SDK
  cli/               # npx aicore
  logger/            # local + queued usage logging
  telemetry-gateway/ # Fastify telemetry ingest
apps/
  dashboard/         # Next.js 15 dashboard
supabase/migrations/ # canonical DB schema
docs/                # product, architecture, ADRs, engineering

License

MIT © AICore