TS-DSPy

Build LLM applications in TypeScript by declaring the shape of the input and output you want, instead of hand-writing prompts and parsing replies. Inspired by DSPy.

Model output is validated at runtime against the shape you declared. If a field you declared as a number comes back as prose, you get a ValidationError naming the field — not a string masquerading as a number.

Programs also optimize themselves from data, the part of DSPy that makes it more than a prompt template library. Give BootstrapFewShot a labelled trainset and a metric and it runs your module, keeps the runs that passed, and puts them back in the prompt as worked examples — optionally generated by a stronger teacher model that your cheaper one then imitates. Runnable end to end in examples/optimizer.ts.

npm install @ts-dspy/core @ts-dspy/openai

import { Signature, InputField, OutputField, Predict, configure } from '@ts-dspy/core';
import { OpenAILM } from '@ts-dspy/openai';

class AnswerQuestion extends Signature {
  static description = 'Answer a factual question concisely.';

  @InputField({ description: 'the question to answer' })
  question!: string;

  @OutputField({ description: 'a concise answer' })
  answer!: string;

  @OutputField({ description: 'confidence between 0 and 1', type: 'number' })
  confidence!: number;
}

configure({ lm: new OpenAILM({ apiKey: process.env.OPENAI_API_KEY }) });

const result = await new Predict(AnswerQuestion).forward({
  question: 'What is the capital of France?',
});

console.log(result.answer); // "Paris"
console.log(result.confidence); // 0.98 — a number, verified at runtime

Packages

Package	Provider
`@ts-dspy/core`	Signatures, modules, validation. No provider.
`@ts-dspy/openai`	OpenAI, plus any OpenAI-compatible endpoint, via the official `openai` SDK
`@ts-dspy/gemini`	Google Gemini, via `@google/genai` (Gemini API or Vertex AI)
`@ts-dspy/anthropic`	Anthropic Claude, via `@anthropic-ai/sdk`

Install core plus whichever providers you use. Each provider defaults to a current model for that vendor; pass model to pin one yourself.

@ts-dspy/openai also exports OpenAICompatibleLM for the many servers that speak the OpenAI chat-completions API — Ollama, LM Studio, vLLM, Groq, Together, and OpenRouter. It requires baseURL and model, defaults the API key to a placeholder for local servers that ignore it, and takes model capabilities from config rather than assuming OpenAI's:

import { OpenAICompatibleLM } from '@ts-dspy/openai';

const lm = new OpenAICompatibleLM({
  baseURL: 'http://localhost:11434/v1',
  model: 'llama3.2',
});

npm run example:ollama runs it end to end against a local Ollama, with no cloud key involved.

Requires Node.js 22 or newer. Packages ship both ESM and CommonJS builds.

Downloads

The chart is regenerated from the public npm downloads API on the 2nd of each month by .github/workflows/npm-downloads-chart.yml. To refresh it by hand: npm run chart:downloads.

Concepts

Signatures

A signature declares a task's inputs and outputs. Use a class with decorators when you want descriptions and types:

class AnalyzeReview extends Signature {
  static description = 'Analyze a product review.';

  @InputField({ description: 'the review text' })
  review!: string;

  @OutputField({ description: 'positive, negative, or neutral' })
  sentiment!: string;

  @OutputField({ description: 'rating from 1 to 5', type: 'int' })
  rating!: number;

  @OutputField({ description: 'key themes', type: 'string[]' })
  themes!: string[];

  @OutputField({ description: 'follow-up question', required: false })
  followUp?: string;
}

Or a string, for quick work: 'question -> answer: string, confidence: float'.

Field types: string (default), number/float, int/integer, boolean/bool, string[], number[], array/list, object/json, enum. Set required: false to make a field optional.

An enum field pins the answer to a closed set, so the model cannot invent a fourth value that still passes validation:

@OutputField({ description: 'overall sentiment', type: 'enum', values: ['positive', 'negative', 'neutral'] })
sentiment!: string;

String signatures declare the same thing inline, pipe-separated: 'review -> sentiment: enum(positive|negative|neutral)'.

Class signatures need experimentalDecorators in your tsconfig.json.

Or zod schemas, which need no decorators and infer their own types — see Zod signatures.

Modules

Predict — one call, validated against the signature.
ChainOfThought — reasons in free text first, then answers; the result adds a reasoning field.
RespAct — a reason-and-act loop that calls the tools you provide until it can answer.

All three accept per-call options that are passed through to the provider SDK:

await predict.forward({ question: '...' }, { temperature: 0, timeout: 30_000, retries: 2 });

Pass signal to cancel a call that is already in flight — a React effect tearing down, or a server request whose client disconnected. It composes with timeout: whichever fires first ends the call.

const controller = new AbortController();
// React: return () => controller.abort() from the effect.

await predict.forward({ question: '...' }, { signal: controller.signal, timeout: 30_000 });

When a provider supports native structured output, Predict and ChainOfThought use it — the model is constrained to your schema rather than merely asked for it — and fall back to parsing labelled text otherwise.

Streaming

stream() runs the same prediction as forward(), but yields the output fields as they fill in:

for await (const partial of predict.stream({ question: '...' })) {
  render(partial.answer); // grows as tokens arrive
}

Each yield is a snapshot of the fields parsed so far. The last yield is the complete output, validated against the signature exactly as forward() validates it — a stream that ends in something the signature rejects still throws a ValidationError. The generator's return value is the Prediction wrapper, for callers who drive next() by hand, since for await discards return values.

Only that last snapshot is guaranteed to match the declared types — coercion belongs to validation, so a field declared number may still be the raw '0.' the model is part-way through writing. Snapshots are typed as PartialOutput<T> accordingly: every field optional, and possibly still a string.

const stream = predict.stream({ question: '...' }, { signal: controller.signal });

let step = await stream.next();
while (!step.done) step = await stream.next();

step.value.confidence; // the validated Prediction

Providers with native structured output stream JSON, read by an incremental parser — exported as parsePartialJson — that recovers the fields present in a document truncated mid-string or mid-key. Everything else streams labelled text. A model that cannot stream falls back to a single call, yielded once, rather than failing. Pass an AbortSignal to cancel; breaking out of the loop closes the provider's stream.

Batch and concurrency

Every module inherits batch(), a bounded worker pool over a list of inputs. Results come back in input order, whatever order the calls finished in, and a failing input is captured rather than thrown — one bad row does not destroy a ten-thousand-row job.

const results = await predict.batch(rows, {
  concurrency: 16, // in flight at once; defaults to 8
  onProgress: (done, total) => bar.update(done / total),
});

for (const [i, result] of results.entries()) {
  if (result.status === 'fulfilled') save(rows[i], result.value);
  else quarantine(rows[i], result.reason);
}

Pass stopOnError: true to reject the whole batch on the first failure instead, or a signal to stop starting new inputs on cancellation — both reject rather than returning the inputs that already finished, so wrap a cancellable batch in a try. Every other option is passed through to each underlying call. The pool itself is exported as mapWithConcurrency(items, worker, options) for anything else with a rate limit.

Evaluation

evaluate runs a program over a dataset of Examples and grades every prediction, so a prompt or signature change comes with a number attached rather than a hunch.

import { evaluate, exactMatch, Example, Predict, formatReport } from '@ts-dspy/core';

const dataset = [
  new Example({ question: 'Capital of France?', answer: 'Paris' }).withInputs('question'),
  new Example({ question: 'Capital of Japan?', answer: 'Tokyo' }).withInputs('question'),
];

const report = await evaluate(new Predict(AnswerQuestion), dataset, exactMatch, {
  concurrency: 8,
});

report.score; // mean across every example
report.results; // per example: inputs, expected, prediction, score, error?
report.usage; // tokens and latency for this run only

console.log(formatReport(report));

A metric is (example, prediction) => number | boolean, so writing your own is a one-line function. Built in: exactMatch, normalizedMatch, numericMatch, fieldAccuracy for per-field partial credit, and tokenF1 for free text.

An example whose program or metric throws is recorded as a zero with the error attached and the run continues — an evaluation that dies on row 40 of 500 tells you nothing. Usage is measured by diffing the model's own counters around the run; as everywhere else, there is no cost estimate.

Validation

import { ValidationError } from '@ts-dspy/core';

try {
  const result = await predict.forward({ question: '...' });
} catch (error) {
  if (error instanceof ValidationError) {
    for (const issue of error.issues) {
      console.error(`${issue.field} (${issue.expected}): ${issue.message}`);
    }
    console.error('raw model output:', error.rawOutput);
  }
}

Coercion is deliberately lenient — models emit text, so "42" satisfies a number field and "a, b, c" satisfies a string[]. What is not lenient is failure: anything that cannot be coerced throws rather than silently passing through.

Self-repair

A model that fails validation has often understood the task and merely fumbled the shape. Pass repairAttempts to spend that many extra round-trips telling it exactly what went wrong before giving up:

const result = await predict.forward({ question: '...' }, { repairAttempts: 1 });

The follow-up prompt names each failing field with its declared type and the value that actually arrived. It works on both of Predict's paths — native structured output and labelled text — and ChainOfThought inherits it, retrying only the answering step rather than reasoning again. The default is 0, so validation failures throw immediately unless you opt in; once the attempts are spent the last ValidationError is rethrown. Attempts are capped at 10, and the loop stops early if an attempt reproduces the previous failure exactly.

Provider errors

A failed provider call throws a typed subclass of LMError whenever the SDK gives us enough to tell, so retry logic is a catch on a class rather than a sniff at an HTTP status number:

import { RateLimitError, ContextLengthError, LMError } from '@ts-dspy/core';

try {
  return await predict.forward({ question });
} catch (error) {
  if (error instanceof RateLimitError) return queue.retryLater(question);
  if (error instanceof ContextLengthError)
    return predict.forward({ question: shorten(question) });
  if (error instanceof LMError) log.error(error.provider, error.status, error.cause);
  throw error;
}

Error	Thrown when
`RateLimitError`	A rate or quota limit was hit. Worth retrying after a backoff.
`AuthError`	The key is missing, wrong, or not entitled to the model. Retrying will not help.
`ContextLengthError`	The prompt did not fit the context window.
`ContentFilterError`	Safety classifiers declined the request or the reply. Carries `category`.
`TimeoutError`	The request timed out before a reply arrived.
`LMError`	Anything else. Every class above extends it, so existing handlers still work.

What each SDK reports differs, and the classification follows that rather than pretending otherwise: OpenAI's code is the only dependable signal for a context-length overflow, Anthropic's typed error.type union is the cleanest of the three, and Gemini reports nothing but an HTTP status.

Content filtering is the exception — all three providers answer 200 OK and leave a marker in the body (stop_reason: 'refusal', finishReason: 'SAFETY', finish_reason: 'content_filter'), so ContentFilterError comes from inspecting the response. AnthropicRefusalError is now a deprecated alias of ContentFilterError — an alias of that class rather than a subclass, so an instanceof check under the old name now matches any provider's filter. Test error.provider to tell them apart.

Zod signatures

signature() builds a signature from zod schemas. The shape lives in the type system rather than in runtime metadata, so results are inferred exactly — no type argument, and no experimentalDecorators:

import { z } from 'zod';
import { signature, Predict } from '@ts-dspy/core';

const AnalyzeReview = signature({
  description: 'Analyze a product review.',
  input: z.object({ review: z.string() }),
  output: z.object({
    sentiment: z.enum(['positive', 'negative', 'neutral']),
    rating: z.number().int().min(1).max(5),
    themes: z.array(z.string()),
    followUp: z.string().optional(),
  }),
});

const r = await new Predict(AnalyzeReview).forward({ review });

r.sentiment; // 'positive' | 'negative' | 'neutral' — inferred
r.themes.join(', '); // string[]
r.followUp?.trim(); // string | undefined

The zod schema is the validator, so anything you can express is enforced: enums, unions, nested objects, numeric bounds, string formats, and object-level refinements — none of which the flat decorator field-type list can spell. Input keys are typed too, so a misspelt input is a compile error.

On the provider structured-output path the same schema becomes the JSON Schema sent to the model, rewritten for OpenAI strict mode: every property in required, additionalProperties: false at every level of nesting, and optional fields expressed as type: [base, 'null'].

Decorator and string signatures keep working unchanged; this is a third form, not a replacement.

Output types

A zod signature infers its output type, as above. Decorators record fields at runtime, so TypeScript cannot infer per-field types from the class — results from a class signature are typed loosely by default. Name the shape when you want precise types:

type ReviewAnalysis = { sentiment: string; rating: number; themes: string[] };

const analysis = await new Predict<typeof AnalyzeReview, ReviewAnalysis>(AnalyzeReview).forward(
  { review }
);

analysis.themes.join(', '); // typed as string[]

Runtime validation comes from the signature either way.

Tools

const agent = new RespAct(AnswerQuestion, {
  tools: {
    search: {
      description: 'Search the web. Input: a query string. Returns snippets.',
      function: async (query: string) => search(query),
    },
  },
  maxSteps: 8,
  onEvent: (event) => console.log(event),
});

Give a tool a parameters schema — JSON Schema or Zod — and it takes named, validated arguments instead of one string:

const agent = new RespAct(AnswerQuestion, {
  tools: {
    flights: {
      description: 'Find flights between two airports on a date.',
      parameters: z.object({ from: z.string(), to: z.string(), date: z.string() }),
      function: ({ from, to, date }) => search(from, to, date),
    },
  },
});

RespAct picks its execution path from the model's capabilities. Against a provider reporting supportsFunctionCalling: true — all three of ours do — tools are declared in the request and the model's calls come back as structured data, so several tools can run in one turn. Against anything else, the loop falls back to prompting for Action: / Action Input: and parsing the reply, which works on any completion model. Both paths run the same tools and emit the same events; pass forceTextMode: true to pin a tool-capable model to the text loop.

Tool descriptions are what the model uses to decide when to call each tool, so they earn the detail. Never pass model output to eval() — see examples/utils.ts for a bounded arithmetic evaluator.

Images

ChatMessage.content is string | ContentPart[], so one message can carry text and images together. Declare an image input with @ImageField — or the image type in a string signature — and render the message with buildPromptContent:

import { Signature, ImageField, OutputField, buildPromptContent } from '@ts-dspy/core';

class ReadSign extends Signature {
  static description = 'Read the sign in the photo.';

  @ImageField({ description: 'photo of the sign' })
  photo!: string;

  @OutputField({ description: 'the words on the sign' })
  words!: string;
}

const content = buildPromptContent(ReadSign, { photo: 'data:image/png;base64,...' });
await lm.chat([{ role: 'user', content }]);

imagePart() accepts an https:// URL, a data: URI, or an explicit { kind: 'base64', data, mediaType } source. OpenAI receives image_url parts, Anthropic image blocks, and Gemini inlineData (Gemini fetches no arbitrary web URLs, so pass bytes or a Files API URI). Only user turns can carry an image, so system and assistant content is flattened to text. Modules still build string prompts, where an image input renders as an [image: image/png] placeholder — send images through lm.chat() for now. A plain string content behaves exactly as it did, and supportsVision is reported per model instead of being hardcoded to true.

Testing

@ts-dspy/core/testing ships the test doubles the library's own suite uses, so you never have to hand-roll a fake model. Nothing there touches the network.

import { MockLM, CassetteLM } from '@ts-dspy/core/testing';

const lm = new MockLM({ responses: ['answer: Paris\nconfidence: 0.95'] });
const result = await new Predict(AnswerQuestion, lm).forward({ question: 'Capital?' });

lm.lastPrompt(); // the prompt the module actually sent

CassetteLM records real provider replies into a JSON file once, then replays them forever:

// Once, with a key:
const recorder = CassetteLM.record('cassettes/answer.json', new OpenAILM({ apiKey }));
await new Predict(AnswerQuestion, recorder).forward({ question: 'Capital?' });

// In CI, with no key and no network:
const replay = CassetteLM.replay('cassettes/answer.json');

Cassettes are an array of { key, request, response } entries keyed by a hash of the request, so they review like any other fixture and an unrecorded request fails loudly instead of calling out.

Tracing

configure({ tracing: true }) records every module invocation: the prompt as sent, the raw reply, the parsed output, the tokens that call cost, and how long it took. inspectHistory(n) reads the last n back out of a bounded in-memory buffer — including inside a catch, which is where you usually want it.

import { clearHistory, configure, inspectHistory } from '@ts-dspy/core';

configure({ lm, tracing: true });

try {
  await triage.forward({ ticket });
} catch (error) {
  const [failed] = inspectHistory(1);
  console.error(failed.rawLMInput, '\n---\n', failed.rawLMOutput);
}

Multi-step modules record each call under calls, so a ChainOfThought trace holds both the reasoning prompt and the final one. The buffer keeps 100 entries by default (traceHistorySize), and clearHistory() empties it.

Pass onTrace to forward entries to Langfuse, OpenTelemetry, or your own logger as they are recorded — the library never prints:

configure({ lm, tracing: true, onTrace: (entry) => logger.debug(entry) });

Tracing is off by default and costs a single boolean check while off.

Caching

configure({ cache: true }) replays a previous answer instead of paying for a repeated prompt. It is off by default, because replaying an old answer changes what a program does.

import { configure, MemoryCache, type Cache } from '@ts-dspy/core';

configure({ cache: true }); // process-wide LRU, 1000 entries
configure({ cache: new MemoryCache({ maxSize: 10_000 }) }); // or size it yourself

generate, chat, and generateStructured are all cached for every provider. The key is a SHA-256 hash of the provider, the model, the prompt or messages, the sampling parameters (temperature, topP, maxTokens, stopSequences, and the penalties), and the JSON schema on structured calls — so two calls that differ in any of those never collide. Errors are never cached.

A cache hit costs nothing, and getUsage() says so: hits land in a cacheHits counter and stay out of requestCount and the token totals, so cost accounting still reflects real provider traffic.

cache also accepts your own implementation. Both methods may be async, so Redis, SQLite, or a directory of files fits without a wrapper:

const redisCache: Cache = {
  async get(key) {
    const hit = await redis.get(key);
    return hit === null ? undefined : JSON.parse(hit);
  },
  async set(key, value) {
    await redis.set(key, JSON.stringify(value), { EX: 86_400 });
  },
};

configure({ cache: redisCache });

Examples

git clone https://github.com/ardada2468/LLMTypeSafe.git
cd LLMTypeSafe
npm install
npm run build

export OPENAI_API_KEY="sk-..."
npm run example:openai
npm run example:zod

See examples/ for OpenAI, Gemini, Anthropic, zod-signature, and tool-use programs.

Development

npm install
npm run build              # tsup, per package (core first — providers need its types)
npm test                   # vitest
npm run lint
npm run typecheck          # packages and examples
npm run format:check
npm run verify:packaging   # pack, install, and import as a real consumer would

Every one of these runs in CI on each pull request, across Node 22, 24, and 26. The release workflow runs the same set before it can publish, so the release path cannot drift from the path changes are reviewed through.

verify:packaging is the one worth knowing about: it packs the tarballs, installs them into a throwaway project, and imports them from both ESM and CommonJS. Builds, type checks, and unit tests all run against workspace symlinks, so none of them can see a wrong dependency range — which is how 0.4.2 shipped providers depending on a version of @ts-dspy/core that release did not satisfy.

Releases run on changesets: add one with npx changeset when you change a published package, or npx changeset --empty for a change that intentionally ships no release. CI fails a pull request that changes a published package without one.

Publishing uses npm trusted publishing, so there is no npm token to hold or rotate. See RELEASING.md.

License

MIT — see LICENSE.

Name		Name	Last commit message	Last commit date
.changeset		.changeset
.github/workflows		.github/workflows
assets		assets
examples		examples
packages		packages
scripts		scripts
site		site
.gitignore		.gitignore
.prettierignore		.prettierignore
.prettierrc.json		.prettierrc.json
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
RELEASING.md		RELEASING.md
eslint.config.js		eslint.config.js
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
vitest.config.ts		vitest.config.ts

Navigation Menu

Repository files navigation

TS-DSPy

Packages

Downloads

Concepts

Signatures

Modules

Streaming

Batch and concurrency

Evaluation

Validation

Self-repair

Provider errors

Zod signatures

Output types

Tools

Images

Testing

Tracing

Caching

Examples

Development

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

TS-DSPy

Packages

Downloads

Concepts

Signatures

Modules

Streaming

Batch and concurrency

Evaluation

Validation

Self-repair

Provider errors

Zod signatures

Output types

Tools

Images

Testing

Tracing

Caching

Examples

Development

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages