Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Changelog

## [4.36.3]

- Target the canonical `/v1` sync API routes (`/v1/transcribe`, `/v1/warm`); the unprefixed paths remain served for older SDK versions

## [4.36.2]

- Add opt-in `timestamps` sync config option — when `true`, sync words carry accurate `start`/`end` timings at a small latency cost. By default no timings are computed or returned; `SyncWord.start`/`end` are now optional and absent unless requested

## [4.36.1]

- Rename the sync transcription surface introduced in 4.36.0 (never published under the old names): `client.direct` → `client.sync`, `Direct*` types → `Sync*`, `directBaseUrl` → `syncBaseUrl`; `word_boost` → `keyterms_prompt`
- The default sync speech model is now `universal-3-5-pro` (sent as the `X-AAI-Model` routing header)

## [4.36.0]

- Add `client.sync` — synchronous transcription: audio in, transcript out, one request against the sync API host, with no job id or polling. Accepts a local file path, raw bytes, a Blob, or a readable stream (not URLs), plus config for `prompt`, `keyterms_prompt`, `conversation_context`, `language_codes`, and raw PCM (`sample_rate` + `channels`)
- Add `SyncTranscriber.warm()` — pre-opens the connection to the sync API so the next `transcribe()` skips the DNS + TCP + TLS handshake
- Add `SyncTranscriptError` with `status`, machine-readable `errorCode`, and `retryAfter` (seconds) on 429/503 responses
- Add `syncBaseUrl` client option (defaults to `https://sync.assemblyai.com`)

## [4.35.4]

- `sampleRate` is now optional when `encoding` is `opus` or `ogg_opus` (the Opus stream is self-describing and the server ignores the value). It remains required for PCM encodings and for dual-channel mode; omitting it there now throws at construction time
Expand Down
78 changes: 78 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ const client = new AssemblyAI({
- `client.transcripts.get(id)` — Retrieve a transcript by ID
- `client.transcripts.list()` — List transcripts with pagination
- `client.transcripts.delete(id)` — Delete a transcript
- `client.sync.transcribe(audio, config?, options?)` — Synchronous transcription: audio in, transcript out, one request (no polling)
- `client.streaming.transcriber(params)` — Create a real-time streaming session

## Common patterns
Expand Down Expand Up @@ -174,6 +175,83 @@ const srt = await client.transcripts.subtitles(id, "srt");
const vtt = await client.transcripts.subtitles(id, "vtt");
```

## Sync transcription (pre-recorded, single request)

`client.sync` posts a whole audio file and returns the finished transcript in one
round trip — no job id, no polling, no status enum. It targets the sync API host
(`sync.assemblyai.com`, override with the `syncBaseUrl` client option), distinct
from `client.transcripts`' async job API. Use it for short clips where you want the
answer inline; use `client.transcripts` for long-form audio, URLs, or the rich
audio-intelligence features (speaker labels, chapters, sentiment, …) the sync API
doesn't expose. This is the Node counterpart of the Python SDK's `SyncTranscriber`.

```typescript
const result = await client.sync.transcribe("./call.wav");
console.log(result.text, result.session_id);
for (const w of result.words) {
console.log(w.text, w.confidence); // w.start/w.end need timestamps: true (see below)
}
```

**Input**: a local file path (Node/Bun/Deno), raw bytes (`Uint8Array`/`ArrayBuffer`),
a Blob/File, or a readable stream. **Not** a URL — pass a path/bytes or use
`client.transcripts` for URL ingestion.

**Config** (all optional, second argument):

```typescript
const result = await client.sync.transcribe("./call.wav", {
prompt: "Transcribe verbatim. Preserve disfluencies.", // max 4096 chars, rejected over
keyterms_prompt: ["AssemblyAI", "Lemur", "U3-Pro"], // max 2048 chars total, rejected over
language_codes: ["es"], // or e.g. ["en", "es"] for multilingual; defaults to English; ignored when prompt is set
conversation_context: [
// prior turns oldest-first; capped at 100 turns / 4096 chars — trimmed, not rejected
"I'd like to book a flight to Denver.",
"Sure, what date were you thinking?",
],
});
```

**Word timestamps** are opt-in. By default words carry `text` and `confidence` only —
`start`/`end` are not calculated. `timestamps: true` computes accurate per-word
timings at a small latency cost:

```typescript
const result = await client.sync.transcribe("./call.wav", { timestamps: true });
for (const w of result.words) {
console.log(w.text, w.start, w.end); // milliseconds
}
```

**Raw PCM** (S16LE) needs `sample_rate` + `channels`; WAV reads them from its header.
Setting either field routes the audio as `audio/pcm`, and both must be present:

```typescript
const result = await client.sync.transcribe(rawPcmBytes, {
sample_rate: 16000,
channels: 1,
});
```

**Model**: `config.model` defaults to `"universal-3-5-pro"` and is sent as the
`X-AAI-Model` routing header — never in the request body.

**Errors**: failures throw `SyncTranscriptError` with `.status`, a
machine-readable `.errorCode` (snake_cased problem-details title: `bad_audio`,
`audio_too_short`, `audio_too_large`, `capacity_exceeded`, `inference_timeout`, …),
and `.retryAfter` (seconds) on 429/503. Audio limits: 80 ms–120 s, ≤40 MB, 16-bit,
mono/stereo, sample rate ∈ {8000, 16000, 22050, 24000, 32000, 44100, 48000}.

**Pre-warming**: the sync API is one request/response, so a `transcribe()` that
connects on demand pays the full DNS + TCP + TLS handshake on the critical path.
Call `await client.sync.warm()` as soon as you know audio is coming (e.g. while
it is still being recorded); it returns `true` once the socket is open, `false` on
a transport failure. Call it shortly before `transcribe()` — the pooled connection
idles out after a few seconds.

**Client-side timeout**: third argument — `client.sync.transcribe(audio, {}, { timeout: 30_000 })`
(default 60 s, kept above the server's 30 s deadline).

## Important gotchas

- **.transcribe() polls until complete** — use .submit() for fire-and-forget
Expand Down
98 changes: 98 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,104 @@ const res = await client.transcripts.delete(transcript.id);

</details>

### Transcribe audio synchronously

`client.sync` posts a whole audio file and returns the finished transcript in one
round trip — no job id, no polling. Use it for short clips where you want the answer
inline; use `client.transcripts` for long-form audio, URLs, or the rich
audio-intelligence features the sync API doesn't expose.

```typescript
const result = await client.sync.transcribe("./call.wav");
console.log(result.text, result.session_id);
```

The input can be a local file path, raw audio bytes, a Blob, or a readable
stream — but not a URL.

<details>
<summary>Configure the transcription</summary>

```typescript
const result = await client.sync.transcribe("./call.wav", {
prompt: "Transcribe verbatim. Preserve disfluencies.", // max 4096 chars
keyterms_prompt: ["AssemblyAI", "Lemur"], // max 2048 chars total
language_codes: ["es"], // or e.g. ["en", "es"] for multilingual; defaults to English
conversation_context: [
// prior turns, oldest first
"I'd like to book a flight to Denver.",
"Sure, what date were you thinking?",
],
});
```

Raw S16LE PCM audio needs `sample_rate` and `channels`; WAV reads them from its
header.

```typescript
const result = await client.sync.transcribe(rawPcmBytes, {
sample_rate: 16_000,
channels: 1,
});
```

</details>

<details>
<summary>Get word timestamps</summary>

Word timestamps are opt-in. By default each word in `result.words` carries
`text` and `confidence` only — `start`/`end` are absent. Set `timestamps: true`
to get accurate per-word timings at a small latency cost.

```typescript
const result = await client.sync.transcribe("./call.wav", {
timestamps: true,
});
for (const word of result.words) {
console.log(word.text, word.start, word.end); // milliseconds
}
```

</details>

<details>
<summary>Pre-warm the connection</summary>

The sync API is a single request/response, so a `transcribe()` that connects on
demand pays the full DNS + TCP + TLS handshake on the critical path. Call `warm()`
as soon as you know audio is coming — for example while it is still being
recorded — so the next `transcribe()` reuses the open connection.

```typescript
await client.sync.warm(); // fire as recording starts
const audio = await recordUntilDone();
const result = await client.sync.transcribe(audio); // reuses the hot connection
```

</details>

<details>
<summary>Handle errors</summary>

Failures throw a `SyncTranscriptError` with the HTTP `status`, a machine-readable
`errorCode` (`bad_audio`, `audio_too_large`, `capacity_exceeded`, …), and
`retryAfter` (seconds) on 429/503 responses.

```typescript
import { SyncTranscriptError } from "assemblyai";

try {
const result = await client.sync.transcribe("./call.wav");
} catch (error) {
if (error instanceof SyncTranscriptError) {
console.error(error.status, error.errorCode, error.retryAfter);
}
}
```

</details>

### Transcribe streaming audio

Refer to [AssemblyAI's streaming documentation](https://www.assemblyai.com/docs/streaming/getting-started/transcribe-streaming-audio) for full code examples.
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "assemblyai",
"version": "4.35.4",
"version": "4.36.3",
"description": "The AssemblyAI JavaScript SDK provides an easy-to-use interface for interacting with the AssemblyAI API, which supports async and real-time transcription, as well as the latest LeMUR models.",
"engines": {
"node": ">=18"
Expand Down
26 changes: 18 additions & 8 deletions src/services/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,34 +19,44 @@ export abstract class BaseService {
this.userAgent = buildUserAgent(params.userAgent || {});
}
}
protected async fetch(
protected async fetchResponse(
input: string,
init?: RequestInit | undefined,
): Promise<Response> {
init = { ...DEFAULT_FETCH_INIT, ...init };
let headers = {
let headers: Record<string, string> = {
Authorization: this.params.apiKey,
"Content-Type": "application/json",
};
// FormData bodies must let fetch set the multipart boundary itself.
if (!(init.body instanceof FormData)) {
headers["Content-Type"] = "application/json";
}
if (DEFAULT_FETCH_INIT?.headers)
headers = { ...headers, ...DEFAULT_FETCH_INIT.headers };
if (init?.headers) headers = { ...headers, ...init.headers };
if (init?.headers)
headers = { ...headers, ...(init.headers as Record<string, string>) };

if (this.userAgent) {
(headers as Record<string, string>)["User-Agent"] = this.userAgent;
headers["User-Agent"] = this.userAgent;
if (conditions.browser || conditions.default) {
// chromium browsers have a bug where the user agent can't be modified
if (typeof window !== "undefined" && "chrome" in window) {
(headers as Record<string, string>)["AssemblyAI-Agent"] =
this.userAgent;
headers["AssemblyAI-Agent"] = this.userAgent;
}
}
}
init.headers = headers;

if (!input.startsWith("http")) input = this.params.baseUrl + input;

const response = await fetch(input, init);
return await fetch(input, init);
}

protected async fetch(
input: string,
init?: RequestInit | undefined,
): Promise<Response> {
const response = await this.fetchResponse(input, init);

if (response.status >= 400) {
let json: JsonError | undefined;
Expand Down
19 changes: 19 additions & 0 deletions src/services/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import { BaseServiceParams } from "..";
import { SyncTranscriber } from "./sync";
import { SyncTranscriptError } from "../utils/errors";
import { LemurService } from "./lemur";
import {
RealtimeTranscriber,
Expand All @@ -23,6 +25,7 @@ import {

const defaultBaseUrl = "https://api.assemblyai.com";
const defaultStreamingUrl = "https://streaming.assemblyai.com";
const defaultSyncUrl = "https://sync.assemblyai.com";

class AssemblyAI {
/**
Expand Down Expand Up @@ -50,6 +53,11 @@ class AssemblyAI {
*/
public streaming: StreamingTranscriberFactory;

/**
* The synchronous transcription service.
*/
public sync: SyncTranscriber;

/**
* Create a new AssemblyAI client.
* @param params - The parameters for the service, including the API key and base URL, if any.
Expand All @@ -69,11 +77,22 @@ class AssemblyAI {
...params,
baseUrl: params.streamingBaseUrl || defaultStreamingUrl,
});

let syncBaseUrl = params.syncBaseUrl || defaultSyncUrl;
if (syncBaseUrl.endsWith("/")) {
syncBaseUrl = syncBaseUrl.slice(0, -1);
}
this.sync = new SyncTranscriber({
...params,
baseUrl: syncBaseUrl,
});
}
}

export {
AssemblyAI,
SyncTranscriber,
SyncTranscriptError,
LemurService,
RealtimeTranscriberFactory,
RealtimeTranscriber,
Expand Down
1 change: 1 addition & 0 deletions src/services/sync/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from "./service";
Loading
Loading