Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 25 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The open-source, local-first AI video editor — chat, CLI, or browser timeline.
<a href="./LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT"></a>
<a href="./package.json"><img src="https://img.shields.io/badge/node-%E2%89%A524-brightgreen" alt="Node 24+"></a>
<a href="https://www.npmjs.com/package/@makemyclip/editor"><img src="https://img.shields.io/npm/v/@makemyclip/editor.svg" alt="npm"></a>
<img src="https://img.shields.io/badge/tests-356%20passing-brightgreen" alt="Tests: 356 passing">
<img src="https://img.shields.io/badge/tests-539%20passing-brightgreen" alt="Tests: 539 passing">
<img src="https://img.shields.io/badge/telemetry-none-brightgreen" alt="Telemetry: none">
</p>

Expand Down Expand Up @@ -143,26 +143,42 @@ Legend: ✅ great fit · ⚠️ works but not the best · ❌ doesn't fit.
</details>

<details>
<summary><b>Architecture</b> — three surfaces, one registry, one append-only op log</summary>
<summary><b>Architecture</b> — three surfaces, one composition document, one append-only op log</summary>

```
Claude Code → skill triggers → npx -y @makemyclip/editor <tool>
Claude Code → skill triggers → npx -y @makemyclip/editor <tool | timeline …>
Browser UI → /api/tools/:name TOOL_REGISTRY (19 tools)
Browser UI → /api/tools/:name · /api/timeline/verbs registry + verb layer
Chat panel → /api/chat → AI SDK + Anthropic + tool dispatch
Chat panel → /api/chat → AI SDK + Anthropic + timeline verbs
Tool handlers (TypeScript)
FFmpeg subprocess (args as array, no shell)
session.json (append-only op log)
┌───────────────────────┴───────────────────────┐
▼ ▼
Composition document session.json
(non-destructive timeline, (append-only op log
op-log undo/redo) of every tool call)
```

Two layers of state, by design. The **composition document** is the source of truth for assembled edits: a non-destructive, multi-track timeline that `clip timeline`, the browser UI, and the chat agent all mutate through one op-aware path, with a coupled op log that powers undo/redo. **`session.json`** is an append-only op log of every tool invocation — `{ id, tool, args, result, timestamp }` — written through the same `appendOp()` path by the single-file tools (`clip trim`, `/api/tools/:name`, …); it is the audit trail and recovery layer. Both layers are written through one serialized path, so any combination of human + agent edits stays consistent.

**Build a composition end-to-end** (CLI shown; the browser UI and chat agent drive the same document):

```bash
clip timeline new # start an empty composition
clip timeline add-media intro.mp4 # ingest + append a clip
clip timeline add-media demo.mp4
clip timeline transition <afterClipId> fade 0.5
clip timeline show # inspect tracks, clips, timings
clip timeline export final.mp4 # compile the document to a render
```

The session is the source of truth: every editing operation appends one entry with `{ id, tool, args, result, timestamp }`. The CLI, browser UI, and chat panel all write through the same `appendOp()` path, so any combination of human + agent edits stays consistent.
Every edit is undoable (`clip timeline undo` / `redo`, `clip timeline log`), and `clip timeline at <sec>` / `frame <sec>` give an agent read-only eyes on the document. Run `clip timeline --help` for the full subcommand list.

- **Language:** TypeScript (Node 24+) and React 19 (browser UI)
- **Timeline schema:** [Zod](https://zod.dev/) (shared with the [MakeMyClip.com](https://makemyclip.com) web app)
Expand Down Expand Up @@ -193,7 +209,7 @@ Only the chat panel inside `clip ui` needs `ANTHROPIC_API_KEY`. The CLI, the bro
### Capabilities & limits

**Is it production-ready?**
This release is feature-complete for local editing — 19 tools, 356 tests passing, browser UI shipped, chat panel shipped. The API surface is still pre-1.0 (tool schemas may change in minor ways before 1.0). Use it for real work; pin a version in CI.
This release is feature-complete for local editing — 19 tools, 539 tests passing, browser UI shipped, chat panel shipped. The API surface is still pre-1.0 (tool schemas may change in minor ways before 1.0). Use it for real work; pin a version in CI.

**What's the maximum video size or duration?**
No hardcoded limit. FFmpeg streams the file rather than loading it into memory, and the browser UI streams uploads to disk — multi-GB recordings work fine. This release isn't tuned for projects with hundreds of clips or runtime above 30 minutes; for those, drive the CLI directly.
Expand Down
16 changes: 15 additions & 1 deletion SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ license: MIT
compatibility: Requires Node 24+. FFmpeg is auto-downloaded via ffmpeg-static on first use; override with MAKEMYCLIP_FFMPEG_PATH.
metadata:
author: makemyclip
version: "0.0.1"
homepage: https://github.com/MakeMyClip/editor
runtime: node
install: npx skills add MakeMyClip/editor
Expand Down Expand Up @@ -299,6 +298,21 @@ Two-pass `vidstab`: pass 1 (`vidstabdetect`) analyzes motion and writes a transf

Requires `vidstab`-enabled ffmpeg. The bundled `ffmpeg-static` includes it; the tool fails with a clear error otherwise.

### Build a multi-clip composition — the timeline (implemented)

The single-file tools above each produce a new output file. To assemble several clips into one edit — with non-destructive trims, transitions, and **undo/redo** — drive the **timeline**: a composition document that is the source of truth for the assembled edit. The CLI, the browser UI, and this skill all mutate the same document.

```bash
npx -y @makemyclip/editor timeline new # start an empty composition
npx -y @makemyclip/editor timeline add-media intro.mp4 # ingest + append a clip
npx -y @makemyclip/editor timeline add-media demo.mp4
npx -y @makemyclip/editor timeline transition <afterClipId> fade 0.5
npx -y @makemyclip/editor timeline show # read tracks, clips, timings
npx -y @makemyclip/editor timeline export final.mp4 # compile the document to a render
```

Prefer the timeline over chaining single-file tools whenever the user is *assembling* a video (multiple clips, transitions, a final render): edits are non-destructive and reversible (`timeline undo` / `redo`, `timeline log`), and `timeline at <sec>` / `timeline frame <sec>` give you read-only eyes on the current document before you change it. Run `npx -y @makemyclip/editor timeline --help` for the full subcommand list.

### Open the local UI (implemented)

```bash
Expand Down
2 changes: 1 addition & 1 deletion llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
- **Backend:** FFmpeg (bundled via `ffmpeg-static`)
- **Surfaces:** Claude Code skill · `clip` CLI · browser UI at `127.0.0.1:5573`
- **AI integration:** Anthropic via Vercel AI SDK (optional; rest of editor works without API key)
- **Tests:** 356 passing
- **Tests:** 539 passing
- **License:** MIT (code) + GPL (bundled FFmpeg binary, subprocess-isolated)

## How to install
Expand Down
5 changes: 5 additions & 0 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ Specialty:
clip stabilize <input> [<shakiness>] [<smoothing>] [<accuracy>] [<zoom>]
Two-pass vidstab. Defaults: shakiness=5, smoothing=10, accuracy=9, zoom=5.

Timeline (non-destructive composition — the source of truth for assembled edits):
clip timeline <subcommand> Build and export a multi-clip composition with
undo/redo. new → add-media → … → export.
clip timeline --help List the timeline subcommands.

UI:
clip ui Start the local browser UI on http://127.0.0.1:5573.
Renders the session log; click an op to play its output.
Expand Down
6 changes: 3 additions & 3 deletions src/ffmpeg/args/transition.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/**
* Subset of ffmpeg's `xfade` transition kinds. Chosen to match the common
* iMovie / CapCut palette without overwhelming the agent with 40+ obscure
* options. Each one maps 1:1 to an `xfade` `transition=<name>` value.
* Subset of ffmpeg's `xfade` transition kinds. Chosen to cover the common
* consumer-editor transition palette without overwhelming the agent with 40+
* obscure options. Each one maps 1:1 to an `xfade` `transition=<name>` value.
*/
export type TransitionKind =
| 'fade'
Expand Down
50 changes: 39 additions & 11 deletions src/timeline/compile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@ import type { MediaId } from './schema.js';
* per-clip transform, chromaKey (which needs a background layer), or timeline
* gaps/overlaps — throw a clear `CompileError` rather than emitting a wrong graph.
*
* Known interaction (v1): a per-clip fadeOut/fadeIn placed on the SAME boundary
* as a transition double-darkens, since the xfade already supplies the blend over
* that window. Prefer one or the other on a given cut; fixing the overlap is a
* follow-up.
* Transition boundaries own their blend: a per-clip fadeOut/fadeIn on the SAME
* cut as a transition is dropped during segment normalization (see the
* `FadeSuppression` plumbing), so the xfade's blend isn't stacked over a fade
* over the same window. Fades on outer boundaries (open from black, close to
* black) and on plain cuts are untouched.
*/

export interface MediaInfo {
Expand Down Expand Up @@ -151,9 +152,22 @@ interface FilterChains {
audio: string[];
}

/** Whether a clip's leading/trailing per-clip fade should be dropped because a
* transition already owns the blend over that boundary (else they double-darken). */
interface FadeSuppression {
in: boolean;
out: boolean;
}

/** Translate a clip's effect stack into video/audio filter fragments, applied
* after the geometry/source fragments. Order: color → speed → fades. */
function effectFilters(clip: Clip, outDur: number): FilterChains {
* after the geometry/source fragments. Order: color → speed → fades. A fade on a
* boundary that carries a transition is skipped (`suppress`) so the xfade's blend
* isn't stacked over a per-clip fade on the same window. */
function effectFilters(
clip: Clip,
outDur: number,
suppress: FadeSuppression = { in: false, out: false },
): FilterChains {
const video: string[] = [];
const audio: string[] = [];

Expand Down Expand Up @@ -181,11 +195,13 @@ function effectFilters(clip: Clip, outDur: number): FilterChains {
for (const e of clip.effects) {
// Clamp the fade to the clip so a fade longer than the clip still reaches
// full black/silence within the available window instead of ramping partway.
if (e.type === 'fadeIn') {
// Skip the fade entirely when a transition owns this boundary (the xfade
// already blends that window — a per-clip fade on top double-darkens it).
if (e.type === 'fadeIn' && !suppress.in) {
const d = Math.min(e.durationSec, outDur);
video.push(`fade=t=in:st=0:d=${d}`);
audio.push(`afade=t=in:st=0:d=${d}`);
} else if (e.type === 'fadeOut') {
} else if (e.type === 'fadeOut' && !suppress.out) {
const d = Math.min(e.durationSec, outDur);
const st = Math.max(0, outDur - d);
video.push(`fade=t=out:st=${st}:d=${d}`);
Expand Down Expand Up @@ -231,12 +247,13 @@ function buildSegmentStep(
clip: Clip,
index: number,
ctx: CompileContext,
fadeSuppress: FadeSuppression = { in: false, out: false },
): FfmpegStep {
assertCompilable(clip);
const outDur = outputDuration(clip);
const { width, height, fps, background } = comp;
const out = segPath(ctx.dir, index, clip.id);
const { video: fx, audio: afx } = effectFilters(clip, outDur);
const { video: fx, audio: afx } = effectFilters(clip, outDur, fadeSuppress);
const textFiles: StepSideFile[] = [];

const inputArgs: string[] = [];
Expand Down Expand Up @@ -496,9 +513,20 @@ export function compileTimeline(comp: Composition, ctx: CompileContext): FfmpegP

const steps: FfmpegStep[] = [];

// 1. Normalize every clip to a segment.
// 1. Normalize every clip to a segment. Drop a per-clip fade on a boundary that
// a transition already blends: fadeOut when a transition follows this clip,
// fadeIn when one precedes it (the preceding clip has a transition after it).
// The fold (step 2) only xfades a transition that has a FOLLOWING clip, so a
// transition keyed to the last clip (e.g. left dangling after the next clip was
// removed) blends nothing — keep that clip's fadeOut rather than silently
// dropping a fade-to-black to a hard cut.
const segments = clips.map((clip, i) => {
const step = buildSegmentStep(comp, clip, i, ctx);
const prevClip = clips[i - 1];
const fadeSuppress: FadeSuppression = {
in: prevClip ? transitionAfter.has(prevClip.id) : false,
out: i < clips.length - 1 && transitionAfter.has(clip.id),
};
const step = buildSegmentStep(comp, clip, i, ctx, fadeSuppress);
steps.push(step);
return { clip, output: step.output, durationSec: outputDuration(clip) };
});
Expand Down
8 changes: 5 additions & 3 deletions src/timeline/document-store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -253,15 +253,17 @@ export async function applyVerbs(
// against, so if another in-process writer commits between the read and the
// apply, re-lower against the fresh doc rather than land overlapping clips. A
// retry re-runs the impure lowering (it may re-`ingest`), which is acceptable
// on the rare conflict path.
for (let attempt = 1; attempt <= 4; attempt++) {
// on the rare conflict path. Retry to the same cap as the inner commit loop so
// a burst of N concurrent edits each re-lower and land instead of the (N-cap)
// tail silently losing to a `CompositionConflictError`.
for (let attempt = 1; attempt <= MAX_COMMIT_ATTEMPTS; attempt++) {
const current = await readComposition();
const ops = await lowerVerbs(current, verbs, ctx);
try {
const doc = await mutateComposition(ops, { expectedBaseRev: current.rev });
return { doc, ops };
} catch (err) {
if (err instanceof CompositionConflictError && attempt < 4) continue;
if (err instanceof CompositionConflictError && attempt < MAX_COMMIT_ATTEMPTS) continue;
throw err;
}
}
Expand Down
71 changes: 66 additions & 5 deletions src/workspace.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { randomBytes } from 'node:crypto';
import { lstatSync, readlinkSync, realpathSync } from 'node:fs';
import { mkdir } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { isAbsolute, relative, resolve, sep } from 'node:path';
import { basename, dirname, isAbsolute, relative, resolve, sep } from 'node:path';

const WORKSPACE_ENV = 'MAKEMYCLIP_WORKSPACE';

Expand Down Expand Up @@ -39,21 +40,81 @@ export class WorkspaceBoundaryError extends Error {
}
}

const MAX_SYMLINK_HOPS = 40;

/**
* Canonicalize `p` for the workspace containment check: resolve symlinks on the
* existing prefix so the comparison runs on REAL targets, while tolerating a
* not-yet-created trailing path (a tool may be about to create it). Symlinks are
* followed explicitly — including a DANGLING one, whose (possibly out-of-tree)
* target is resolved so it can't be mistaken for an in-workspace leaf. Only a
* genuine `ENOENT` is treated as a missing tail; any other error (`EACCES`,
* `ELOOP`, a symlink cycle) throws so the caller can fail closed rather than
* decide containment on a path it could not fully resolve.
*/
function canonicalizeForCheck(p: string): string {
let current = resolve(p);
const tail: string[] = [];
let hops = 0;
for (;;) {
let stats: ReturnType<typeof lstatSync>;
try {
stats = lstatSync(current);
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') throw err;
const parent = dirname(current);
if (parent === current) return resolve(p);
tail.unshift(basename(current));
current = parent;
continue;
}
if (stats.isSymbolicLink()) {
if (++hops > MAX_SYMLINK_HOPS) {
throw new Error(`Too many symlink hops resolving "${p}".`);
}
// Follow one hop toward the target (keeping the not-yet-resolved tail) so a
// dangling or outward link is canonicalized to its real destination.
current = resolve(dirname(current), readlinkSync(current));
continue;
}
// A real, non-symlink entry: realpath collapses any symlinks in ITS prefix.
const real = realpathSync(current);
return tail.length > 0 ? resolve(real, ...tail) : real;
}
}

/**
* Resolve `path` and CONFINE it to the workspace — the trust boundary for
* untrusted input (the agent's verbs/tools and the localhost `clip ui` routes).
* Relative paths resolve against the workspace; absolute paths must already sit
* inside it. Throws `WorkspaceBoundaryError` on traversal or any path outside the
* tree. The trusted CLI keeps `resolveInput`, where a user-typed path is consent.
*
* Containment is by resolved-path prefix; it does NOT follow symlinks, so a
* symlink placed inside the workspace can still point out — hardening that is a
* follow-up, but `..`-traversal and absolute out-of-tree paths are rejected here.
* Containment is checked on the REAL (symlink-collapsed) paths, so an in-workspace
* symlink — existing-target OR dangling — cannot smuggle a read outside the tree:
* each side is canonicalized via `canonicalizeForCheck` before comparing (a
* symlinked workspace root, e.g. macOS `/var` -> `/private/var`, is collapsed too
* so legit paths aren't falsely rejected). If a path can't be fully canonicalized
* (permission error, symlink loop) it fails closed. The lexical resolved path is
* returned, so an in-workspace symlink to an in-workspace target still works.
*
* Residual: an open-time TOCTOU race (the path is re-followed when FFmpeg opens
* it) is not closed here — that needs `O_NOFOLLOW` at the open site.
*/
export function resolveInWorkspace(path: string): string {
const workspace = getWorkspace();
const resolved = isAbsolute(path) ? resolve(path) : resolve(workspace, path);
const rel = relative(workspace, resolved);
let realWorkspace: string;
let realResolved: string;
try {
realWorkspace = canonicalizeForCheck(workspace);
realResolved = canonicalizeForCheck(resolved);
} catch {
// Couldn't canonicalize (EACCES / symlink loop / …) — don't decide on a
// partially-resolved path; refuse.
throw new WorkspaceBoundaryError(path, workspace);
}
const rel = relative(realWorkspace, realResolved);
// Outside the tree if the relative path is empty (the workspace dir itself),
// escapes via '..', or is absolute (e.g. a different Windows drive).
if (rel === '' || rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel)) {
Expand Down
Loading
Loading