diff --git a/AGENTS.md b/AGENTS.md index af7b0ab..0bfdfd2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,14 +8,14 @@ OpenTake 是 Palmier Pro 的跨平台社区分支：Rust core（Tauri 2 + React ``` PRIMARY-CN/ ├── palmier-pro-upstream/ # 上游只读参考（Swift macOS 视频编辑器，GPL-3.0） -│ └── Sources/PalmierPro/ # 209 .swift，~43K 行，编辑逻辑的真理来源 -└── OpenTake/ # 本项目 - ├── docs/ # 架构 / 路线图 / 模块移植规格 +│ └── Sources/PalmierPro/ # 上游编辑逻辑的真理来源 +└── OpenTake/ # 本项目（当前工作仓库） + ├── assets/ # 品牌图标与静态资源 + ├── crates/ # Rust workspace（domain / ops / project / media / render / agent / gen / core） + ├── docs/ # 架构 / 路线图 / 规格 / 上游拆解 │ └── _analysis/ # 上游拆解报告（4 份横切分析） - ├── crates/ # Rust workspace（待创建） - ├── src-tauri/ # Tauri 2 桌面壳（待创建） - ├── web/ # React + TypeScript 前端（待创建） - └── services/ # 可选后端服务（待创建） + ├── src-tauri/ # Tauri 2 桌面壳 + └── web/ # React + TypeScript 前端 ``` ## 从何处开始 @@ -46,9 +46,10 @@ PRIMARY-CN/ | 桌面壳 | Tauri 2 | | 前端 | React + TypeScript + Vite | | 状态管理 | Zustand（前端只读镜像） | -| 编解码 | ffmpeg-next（libav*） | +| 编解码 | ffmpeg-sidecar（调用系统 ffmpeg/ffprobe） | | 帧合成 | wgpu（自写合成器） | | 音频播放 | cpal | +| 语义搜索 | ort + SigLIP2（tokenizers 预处理） | | MCP server | rmcp（streamable-http-server） | ## 移植法则 @@ -75,7 +76,7 @@ PRIMARY-CN/ - 所有数值常量走 `AppTheme`，不硬编码。 - 悬停态用 CSS `:hover` + 圆角背景，图标用 lucide-react。 -## 构建（Phase 0 完成后） +## 构建 ```bash # Rust core @@ -90,7 +91,7 @@ cd web && pnpm install && pnpm build cargo tauri dev ``` -当前状态：**设计阶段**，代码尚未产生。ROADMAP Phase 0 为工程脚手架。 +当前状态：核心实现已落地，Rust workspace、Tauri 壳、React 前端、MCP/Agent 层和主要文档都在仓库中；ROADMAP 继续追踪剩余的高风险差距和后续阶段。 ## 上游参考 diff --git a/CLAUDE.md b/CLAUDE.md index d8fe3f1..02f2c9d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -35,7 +35,7 @@ ## 0. 项目身份 - OpenTake = [palmier-io/palmier-pro](https://github.com/palmier-io/palmier-pro)(Swift 原生 macOS 视频编辑器,GPL-3.0)的**跨平台社区分支**。**忠实 1:1 复刻其编辑逻辑与 UI**(用户反复强调:别自己发明,照上游源码复刻;除登录/账号外都对齐),再加更强 Agent 能力。 -- 栈:Tauri 2 + Rust workspace + React/TS;媒体 = FFmpeg(sidecar)+ wgpu(合成)+ cpal + whisper-rs + candle/ort。 +- 栈:Tauri 2 + Rust workspace + React/TS;媒体 = ffmpeg-sidecar + wgpu(合成)+ cpal + whisper-rs + ort/SigLIP2。 - GitHub `appergb/OpenTake`(Public,main 受保护)。许可 GPL-3.0。 ## 1. 目录 diff --git a/DECISIONS.md b/DECISIONS.md index 79599d9..58c8fec 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -23,7 +23,7 @@ PRIMARY-CN/ | 整体架构 | **Tauri 2**(Rust core + Web 前端) | 与 Rust 核心天然契合;二进制小;一套代码覆盖三桌面平台 | | 核心语言 | **Rust** | 领域模型 / 编辑操作 / 工程格式 / 媒体管线 / MCP server | | 前端 | **React + TypeScript** | 生态最全,AI/人协作最顺,契合既有 web 规则 | -| 媒体引擎 | **FFmpeg 绑定**(`ffmpeg-next` 等) | 替代 AVFoundation:解码/合成/导出/缩略图/波形;LGPL/GPL 与本项目 GPL-3.0 兼容 | +| 媒体引擎 | **FFmpeg 调用层**(`ffmpeg-sidecar`, 调用系统 `ffmpeg`/`ffprobe`) | 替代 AVFoundation:解码/合成/导出/缩略图/波形;LGPL/GPL 与本项目 GPL-3.0 兼容 | | 目标平台 | **桌面优先**:macOS / Windows / Linux | 先把跨平台桌面做扎实(已实现"让 mac 用户也能用 + 扩到 Win/Linux") | | 生成式 AI | **自建后端 / BYOK** | 上游 genAI 闭源(Convex+Clerk),代码不在仓库,必须新建:轻量代理对接 fal.ai/Replicate/各厂商 API | diff --git a/README.ja.md b/README.ja.md index 5deeec6..ca72ad3 100644 --- a/README.ja.md +++ b/README.ja.md @@ -98,11 +98,11 @@ OpenTake は CapCut / DaVinci Resolve / Final Cut Pro の代替品ではあり | 機能 | 技術 | |:--|:--| -| コーデック | FFmpeg (`ffmpeg-next`) | +| コーデック | ffmpeg-sidecar（システムの ffmpeg/ffprobe を呼び出し） | | コンポジター | wgpu カスタムコンポジター | | 音声再生 | cpal | | 文字起こし | whisper-rs | -| 意味検索 | candle / ort + SigLIP2 | +| 意味検索 | ort + SigLIP2 + tokenizers | ### 🌐 BYOK AI生成 @@ -233,7 +233,7 @@ cd web && pnpm install && pnpm build cd .. && cargo tauri dev ``` -> ⚠️ **現在の状態**: 初期設計段階。アーキテクチャ、ロードマップ、モジュール移植マップは完了。コード実装中。 +> ⚠️ **現在の状態**: Rustワークスペース、Tauriシェル、React UI、MCP/Agent層、主要ドキュメントは揃っており、ROADMAP は残るギャップと高リスク項目を追跡しています。 --- @@ -241,7 +241,7 @@ cd .. && cargo tauri dev | バージョン | 日付 | マイルストーン | |:--|:--|:--| -| `0.1.0-dev` | 2026-06 | Phase 0+1: Cargo workspace + Domain models + Edit ops | +| `0.1.0-dev` | 2026-06 | 開発中スナップショット: ワークスペース、主要 crate、Tauri シェル、React UI、MCP/Agent 層 | | *(planned)* `1.0.0` | TBD | Phase 10: フルリリース | 📖 [完全なロードマップ](docs/ROADMAP.md) diff --git a/README.md b/README.md index 6b959d2..b3e1645 100644 --- a/README.md +++ b/README.md @@ -112,11 +112,11 @@ Built-in Agent chat panel shares tool definitions and system prompt with MCP. | Capability | Technology | |:--|:--| -| Codec | FFmpeg (`ffmpeg-next`) — battle-tested Rust bindings | +| Codec | ffmpeg-sidecar (system ffmpeg/ffprobe) | | Compositor | wgpu custom compositor — multi-track layering + per-frame property sampling + affine/crop/blend | | Audio Playback | cpal | | Transcription | whisper-rs (word/segment timestamps) | -| Semantic Search | candle / ort + SigLIP2 dual-encoder | +| Semantic Search | ort + SigLIP2 + tokenizers | ### 🌐 BYOK AI Generation @@ -273,7 +273,7 @@ cd web && pnpm install && pnpm build cd .. && cargo tauri dev ``` -> ⚠️ **Current Status**: Early design phase. Architecture, roadmap, and module port maps are complete; code implementation in progress. +> ⚠️ **Current Status**: Active implementation. The Rust workspace, Tauri shell, React UI, MCP/Agent layer, and core docs are in place; the roadmap now tracks the remaining parity gaps and hard blockers. The sibling directory `palmier-pro-upstream/` contains upstream Swift sources for reference during porting. @@ -283,7 +283,7 @@ The sibling directory `palmier-pro-upstream/` contains upstream Swift sources fo | Version | Date | Milestone | |:--|:--|:--| -| `0.1.0-dev` | 2026-06 | Phase 0+1: Cargo workspace + Domain models + Edit ops + Tauri scaffold | +| `0.1.0-dev` | 2026-06 | Active development snapshot: workspace, core crates, Tauri shell, React UI, and MCP/Agent surface | | *(planned)* `0.2.0` | TBD | Phase 2: Persistence + Media import + Thumbnails + Waveform | | *(planned)* `0.3.0` | TBD | Phase 3: Timeline UI + Preview + MCP Server | | *(planned)* `0.4.0` | TBD | Phase 4: GPU Compositor (wgpu) + Text rasterization | diff --git a/README.zh-CN.md b/README.zh-CN.md index 28cecbd..cd0cbb1 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -100,11 +100,11 @@ Agent 操作时间线时，每次工具返回附带 `context_signal`： | 能力 | 技术 | |:--|:--| -| 编解码 | FFmpeg (`ffmpeg-next`) — 成熟 Rust 绑定 | +| 编解码 | ffmpeg-sidecar（调用系统 ffmpeg/ffprobe） | | 帧合成 | wgpu 自写合成器 — 多轨叠加 + 逐帧属性采样 + 仿射/裁剪/混合 | | 音频播放 | cpal | | 语音转写 | whisper-rs (word/segment 时间戳) | -| 语义搜索 | candle / ort + SigLIP2 图文双编码器 | +| 语义搜索 | ort + SigLIP2 + tokenizers | ### 🌐 BYOK 生成式 AI @@ -251,7 +251,7 @@ cd web && pnpm install && pnpm build cd .. && cargo tauri dev ``` -> ⚠️ **当前状态**: 早期设计阶段。架构设计、路线图、模块移植地图已完成，代码正在落地中。 +> ⚠️ **当前状态**: 核心实现已落地，Rust 工作空间、Tauri 壳、React UI、MCP/Agent 层和核心文档都已就位；路线图现在主要追踪剩余的对齐差距和高风险阻塞点。 --- @@ -259,7 +259,7 @@ cd .. && cargo tauri dev | 版本 | 日期 | 里程碑 | |:--|:--|:--| -| `0.1.0-dev` | 2026-06 | Phase 0+1: Cargo workspace + Domain models + Edit ops + Tauri scaffold | +| `0.1.0-dev` | 2026-06 | 进行中的开发快照：工作空间、核心 crate、Tauri 壳、React UI 与 MCP/Agent 层 | | *(planned)* `0.2.0` | TBD | Phase 2: Persistence + Media import + Thumbnails + Waveform | | *(planned)* `0.3.0` | TBD | Phase 3: Timeline UI + Preview + MCP Server | | *(planned)* `0.4.0` | TBD | Phase 4: GPU Compositor (wgpu) + Text rasterization | diff --git a/crates/opentake-agent/src/mcp/dispatch.rs b/crates/opentake-agent/src/mcp/dispatch.rs index 450ad6e..02dfc09 100644 --- a/crates/opentake-agent/src/mcp/dispatch.rs +++ b/crates/opentake-agent/src/mcp/dispatch.rs @@ -437,6 +437,7 @@ impl Dispatcher { opacity: a.opacity, transform: a.transform.map(transform_from_arg), text_content: a.content.clone(), + ..Default::default() }; let res = self.apply(EditCommand::SetClipProperties { clip_ids, diff --git a/crates/opentake-media/src/lib.rs b/crates/opentake-media/src/lib.rs index a98448c..5f8b76c 100644 --- a/crates/opentake-media/src/lib.rs +++ b/crates/opentake-media/src/lib.rs @@ -164,6 +164,65 @@ impl MediaEngine { pub fn export_pause(&self) -> ExportPause { self.export_pause.clone() } + + /// Extract the audio track from `input` into `output` as a self-contained + /// audio file. The container/codec is picked from the output extension: + /// `.m4a` → AAC in MP4, `.mp3` → libmp3lame, `.wav` → PCM s16le. Video is + /// dropped (`-vn`). Streams the mux directly (input file → output file), + /// never holding the full audio in memory — suitable for long sources. + /// + /// Returns the output path on success. Errors bubble up as `MediaError::Ffmpeg` + /// when ffmpeg is missing, exits non-zero, or the extension is unsupported. + pub fn extract_audio(&self, input: &Path, output: &Path) -> Result { + extract_audio_file(input, output).map(|_| output.to_path_buf()) + } +} + +/// Run `ffmpeg -y -i -vn ` to mux the audio track +/// into a standalone file. Codec is selected by `output`'s extension so the +/// caller just picks a save-path filter in the native dialog and the right +/// encoder falls out. `-y` overwrites (the save dialog already confirmed). +fn extract_audio_file(input: &Path, output: &Path) -> Result<()> { + let codec_args: Vec<&str> = match output.extension().and_then(|e| e.to_str()) { + Some("m4a") | Some("m4r") | Some("aac") => vec!["-c:a", "aac", "-b:a", "192k"], + Some("mp3") => vec!["-c:a", "libmp3lame", "-b:a", "192k"], + Some("wav") => vec!["-c:a", "pcm_s16le"], + Some(ext) => { + return Err(MediaError::Ffmpeg(format!( + "unsupported audio extension: .{ext} (use m4a, mp3, or wav)" + ))); + } + None => { + return Err(MediaError::Ffmpeg( + "output path has no extension (use .m4a, .mp3, or .wav)".into(), + )); + } + }; + + let mut cmd = std::process::Command::new(ff::ffmpeg_path()); + cmd.arg("-y") + .arg("-i") + .arg(input) + .arg("-vn") + .args(&codec_args) + .arg(output); + + let out = cmd + .output() + .map_err(|e| MediaError::Ffmpeg(format!("ffmpeg spawn: {e}")))?; + if !out.status.success() { + let stderr = String::from_utf8_lossy(&out.stderr); + return Err(MediaError::Ffmpeg(format!( + "ffmpeg exited {}{}", + out.status, + if stderr.trim().is_empty() { + String::new() + } else { + format!(": {}", stderr.trim()) + } + ))); + } + Ok(()) } #[cfg(test)] diff --git a/crates/opentake-ops/src/command.rs b/crates/opentake-ops/src/command.rs index f9c75a9..dc65758 100644 --- a/crates/opentake-ops/src/command.rs +++ b/crates/opentake-ops/src/command.rs @@ -18,8 +18,6 @@ use std::collections::HashSet; -use opentake_domain::{ChromaKey, ClipType, ColorGrade, Effect, Mask, Timeline, Transform}; - use crate::editor_state::EditorState; use crate::engines::FrameRange; use crate::id::IdGen; @@ -28,7 +26,9 @@ use crate::ops::move_clips::ClipMove; use crate::ops::place::PlaceSpec; use crate::ops::ripple::RippleOutcome; use crate::ops::trim::TrimEdit; - +use opentake_domain::{ + ChromaKey, ClipType, ColorGrade, Crop, Effect, Interpolation, Mask, Timeline, Transform, +}; /// Why a command did not apply. #[derive(Clone, PartialEq, Eq, Debug)] pub enum EditError { @@ -37,7 +37,6 @@ pub enum EditError { /// A ripple edit was refused to preserve sync-lock alignment. Refused(String), } - impl std::fmt::Display for EditError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { match self { @@ -46,9 +45,7 @@ impl std::fmt::Display for EditError { } } } - impl std::error::Error for EditError {} - /// Outcome of a successfully-attempted command. 1:1 shape from `ARCHITECTURE.md §5`. #[derive(Clone, PartialEq, Eq, Debug)] pub struct EditResult { @@ -63,7 +60,6 @@ pub struct EditResult { /// Human-readable one-line summary. pub summary: String, } - /// One entry for [`EditCommand::AddClips`] / `InsertClips`. #[derive(Clone, Debug)] pub struct ClipEntry { @@ -78,7 +74,6 @@ pub struct ClipEntry { pub has_audio: bool, pub add_linked_audio: bool, } - impl ClipEntry { fn to_spec(&self) -> PlaceSpec { PlaceSpec { @@ -94,7 +89,6 @@ impl ClipEntry { } } } - /// One id + new-name pair for [`EditCommand::RenameMedia`] / /// [`EditCommand::RenameFolder`]. A single rename is a one-element vec, so the /// batch and single forms apply in the same undo group (1:1 with upstream's @@ -104,7 +98,6 @@ pub struct RenameEntry { pub id: String, pub name: String, } - /// A text overlay entry for [`EditCommand::AddTexts`]. The transform is supplied /// fully resolved (text measurement is a media/UI concern this leaf doesn't do). #[derive(Clone, Debug)] @@ -116,7 +109,6 @@ pub struct TextEntry { pub text_style: opentake_domain::TextStyle, pub transform: Transform, } - /// A single clip property assignment for [`EditCommand::SetClipProperties`]. /// `None` fields are left unchanged; setting a scalar clears the matching /// keyframe track (mirrors `applyPropertyChanges`). @@ -130,8 +122,21 @@ pub struct ClipProperties { pub opacity: Option, pub transform: Option, pub text_content: Option, + /// Per-clip crop insets (normalized 0–1). Setting this clears `crop_track`. + pub crop: Option, + /// Fade-in length in frames. Setting this clamps to the clip duration. + pub fade_in_frames: Option, + /// Fade-out length in frames. Setting this clamps to the clip duration. + pub fade_out_frames: Option, + /// Fade-in interpolation mode. + pub fade_in_interpolation: Option, + /// Fade-out interpolation mode. + pub fade_out_interpolation: Option, + /// Horizontal flip flag (writes to `transform.flip_horizontal`). + pub flip_horizontal: Option, + /// Vertical flip flag (writes to `transform.flip_vertical`). + pub flip_vertical: Option, } - /// Which keyframe track [`EditCommand::SetKeyframes`] targets. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum KeyframeProperty { @@ -142,7 +147,6 @@ pub enum KeyframeProperty { Scale, Crop, } - /// A keyframe payload for [`EditCommand::SetKeyframes`]. Exactly one variant is /// used per command, matching `property`. #[derive(Clone, Debug)] @@ -151,7 +155,6 @@ pub enum KeyframePayload { Pair(opentake_domain::KeyframeTrack), Crop(opentake_domain::KeyframeTrack), } - /// The unified editing command. Every editing surface routes through this. #[derive(Clone, Debug)] pub enum EditCommand { @@ -165,6 +168,16 @@ pub enum EditCommand { }, /// Move clips (expanded to linked partners by the caller) to new tracks/frames. MoveClips { moves: Vec }, + /// Deep-copy clips (Option/Alt-drag duplicate) to new positions. Each clip + /// is cloned with all its fields (keyframe tracks / grade / chroma / masks / + /// effects / text / transform / crop / fades), gets a fresh id, is shifted + /// by `offset_frames`, lands on `target_track_indexes[i]`, and has its + /// `link_group_id` cleared (a copy is not linked to the original's group). + DuplicateClips { + clip_ids: Vec, + offset_frames: i32, + target_track_indexes: Vec, + }, /// Remove clips (expanded to linked partners), pruning emptied tracks. RemoveClips { clip_ids: Vec }, /// Split a clip at a frame (splits linked partners too). @@ -281,12 +294,25 @@ pub enum EditCommand { /// Delete folders recursively (subfolders + their assets) and cascade-remove /// clips referencing any deleted asset. DeleteFolder { folder_ids: Vec }, + /// Replace a clip's `media_ref` in place, preserving all editing attributes + /// (transform / crop / keyframe tracks / grade / masks / effects / fade). + /// When the new media is shorter than the clip's current duration, the + /// duration is truncated and `trim_end_frame` is clamped to fit. Optional + /// fields override the inferred defaults; `media_type` (when set) also + /// implies `source_clip_type` unless `source_clip_type` is explicitly given. + SwapMedia { + clip_id: String, + media_ref: String, + media_type: Option, + source_clip_type: Option, + duration_frames: Option, + trim_start_frame: Option, + }, /// Undo the last committed command. Undo, /// Redo the last undone command. Redo, } - /// Apply `command` to `state`, minting any new ids from `ids`. See the module /// docs for the transaction model. pub fn apply( @@ -323,7 +349,6 @@ pub fn apply( }, )) } - EditCommand::AddClips { entries } => add_clips(state, entries, ids), EditCommand::InsertClips { track_index, @@ -331,6 +356,11 @@ pub fn apply( entries, } => insert_clips(state, track_index, at_frame, entries, ids), EditCommand::MoveClips { moves } => move_clips(state, moves, ids), + EditCommand::DuplicateClips { + clip_ids, + offset_frames, + target_track_indexes, + } => duplicate_clips_cmd(state, clip_ids, offset_frames, target_track_indexes, ids), EditCommand::RemoveClips { clip_ids } => remove_clips(state, clip_ids), EditCommand::SplitClip { clip_id, at_frame } => split(state, clip_id, at_frame, ids), EditCommand::TrimClips { edits } => trim(state, edits), @@ -400,11 +430,25 @@ pub fn apply( EditCommand::RenameFolder { entries } => rename_folder(state, entries), EditCommand::DeleteMedia { asset_ids } => delete_media(state, asset_ids), EditCommand::DeleteFolder { folder_ids } => delete_folder(state, folder_ids), + EditCommand::SwapMedia { + clip_id, + media_ref, + media_type, + source_clip_type, + duration_frames, + trim_start_frame, + } => swap_media( + state, + clip_id, + media_ref, + media_type, + source_clip_type, + duration_frames, + trim_start_frame, + ), } } - // MARK: - Transaction helper - /// Run `work` inside a transaction: snapshot, mutate, commit-if-changed. `work` /// returns the affected clip ids on success. Validation/refusal errors propagate /// without committing. @@ -424,7 +468,6 @@ fn transact( let summary = summarize(&affected); Ok(result(state, changed, action_name, affected, &summary)) } - fn result( state: &EditorState, changed: bool, @@ -440,9 +483,7 @@ fn result( summary: summary.to_string(), } } - // MARK: - Command implementations - fn add_clips( state: &mut EditorState, entries: Vec, @@ -490,7 +531,6 @@ fn add_clips( }, ) } - fn insert_track_cmd( state: &mut EditorState, kind: ClipType, @@ -509,7 +549,6 @@ fn insert_track_cmd( }, ) } - fn set_track_props( state: &mut EditorState, track_index: usize, @@ -541,7 +580,6 @@ fn set_track_props( }, ) } - fn insert_clips( state: &mut EditorState, track_index: usize, @@ -599,7 +637,6 @@ fn insert_clips( }, ) } - fn move_clips( state: &mut EditorState, moves: Vec, @@ -624,7 +661,53 @@ fn move_clips( }, ) } - +/// Option/Alt-drag duplicate: deep-copy each clip to a new position. See +/// [`EditCommand::DuplicateClips`]. +fn duplicate_clips_cmd( + state: &mut EditorState, + clip_ids: Vec, + offset_frames: i32, + target_track_indexes: Vec, + ids: &dyn IdGen, +) -> Result { + if clip_ids.is_empty() { + return Err(EditError::Invalid( + "Missing or empty 'clipIds' array".into(), + )); + } + if target_track_indexes.len() != clip_ids.len() { + return Err(EditError::Invalid(format!( + "targetTrackIndexes length ({}) must match clipIds length ({})", + target_track_indexes.len(), + clip_ids.len() + ))); + } + for id in &clip_ids { + if state.find_clip(id).is_none() { + return Err(EditError::Invalid(format!("Clip not found: {id}"))); + } + } + let action_name = if clip_ids.len() == 1 { + "Duplicate Clip" + } else { + "Duplicate Clips" + }; + let n = clip_ids.len(); + transact( + state, + action_name, + move |_| format!("Duplicated {n} clip(s)"), + |st| { + Ok(ops::duplicate_clips( + &mut st.timeline, + &clip_ids, + offset_frames, + &target_track_indexes, + ids, + )) + }, + ) +} fn remove_clips(state: &mut EditorState, clip_ids: Vec) -> Result { if clip_ids.is_empty() { return Err(EditError::Invalid( @@ -661,7 +744,6 @@ fn remove_clips(state: &mut EditorState, clip_ids: Vec) -> Result) -> Result { if edits.is_empty() { return Err(EditError::Invalid("Missing or empty trim edits".into())); @@ -714,7 +795,6 @@ fn trim(state: &mut EditorState, edits: Vec) -> Result, @@ -783,7 +863,6 @@ fn set_clip_properties( }, ) } - /// Apply a property bundle to one clip in place. `partner` marks the call as a /// linked-partner propagation (only timing fields are set then). 1:1 port of /// `applyPropertyChanges`. @@ -797,7 +876,6 @@ fn apply_property_changes( return; }; let clip = &mut timeline.tracks[ti].clips[ci]; - if let Some(v) = props.duration_frames { clip.duration_frames = v; clip.clamp_keyframes_to_duration(); @@ -832,11 +910,34 @@ fn apply_property_changes( if let Some(t) = props.transform { clip.transform = t; } + if let Some(c) = props.crop { + clip.crop = c; + clip.crop_track = None; + } + if let Some(v) = props.fade_in_frames { + clip.fade_in_frames = v.max(0); + clip.clamp_fades_to_duration(); + } + if let Some(v) = props.fade_out_frames { + clip.fade_out_frames = v.max(0); + clip.clamp_fades_to_duration(); + } + if let Some(i) = props.fade_in_interpolation { + clip.fade_in_interpolation = i; + } + if let Some(i) = props.fade_out_interpolation { + clip.fade_out_interpolation = i; + } + if let Some(f) = props.flip_horizontal { + clip.transform.flip_horizontal = f; + } + if let Some(f) = props.flip_vertical { + clip.transform.flip_vertical = f; + } if let Some(c) = &props.text_content { clip.text_content = Some(c.clone()); } } - fn set_keyframes( state: &mut EditorState, clip_id: String, @@ -894,7 +995,6 @@ fn set_keyframes( }, ) } - fn stamp_keyframe( state: &mut EditorState, clip_id: String, @@ -969,7 +1069,6 @@ fn stamp_keyframe( }, ) } - fn remove_keyframe( state: &mut EditorState, clip_id: String, @@ -1045,7 +1144,6 @@ fn remove_keyframe( }, ) } - fn move_keyframe( state: &mut EditorState, clip_id: String, @@ -1147,7 +1245,6 @@ fn move_keyframe( }, ) } - fn set_keyframe_interpolation( state: &mut EditorState, clip_id: String, @@ -1202,7 +1299,6 @@ fn set_keyframe_interpolation( }, ) } - // MARK: - Advanced pixel-effect commands (A-tier) // // These set per-clip visual fields (color grade / chroma key / masks / effects). @@ -1210,7 +1306,6 @@ fn set_keyframe_interpolation( // do NOT propagate to linked partners. Each validates its `clip_ids` and runs // inside the shared `withTimelineSwap` transaction (snapshot -> mutate -> // commit-if-changed + version bump), so undo/redo and versioning come for free. - /// Validate that `clip_ids` is non-empty and every id resolves, then run `mutate` /// for each clip inside one transaction. Shared by the four effect setters. fn set_clip_effect_field( @@ -1244,7 +1339,6 @@ fn set_clip_effect_field( }, ) } - fn set_color_grade( state: &mut EditorState, clip_ids: Vec, @@ -1254,7 +1348,6 @@ fn set_color_grade( clip.color_grade = grade; }) } - fn set_chroma_key( state: &mut EditorState, clip_ids: Vec, @@ -1264,7 +1357,6 @@ fn set_chroma_key( clip.chroma_key = chroma_key; }) } - fn set_masks( state: &mut EditorState, clip_ids: Vec, @@ -1274,7 +1366,6 @@ fn set_masks( clip.masks = masks.clone(); }) } - fn set_effects( state: &mut EditorState, clip_ids: Vec, @@ -1284,7 +1375,6 @@ fn set_effects( clip.effects = effects.clone(); }) } - fn ripple_delete_ranges( state: &mut EditorState, track_index: usize, @@ -1334,7 +1424,6 @@ fn ripple_delete_ranges( } } } - /// Ripple-delete the selected clips (and their link groups), closing the gaps /// and shifting sync-locked followers. Refuses (no mutation) if a follower would /// collide. 1:1 with upstream `rippleDeleteSelectedClips`. @@ -1378,7 +1467,6 @@ fn ripple_delete_clips( } } } - fn add_texts( state: &mut EditorState, entries: Vec, @@ -1459,7 +1547,6 @@ fn add_texts( }, ) } - fn link( state: &mut EditorState, clip_ids: Vec, @@ -1491,7 +1578,6 @@ fn link( }, ) } - fn unlink(state: &mut EditorState, clip_ids: Vec) -> Result { if clip_ids.is_empty() { return Err(EditError::Invalid( @@ -1515,7 +1601,6 @@ fn unlink(state: &mut EditorState, clip_ids: Vec) -> Result, @@ -1555,7 +1640,6 @@ fn remove_tracks( }, ) } - fn create_folder( state: &mut EditorState, name: String, @@ -1584,7 +1668,6 @@ fn create_folder( }, ) } - fn move_to_folder( state: &mut EditorState, asset_ids: Vec, @@ -1608,7 +1691,6 @@ fn move_to_folder( }, ) } - fn rename_media( state: &mut EditorState, entries: Vec, @@ -1649,7 +1731,6 @@ fn rename_media( }, ) } - fn rename_folder( state: &mut EditorState, entries: Vec, @@ -1686,7 +1767,6 @@ fn rename_folder( }, ) } - fn delete_media(state: &mut EditorState, asset_ids: Vec) -> Result { if asset_ids.is_empty() { return Err(EditError::Invalid("assetIds is required".into())); @@ -1712,7 +1792,6 @@ fn delete_media(state: &mut EditorState, asset_ids: Vec) -> Result, @@ -1742,8 +1821,122 @@ fn delete_folder( ) } -// MARK: - Small local helpers +/// Replace a clip's `media_ref` in place, preserving every editing attribute +/// (transform / crop / keyframe tracks / grade / masks / effects / fade / text). +/// The new `media_ref` must exist in the manifest; when the new media is shorter +/// than the clip's current duration, the duration is truncated and +/// `trim_end_frame` is clamped so the source span fits. `media_type`, when set, +/// also implies `source_clip_type` unless `source_clip_type` is explicitly given +/// (matches the spec's "sync media_type" scenario). +fn swap_media( + state: &mut EditorState, + clip_id: String, + media_ref: String, + media_type: Option, + source_clip_type: Option, + duration_frames: Option, + trim_start_frame: Option, +) -> Result { + // 1. Validate clip exists. + let loc = state + .find_clip(&clip_id) + .ok_or_else(|| EditError::Invalid(format!("Clip not found: {clip_id}")))?; + + // 2. Validate media_ref exists in manifest; read its duration (seconds). + let new_asset = state + .manifest + .entries + .iter() + .find(|e| e.id == media_ref) + .ok_or_else(|| EditError::Invalid(format!("media not found: {media_ref}")))?; + + // 3. Convert new media duration (seconds) -> frames using the timeline fps. + let fps = state.timeline.fps; + let new_media_duration_frames = ((new_asset.duration * fps as f64).round() as i32).max(1); + // 4. Snapshot the current clip's timing fields for validation. + let clip = &state.timeline.tracks[loc.track_index].clips[loc.clip_index]; + let current_duration = clip.duration_frames; + let current_trim_start = clip.trim_start_frame; + let current_trim_end = clip.trim_end_frame; + let speed = clip.speed; + + // 5. Validate an explicitly-provided duration. + if let Some(d) = duration_frames { + if d < 1 { + return Err(EditError::Invalid(format!( + "durationFrames must be >= 1 (got {d})" + ))); + } + } + + // 6. Resolve final trim_start (explicit override or current value). + let final_trim_start = trim_start_frame.unwrap_or(current_trim_start); + if final_trim_start < 0 { + return Err(EditError::Invalid(format!( + "trimStartFrame must be >= 0 (got {final_trim_start})" + ))); + } + if final_trim_start >= new_media_duration_frames { + return Err(EditError::Invalid(format!( + "trimStartFrame {final_trim_start} must be < new media duration ({new_media_duration_frames})" + ))); + } + + // 7. Determine final duration: explicit > truncate-to-fit > keep current. + let final_duration = if let Some(d) = duration_frames { + d + } else if new_media_duration_frames < current_duration { + // New media is shorter: fit the clip into the available source span. + let available = (new_media_duration_frames - final_trim_start).max(1); + ((available as f64 / speed.max(0.0001)).round() as i32).max(1) + } else { + current_duration + }; + + // 8. Clamp trim_end so the total source span fits the new media. + let consumed = (final_duration as f64 * speed).round() as i32; + let max_trim_end = (new_media_duration_frames - final_trim_start - consumed).max(0); + let final_trim_end = current_trim_end.min(max_trim_end); + + // 9. media_type implies source_clip_type when the latter is not explicit. + let final_media_type = media_type; + let final_source_clip_type = source_clip_type.or(media_type); + + let summary_media_ref = media_ref.clone(); + let summary_clip_id = clip_id.clone(); + transact( + state, + "Swap Media", + move |_| { + format!( + "Swapped media on {} to {}", + summary_clip_id, summary_media_ref + ) + }, + move |st| { + let loc = st.find_clip(&clip_id).expect("validated above"); + let clip = &mut st.timeline.tracks[loc.track_index].clips[loc.clip_index]; + clip.media_ref = media_ref.clone(); + if let Some(mt) = final_media_type { + clip.media_type = mt; + } + if let Some(sct) = final_source_clip_type { + clip.source_clip_type = sct; + } + if clip.duration_frames != final_duration { + clip.duration_frames = final_duration; + clip.clamp_keyframes_to_duration(); + clip.clamp_fades_to_duration(); + } + clip.trim_start_frame = final_trim_start; + clip.trim_end_frame = final_trim_end; + Ok(vec![clip_id.clone()]) + }, + ) +} + +// MARK: - Small local helpers fn validate_entry(state: &EditorState, e: &ClipEntry, i: usize) -> Result<(), EditError> { if e.track_index >= state.timeline.tracks.len() { return Err(EditError::Invalid(format!( @@ -1785,7 +1978,6 @@ fn validate_entry(state: &EditorState, e: &ClipEntry, i: usize) -> Result<(), Ed } Ok(()) } - fn empty_to_none( track: opentake_domain::KeyframeTrack, ) -> Option> { @@ -1795,14 +1987,12 @@ fn empty_to_none( Some(track) } } - fn has_keyframe_at(t_opt: &Option>, rel: i32) -> bool { t_opt .as_ref() .map(|t| t.keyframes.iter().any(|k| k.frame == rel)) .unwrap_or(false) } - fn set_kf_interp( t_opt: &mut Option>, rel: i32, @@ -1816,13 +2006,11 @@ fn set_kf_interp( } } } - fn loc_clip_id(state: &EditorState, loc: opentake_domain::ClipLocation) -> String { state.timeline.tracks[loc.track_index].clips[loc.clip_index] .id .clone() } - fn find(timeline: &Timeline, clip_id: &str) -> Option<(usize, usize)> { for (ti, t) in timeline.tracks.iter().enumerate() { if let Some(ci) = t.clips.iter().position(|c| c.id == clip_id) { @@ -1831,7 +2019,6 @@ fn find(timeline: &Timeline, clip_id: &str) -> Option<(usize, usize)> { } None } - /// "V1" / "A1" / "I1" style track label. 1:1 port of `timelineTrackDisplayLabel`. fn track_display_label(timeline: &Timeline, track_index: usize) -> String { if track_index >= timeline.tracks.len() { @@ -1862,13 +2049,11 @@ fn track_display_label(timeline: &Timeline, track_index: usize) -> String { } format!("{prefix}{n}") } - #[cfg(test)] mod insert_track_tests { use super::*; use crate::id::SeqIdGen; use opentake_domain::ClipType; - #[test] fn insert_track_on_empty_timeline_creates_compatible_track() { // The drop-onto-empty-timeline path: a brand-new project has no tracks, @@ -1876,7 +2061,6 @@ mod insert_track_tests { let mut state = EditorState::default(); let ids = SeqIdGen::default(); assert_eq!(state.timeline.tracks.len(), 0); - let res = apply( &mut state, EditCommand::InsertTrack { @@ -1888,7 +2072,6 @@ mod insert_track_tests { assert!(res.changed); assert_eq!(state.timeline.tracks.len(), 1); assert_eq!(state.timeline.tracks[0].kind, ClipType::Video); - // A subsequent audio track clamps below the video zone. apply( &mut state, @@ -1901,7 +2084,6 @@ mod insert_track_tests { assert_eq!(state.timeline.tracks.len(), 2); assert_eq!(state.timeline.tracks[1].kind, ClipType::Audio); } - #[test] fn set_track_props_toggles_only_given_fields() { let mut state = EditorState::default(); @@ -1932,7 +2114,6 @@ mod insert_track_tests { assert!(state.timeline.tracks[0].hidden); assert_eq!(state.timeline.tracks[0].sync_locked, prev_sync); } - #[test] fn set_track_props_out_of_range_errors() { let mut state = EditorState::default(); @@ -1950,7 +2131,173 @@ mod insert_track_tests { assert!(err.is_err()); } } +#[cfg(test)] +mod duplicate_clips_tests { + use super::*; + use crate::id::SeqIdGen; + use opentake_domain::{Clip, ClipType, Keyframe, KeyframeTrack, Track}; + + fn state_with_clip() -> EditorState { + let mut tl = Timeline::new(); + let mut t = Track::new("v1", ClipType::Video); + t.clips.push(Clip::new("c1", "asset", 0, 30)); + tl.tracks.push(t); + EditorState::from_timeline(tl) + } + + #[test] + fn duplicate_clips_creates_copy_with_new_id() { + let mut state = state_with_clip(); + let ids = SeqIdGen::new("d-"); + let res = apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec!["c1".into()], + offset_frames: 100, + target_track_indexes: vec![0], + }, + &ids, + ) + .unwrap(); + assert!(res.changed); + assert_eq!(res.action_name, "Duplicate Clip"); + assert_eq!(res.affected_clip_ids.len(), 1); + // Original retained, copy present at frame 100. + assert!(state.timeline.tracks[0].clips.iter().any(|c| c.id == "c1")); + let copy = state.timeline.tracks[0] + .clips + .iter() + .find(|c| c.id == res.affected_clip_ids[0]) + .unwrap(); + assert_eq!(copy.start_frame, 100); + assert_ne!(copy.id, "c1"); + } + + #[test] + fn duplicate_clips_deep_copies_keyframe_tracks() { + let mut state = state_with_clip(); + state.timeline.tracks[0].clips[0].opacity_track = + Some(KeyframeTrack::from_keyframes(vec![ + Keyframe::new(0, 0.0), + Keyframe::new(30, 1.0), + ])); + let ids = SeqIdGen::default(); + let res = apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec!["c1".into()], + offset_frames: 100, + target_track_indexes: vec![0], + }, + &ids, + ) + .unwrap(); + let copy = state.timeline.tracks[0] + .clips + .iter() + .find(|c| c.id == res.affected_clip_ids[0]) + .unwrap(); + let op = copy.opacity_track.as_ref().unwrap(); + assert_eq!( + op.keyframes.iter().map(|k| k.frame).collect::>(), + vec![0, 30] + ); + } + + #[test] + fn duplicate_clips_clears_link_group_id() { + let mut state = state_with_clip(); + state.timeline.tracks[0].clips[0].link_group_id = Some("grp".into()); + let ids = SeqIdGen::default(); + let res = apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec!["c1".into()], + offset_frames: 50, + target_track_indexes: vec![0], + }, + &ids, + ) + .unwrap(); + let copy = state.timeline.tracks[0] + .clips + .iter() + .find(|c| c.id == res.affected_clip_ids[0]) + .unwrap(); + assert!(copy.link_group_id.is_none()); + } + + #[test] + fn duplicate_clips_missing_clip_errors() { + let mut state = state_with_clip(); + let ids = SeqIdGen::default(); + let err = apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec!["nope".into()], + offset_frames: 100, + target_track_indexes: vec![0], + }, + &ids, + ); + assert!(matches!(err, Err(EditError::Invalid(_)))); + } + #[test] + fn duplicate_clips_length_mismatch_errors() { + let mut state = state_with_clip(); + let ids = SeqIdGen::default(); + let err = apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec!["c1".into()], + offset_frames: 100, + target_track_indexes: vec![0, 1], // wrong length + }, + &ids, + ); + assert!(matches!(err, Err(EditError::Invalid(_)))); + } + + #[test] + fn duplicate_clips_empty_ids_errors() { + let mut state = state_with_clip(); + let ids = SeqIdGen::default(); + let err = apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec![], + offset_frames: 100, + target_track_indexes: vec![], + }, + &ids, + ); + assert!(matches!(err, Err(EditError::Invalid(_)))); + } + + #[test] + fn duplicate_clips_is_undoable() { + let mut state = state_with_clip(); + let ids = SeqIdGen::default(); + let version_before = state.version(); + apply( + &mut state, + EditCommand::DuplicateClips { + clip_ids: vec!["c1".into()], + offset_frames: 100, + target_track_indexes: vec![0], + }, + &ids, + ) + .unwrap(); + assert_eq!(state.timeline.tracks[0].clips.len(), 2); + assert!(state.can_undo()); + apply(&mut state, EditCommand::Undo, &ids).unwrap(); + assert_eq!(state.timeline.tracks[0].clips.len(), 1); + assert_eq!(state.timeline.tracks[0].clips[0].id, "c1"); + assert_eq!(state.version(), version_before + 2); // commit + undo + } +} #[cfg(test)] mod keyframe_edit_tests { use super::*; diff --git a/crates/opentake-ops/src/ops/duplicate.rs b/crates/opentake-ops/src/ops/duplicate.rs new file mode 100644 index 0000000..c00738a --- /dev/null +++ b/crates/opentake-ops/src/ops/duplicate.rs @@ -0,0 +1,424 @@ +//! Clip duplication (Option/Alt-drag copy). Deep-clips each source clip — +//! including all keyframe tracks / grade / chroma / masks / effects / text / +//! transform / crop / fades — mints a fresh id, shifts `start_frame` by +//! `offset_frames`, places it on `target_track_indexes[i]`, and clears the +//! link group (a duplicate is its own clip, not part of the original's link +//! group). The destination range is cleared overwrite-style first (mirrors +//! `move_clips`), so a duplicate landing on an existing clip overwrites it. +//! +//! Companion to [`crate::ops::move_clips`]: same destination-clearing + +//! pin-by-id + sort + prune flow, but the source clip stays put and a deep +//! copy is dropped at the target. + +use opentake_domain::{Clip, Timeline}; + +use crate::id::IdGen; +use crate::ops::clear_region::clear_region; +use crate::ops::place::sort_clips; +use crate::ops::tracks::prune_empty_tracks; + +/// Deep-copy each clip in `clip_ids` to a new position: `start_frame` shifted +/// by `offset_frames`, placed on `target_track_indexes[i]` (one target per +/// source, by index). Returns the ids of the newly created clips (in input +/// order). Missing clips or out-of-range / type-incompatible targets are +/// silently skipped (mirrors `move_clips`'s "guard ... continue"). +/// +/// Each duplicate: +/// - gets a fresh id from `ids`, +/// - keeps every field of the source (keyframe tracks, grade, chroma, masks, +/// effects, text, transform, crop, fades — `Clip: Clone` is a deep copy), +/// - has its `link_group_id` cleared (the copy is not linked to the original's +/// partners), +/// - has `start_frame = source.start_frame + offset_frames` (clamped `>= 0`). +pub fn duplicate_clips( + timeline: &mut Timeline, + clip_ids: &[String], + offset_frames: i32, + target_track_indexes: &[usize], + ids: &dyn IdGen, +) -> Vec { + if clip_ids.is_empty() { + return Vec::new(); + } + + // Resolve each source clip + validate its target track. Collect up front so + // the mutation phase can pin tracks by id (pruning could shift indices). + struct Plan { + clone: Clip, + to_track_id: String, + to_frame: i32, + } + let mut plans: Vec = Vec::new(); + for (i, id) in clip_ids.iter().enumerate() { + let Some((ti, ci)) = find(timeline, id) else { + continue; + }; + let Some(&to_track) = target_track_indexes.get(i) else { + continue; + }; + if to_track >= timeline.tracks.len() { + continue; + } + let src_type = timeline.tracks[ti].kind; + let dest_type = timeline.tracks[to_track].kind; + if !dest_type.is_compatible(src_type) { + continue; + } + let clone = timeline.tracks[ti].clips[ci].clone(); + let to_frame = (clone.start_frame + offset_frames).max(0); + plans.push(Plan { + clone, + to_track_id: timeline.tracks[to_track].id.clone(), + to_frame, + }); + } + if plans.is_empty() { + return Vec::new(); + } + + // Clear each destination range (pin by track id) so the duplicate overwrites + // whatever was there, exactly like `move_clips` / `place_clip` do. + for plan in &plans { + if let Some(idx) = timeline + .tracks + .iter() + .position(|t| t.id == plan.to_track_id) + { + clear_region( + timeline, + idx, + plan.to_frame, + plan.to_frame + plan.clone.duration_frames, + false, + ids, + ); + } + } + + // Drop each deep copy at its target frame with a fresh id + no link group. + let mut created = Vec::new(); + for plan in plans { + if let Some(idx) = timeline + .tracks + .iter() + .position(|t| t.id == plan.to_track_id) + { + let mut clip = plan.clone; + clip.id = ids.next_id(); + clip.start_frame = plan.to_frame; + clip.link_group_id = None; + created.push(clip.id.clone()); + timeline.tracks[idx].clips.push(clip); + sort_clips(&mut timeline.tracks[idx]); + } + } + prune_empty_tracks(timeline); + created +} + +fn find(timeline: &Timeline, clip_id: &str) -> Option<(usize, usize)> { + for (ti, t) in timeline.tracks.iter().enumerate() { + if let Some(ci) = t.clips.iter().position(|c| c.id == clip_id) { + return Some((ti, ci)); + } + } + None +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::id::SeqIdGen; + use opentake_domain::{ + ChromaKey, ClipType, ColorGrade, Crop, Effect, Interpolation, Keyframe, KeyframeTrack, + Mask, MaskShape, Point2, Track, + }; + + fn clip(id: &str, start: i32, dur: i32) -> Clip { + Clip::new(id, "asset", start, dur) + } + + fn two_video_tracks() -> Timeline { + let mut tl = Timeline::new(); + let mut v1 = Track::new("v1", ClipType::Video); + v1.clips.push(clip("a", 0, 30)); + let v2 = Track::new("v2", ClipType::Video); + tl.tracks.push(v1); + tl.tracks.push(v2); + tl + } + + #[test] + fn duplicate_keeps_original_and_creates_copy_at_offset() { + let mut tl = two_video_tracks(); + let g = SeqIdGen::new("d-"); + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[0], &g); + assert_eq!(created.len(), 1); + // Original stays put. + assert!(tl.tracks[0] + .clips + .iter() + .any(|c| c.id == "a" && c.start_frame == 0)); + // Copy lands at frame 100 on the same track with a fresh id. + let copy = tl.tracks[0].clips.iter().find(|c| c.id == "d-1").unwrap(); + assert_eq!(copy.start_frame, 100); + assert_eq!(copy.duration_frames, 30); + assert_eq!(copy.media_ref, "asset"); + } + + #[test] + fn duplicate_clears_link_group_id() { + let mut tl = two_video_tracks(); + // Mark the source as linked. + tl.tracks[0].clips[0].link_group_id = Some("grp".into()); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["a".into()], 50, &[0], &g); + let copy = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + assert!( + copy.link_group_id.is_none(), + "duplicate must not inherit link" + ); + // Original keeps its link group. + assert_eq!(tl.tracks[0].clips[0].link_group_id.as_deref(), Some("grp")); + } + + #[test] + fn duplicate_deep_copies_keyframe_tracks() { + let mut tl = two_video_tracks(); + // Give the source an opacity track + volume track with keyframes. + tl.tracks[0].clips[0].opacity_track = Some(KeyframeTrack::from_keyframes(vec![ + Keyframe::new(0, 0.0), + Keyframe::new(30, 1.0), + ])); + tl.tracks[0].clips[0].volume_track = Some(KeyframeTrack::from_keyframes(vec![ + Keyframe::new(0, -6.0), + Keyframe::new(30, 0.0), + ])); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[0], &g); + let copy = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + // Keyframe offsets are clip-relative, so they're identical to the source + // (the copy's start_frame moved, but offsets stay). + let op = copy.opacity_track.as_ref().unwrap(); + assert_eq!( + op.keyframes.iter().map(|k| k.frame).collect::>(), + vec![0, 30] + ); + let vol = copy.volume_track.as_ref().unwrap(); + assert_eq!( + vol.keyframes.iter().map(|k| k.value).collect::>(), + vec![-6.0, 0.0] + ); + // Mutating the copy's track must not touch the original (deep copy). + let copy_op = copy.opacity_track.as_ref().unwrap().clone(); + tl.tracks[0] + .clips + .iter_mut() + .find(|c| c.id == created[0]) + .unwrap() + .opacity_track = None; + assert!(tl.tracks[0].clips[0].opacity_track.is_some()); + assert_eq!( + tl.tracks[0].clips[0].opacity_track.as_ref().unwrap(), + ©_op + ); + } + + #[test] + fn duplicate_deep_copies_grade_masks_effects() { + let mut tl = two_video_tracks(); + let src = &mut tl.tracks[0].clips[0]; + src.color_grade = Some(ColorGrade { + exposure: 0.5, + ..Default::default() + }); + src.chroma_key = Some(ChromaKey::default()); + src.masks = vec![Mask { + shape: MaskShape::Circle { + center: Point2::new(0.5, 0.5), + radius: Point2::new(0.3, 0.3), + }, + feather: 0.05, + invert: false, + }]; + src.effects = vec![Effect::new("gaussianBlur").with_param("radius", 4.0)]; + let orig_color_grade = src.color_grade; + let orig_chroma_key = src.chroma_key; + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[0], &g); + let copy = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + assert_eq!(copy.color_grade, orig_color_grade); + assert_eq!(copy.chroma_key, orig_chroma_key); + assert_eq!(copy.masks.len(), 1); + assert_eq!(copy.effects.len(), 1); + // Mutate the copy's masks; the original must be unaffected (no shared ref). + let copy_masks = copy.masks.clone(); + tl.tracks[0] + .clips + .iter_mut() + .find(|c| c.id == created[0]) + .unwrap() + .masks + .clear(); + assert_eq!(tl.tracks[0].clips[0].masks, copy_masks); + } + + #[test] + fn duplicate_to_different_track_uses_target_index() { + let mut tl = two_video_tracks(); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[1], &g); + // Copy lands on v2 (index 1). + let copy = tl.tracks[1] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + assert_eq!(copy.start_frame, 100); + // Original still on v1. + assert!(tl.tracks[0].clips.iter().any(|c| c.id == "a")); + } + + #[test] + fn duplicate_multiple_clips_preserve_relative_spacing() { + let mut tl = Timeline::new(); + let mut v = Track::new("v", ClipType::Video); + v.clips.push(clip("a", 0, 30)); + v.clips.push(clip("b", 60, 30)); // 30-frame gap + tl.tracks.push(v); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["a".into(), "b".into()], 100, &[0, 0], &g); + assert_eq!(created.len(), 2); + let c0 = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + let c1 = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[1]) + .unwrap(); + // a@0 -> 100, b@60 -> 160; gap of 30 preserved. + assert_eq!(c0.start_frame, 100); + assert_eq!(c1.start_frame, 160); + } + + #[test] + fn duplicate_overwrites_blocking_clip_at_destination() { + let mut tl = two_video_tracks(); + // Place a blocker on v2 at [90,150); duplicating a to v2@100 overwrites the overlap. + tl.tracks[1].clips.push(clip("blocker", 90, 60)); + let g = SeqIdGen::new("r-"); + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[1], &g); + let v2 = tl.tracks.iter().find(|t| t.id == "v2").unwrap(); + let copy = v2.clips.iter().find(|c| c.id == created[0]).unwrap(); + assert_eq!((copy.start_frame, copy.end_frame()), (100, 130)); + // No clip other than the copy covers [100,130). + let covering = v2 + .clips + .iter() + .filter(|c| c.id != created[0] && c.start_frame < 130 && c.end_frame() > 100) + .count(); + assert_eq!(covering, 0); + } + + #[test] + fn duplicate_clamps_start_frame_to_zero() { + let mut tl = two_video_tracks(); + let g = SeqIdGen::default(); + // a starts at 0; offset -50 would put it at -50 -> clamped to 0. + let created = duplicate_clips(&mut tl, &["a".into()], -50, &[0], &g); + let copy = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + assert_eq!(copy.start_frame, 0); + } + + #[test] + fn duplicate_skips_missing_clip() { + let mut tl = two_video_tracks(); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["nope".into()], 100, &[0], &g); + assert!(created.is_empty()); + } + + #[test] + fn duplicate_skips_incompatible_target_track() { + let mut tl = Timeline::new(); + let mut v = Track::new("v", ClipType::Video); + v.clips.push(clip("a", 0, 30)); + let a = Track::new("a", ClipType::Audio); + tl.tracks.push(v); + tl.tracks.push(a); + let g = SeqIdGen::default(); + // Duplicating a video clip onto an audio track -> incompatible -> skipped. + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[1], &g); + assert!(created.is_empty()); + assert!(tl.tracks[1].clips.is_empty()); + } + + #[test] + fn duplicate_skips_out_of_range_target() { + let mut tl = two_video_tracks(); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["a".into()], 100, &[99], &g); + assert!(created.is_empty()); + } + + #[test] + fn duplicate_copies_text_and_transform_fields() { + let mut tl = Timeline::new(); + let mut t = Track::new("t", ClipType::Text); + let mut c = Clip::new("txt", "", 0, 30); + c.media_type = ClipType::Text; + c.source_clip_type = ClipType::Text; + c.text_content = Some("Hello".into()); + c.transform = opentake_domain::Transform::from_center( + opentake_domain::Point { x: 0.25, y: 0.75 }, + 0.5, + 0.5, + ); + c.crop = Crop { + left: 0.1, + top: 0.2, + right: 0.3, + bottom: 0.4, + }; + c.fade_in_frames = 5; + c.fade_in_interpolation = Interpolation::Smooth; + c.rotation_track = Some(KeyframeTrack::from_keyframes(vec![ + Keyframe::with_interpolation(0, 0.0, Interpolation::Linear), + Keyframe::new(10, 0.2), + ])); + t.clips.push(c); + tl.tracks.push(t); + let g = SeqIdGen::default(); + let created = duplicate_clips(&mut tl, &["txt".into()], 50, &[0], &g); + let copy = tl.tracks[0] + .clips + .iter() + .find(|c| c.id == created[0]) + .unwrap(); + assert_eq!(copy.text_content.as_deref(), Some("Hello")); + assert_eq!(copy.transform.center_x, 0.25); + assert_eq!(copy.crop.left, 0.1); + assert_eq!(copy.fade_in_frames, 5); + assert_eq!(copy.fade_in_interpolation, Interpolation::Smooth); + assert!(copy.rotation_track.is_some()); + } +} diff --git a/crates/opentake-ops/src/ops/mod.rs b/crates/opentake-ops/src/ops/mod.rs index dcaf88e..7ea074c 100644 --- a/crates/opentake-ops/src/ops/mod.rs +++ b/crates/opentake-ops/src/ops/mod.rs @@ -4,6 +4,7 @@ //! in place, and the command layer snapshots/commits around them. pub mod clear_region; +pub mod duplicate; pub mod folders; pub mod linking; pub mod move_clips; @@ -14,6 +15,7 @@ pub mod tracks; pub mod trim; pub use clear_region::clear_region; +pub use duplicate::duplicate_clips; pub use folders::{ create_folder, delete_folder, delete_media, move_to_folder, rename_folder, rename_media, }; diff --git a/crates/opentake-ops/tests/command_apply.rs b/crates/opentake-ops/tests/command_apply.rs index 91cf426..55cc588 100644 --- a/crates/opentake-ops/tests/command_apply.rs +++ b/crates/opentake-ops/tests/command_apply.rs @@ -5,7 +5,9 @@ use opentake_domain::{AnimPair, Interpolation, Keyframe, KeyframeTrack}; use opentake_domain::{ChromaKey, ColorGrade, Effect, Mask, MaskShape, Point2}; -use opentake_domain::{Clip, ClipType, MediaManifest, Timeline, Track, Transform}; +use opentake_domain::{ + Clip, ClipType, MediaManifest, MediaManifestEntry, MediaSource, Timeline, Track, Transform, +}; use opentake_ops::{ apply, ClipEntry, ClipMove, ClipProperties, EditCommand, EditError, EditorState, FrameRange, KeyframePayload, KeyframeProperty, SeqIdGen, TextEntry, @@ -519,6 +521,153 @@ fn set_clip_properties_scalar_clears_keyframe_track() { assert!(c.opacity_track.is_none()); // cleared by setting the scalar } +#[test] +fn set_clip_properties_crop_sets_and_clears_track() { + let mut st = state(vec![video_track("v", true, vec![clip("c", 0, 60)])]); + let g = SeqIdGen::default(); + // Pre-existing crop track should be cleared when a static crop is set. + let mut existing = st.timeline.tracks[0].clips[0].clone(); + existing.crop_track = Some(KeyframeTrack::from_keyframes(vec![Keyframe::new( + 0, + opentake_domain::Crop { + left: 0.1, + top: 0.0, + right: 0.0, + bottom: 0.0, + }, + )])); + st.timeline.tracks[0].clips[0] = existing; + + apply( + &mut st, + EditCommand::SetClipProperties { + clip_ids: vec!["c".into()], + properties: ClipProperties { + crop: Some(opentake_domain::Crop { + left: 0.2, + top: 0.1, + right: 0.0, + bottom: 0.0, + }), + ..Default::default() + }, + }, + &g, + ) + .unwrap(); + + let c = &st.timeline.tracks[0].clips[0]; + assert!((c.crop.left - 0.2).abs() < 1e-9); + assert!((c.crop.top - 0.1).abs() < 1e-9); + assert!(c.crop_track.is_none()); // cleared by setting the static value +} + +#[test] +fn set_clip_properties_fade_sets_frames_and_interpolation() { + let mut st = state(vec![video_track("v", true, vec![clip("c", 0, 60)])]); + let g = SeqIdGen::default(); + apply( + &mut st, + EditCommand::SetClipProperties { + clip_ids: vec!["c".into()], + properties: ClipProperties { + fade_in_frames: Some(10), + fade_out_frames: Some(15), + fade_in_interpolation: Some(Interpolation::Smooth), + fade_out_interpolation: Some(Interpolation::Hold), + ..Default::default() + }, + }, + &g, + ) + .unwrap(); + + let c = &st.timeline.tracks[0].clips[0]; + assert_eq!(c.fade_in_frames, 10); + assert_eq!(c.fade_out_frames, 15); + assert_eq!(c.fade_in_interpolation, Interpolation::Smooth); + assert_eq!(c.fade_out_interpolation, Interpolation::Hold); +} + +#[test] +fn set_clip_properties_fade_clamps_to_duration() { + let mut st = state(vec![video_track("v", true, vec![clip("c", 0, 30)])]); + let g = SeqIdGen::default(); + // fade_in 100 on a 30-frame clip should clamp to 30, fade_out to 0. + apply( + &mut st, + EditCommand::SetClipProperties { + clip_ids: vec!["c".into()], + properties: ClipProperties { + fade_in_frames: Some(100), + ..Default::default() + }, + }, + &g, + ) + .unwrap(); + let c = &st.timeline.tracks[0].clips[0]; + assert_eq!(c.fade_in_frames, 30); + assert_eq!(c.fade_out_frames, 0); +} + +#[test] +fn set_clip_properties_flip_writes_to_transform() { + let mut st = state(vec![video_track("v", true, vec![clip("c", 0, 60)])]); + let g = SeqIdGen::default(); + apply( + &mut st, + EditCommand::SetClipProperties { + clip_ids: vec!["c".into()], + properties: ClipProperties { + flip_horizontal: Some(true), + flip_vertical: Some(true), + ..Default::default() + }, + }, + &g, + ) + .unwrap(); + let c = &st.timeline.tracks[0].clips[0]; + assert!(c.transform.flip_horizontal); + assert!(c.transform.flip_vertical); +} + +#[test] +fn set_clip_properties_multiple_fields_at_once() { + let mut st = state(vec![video_track("v", true, vec![clip("c", 0, 60)])]); + let g = SeqIdGen::default(); + apply( + &mut st, + EditCommand::SetClipProperties { + clip_ids: vec!["c".into()], + properties: ClipProperties { + crop: Some(opentake_domain::Crop { + left: 0.1, + top: 0.2, + right: 0.3, + bottom: 0.4, + }), + fade_in_frames: Some(5), + fade_in_interpolation: Some(Interpolation::Smooth), + flip_horizontal: Some(true), + opacity: Some(0.8), + ..Default::default() + }, + }, + &g, + ) + .unwrap(); + let c = &st.timeline.tracks[0].clips[0]; + assert!((c.crop.left - 0.1).abs() < 1e-9); + assert!((c.crop.bottom - 0.4).abs() < 1e-9); + assert_eq!(c.fade_in_frames, 5); + assert_eq!(c.fade_in_interpolation, Interpolation::Smooth); + assert!(c.transform.flip_horizontal); + assert!((c.opacity - 0.8).abs() < 1e-9); + assert!(c.opacity_track.is_none()); // opacity scalar cleared its track +} + // ---- set_keyframes -------------------------------------------------------- #[test] @@ -1054,3 +1203,232 @@ fn ripple_delete_clips_rejects_unknown_clip() { )); assert_eq!(st.version(), 0); } + +// ---- swap_media ------------------------------------------------------------ + +/// Build a manifest entry with `duration` in seconds and an External source. +fn media_entry(id: &str, kind: ClipType, duration_secs: f64) -> MediaManifestEntry { + MediaManifestEntry { + id: id.into(), + name: id.into(), + kind, + source: MediaSource::External { + absolute_path: format!("/abs/{id}"), + }, + duration: duration_secs, + generation_input: None, + source_width: None, + source_height: None, + source_fps: None, + has_audio: None, + folder_id: None, + cached_remote_url: None, + cached_remote_url_expires_at: None, + } +} + +/// Build a state with the given tracks and manifest entries (fps defaults to 30). +fn state_with_media(tracks: Vec, entries: Vec) -> EditorState { + let mut tl = Timeline::new(); + tl.tracks = tracks; + let mut manifest = MediaManifest::new(); + manifest.entries = entries; + EditorState::new(tl, manifest) +} + +#[test] +fn swap_media_replaces_ref_and_preserves_attributes() { + // Clip duration 100 frames (fps=30 -> 100/30 secs). New media same length. + let mut c = clip("c", 0, 100); + c.opacity = 0.7; + c.transform = Transform { + center_x: 0.3, + center_y: 0.4, + width: 0.5, + height: 0.6, + rotation: 15.0, + flip_horizontal: true, + flip_vertical: false, + }; + let v = video_track("v", true, vec![c]); + let entries = vec![ + media_entry("old", ClipType::Video, 100.0 / 30.0), + media_entry("new", ClipType::Video, 100.0 / 30.0), + ]; + let mut st = state_with_media(vec![v], entries); + let g = SeqIdGen::default(); + + let res = apply( + &mut st, + EditCommand::SwapMedia { + clip_id: "c".into(), + media_ref: "new".into(), + media_type: None, + source_clip_type: None, + duration_frames: None, + trim_start_frame: None, + }, + &g, + ) + .unwrap(); + + assert!(res.changed); + assert_eq!(res.action_name, "Swap Media"); + assert_eq!(res.affected_clip_ids, vec!["c".to_string()]); + let clip = &st.timeline.tracks[0].clips[0]; + assert_eq!(clip.media_ref, "new"); + assert_eq!(clip.duration_frames, 100); // unchanged + // Preserved editing attributes + assert!((clip.opacity - 0.7).abs() < 1e-9); + assert!((clip.transform.center_x - 0.3).abs() < 1e-9); + assert!((clip.transform.rotation - 15.0).abs() < 1e-9); + assert!(clip.transform.flip_horizontal); +} + +#[test] +fn swap_media_truncates_when_new_media_shorter() { + // Clip duration 100 frames; new media is 50 frames -> truncate to 50. + let v = video_track("v", true, vec![clip("c", 0, 100)]); + let entries = vec![ + media_entry("old", ClipType::Video, 100.0 / 30.0), + media_entry("short", ClipType::Video, 50.0 / 30.0), + ]; + let mut st = state_with_media(vec![v], entries); + let g = SeqIdGen::default(); + + let res = apply( + &mut st, + EditCommand::SwapMedia { + clip_id: "c".into(), + media_ref: "short".into(), + media_type: None, + source_clip_type: None, + duration_frames: None, + trim_start_frame: None, + }, + &g, + ) + .unwrap(); + + assert!(res.changed); + let clip = &st.timeline.tracks[0].clips[0]; + assert_eq!(clip.media_ref, "short"); + assert_eq!(clip.duration_frames, 50); // truncated to new media length +} + +#[test] +fn swap_media_rejects_missing_media_ref() { + let v = video_track("v", true, vec![clip("c", 0, 100)]); + let entries = vec![media_entry("old", ClipType::Video, 100.0 / 30.0)]; + let mut st = state_with_media(vec![v], entries); + let g = SeqIdGen::default(); + + let err = apply( + &mut st, + EditCommand::SwapMedia { + clip_id: "c".into(), + media_ref: "nonexistent".into(), + media_type: None, + source_clip_type: None, + duration_frames: None, + trim_start_frame: None, + }, + &g, + ) + .unwrap_err(); + + assert!(matches!(err, EditError::Invalid(_))); + assert_eq!(st.version(), 0); // unchanged + // Original media_ref preserved. + assert_eq!(st.timeline.tracks[0].clips[0].media_ref, "asset"); +} + +#[test] +fn swap_media_syncs_media_type_and_source_clip_type() { + // Original clip is video; swap to an audio asset with mediaType=Audio. + let mut c = clip("c", 0, 100); + c.media_type = ClipType::Video; + c.source_clip_type = ClipType::Video; + let v = video_track("v", true, vec![c]); + let entries = vec![ + media_entry("old", ClipType::Video, 100.0 / 30.0), + media_entry("audio1", ClipType::Audio, 100.0 / 30.0), + ]; + let mut st = state_with_media(vec![v], entries); + let g = SeqIdGen::default(); + + apply( + &mut st, + EditCommand::SwapMedia { + clip_id: "c".into(), + media_ref: "audio1".into(), + media_type: Some(ClipType::Audio), + source_clip_type: None, + duration_frames: None, + trim_start_frame: None, + }, + &g, + ) + .unwrap(); + + let clip = &st.timeline.tracks[0].clips[0]; + assert_eq!(clip.media_ref, "audio1"); + assert_eq!(clip.media_type, ClipType::Audio); + assert_eq!(clip.source_clip_type, ClipType::Audio); // implied by media_type +} + +#[test] +fn swap_media_rejects_missing_clip() { + let v = video_track("v", true, vec![]); + let entries = vec![media_entry("new", ClipType::Video, 100.0 / 30.0)]; + let mut st = state_with_media(vec![v], entries); + let g = SeqIdGen::default(); + + let err = apply( + &mut st, + EditCommand::SwapMedia { + clip_id: "missing".into(), + media_ref: "new".into(), + media_type: None, + source_clip_type: None, + duration_frames: None, + trim_start_frame: None, + }, + &g, + ) + .unwrap_err(); + + assert!(matches!(err, EditError::Invalid(_))); + assert_eq!(st.version(), 0); +} + +#[test] +fn swap_media_is_undoable() { + let v = video_track("v", true, vec![clip("c", 0, 100)]); + let entries = vec![ + media_entry("old", ClipType::Video, 100.0 / 30.0), + media_entry("new", ClipType::Video, 100.0 / 30.0), + ]; + let mut st = state_with_media(vec![v], entries); + let g = SeqIdGen::default(); + + apply( + &mut st, + EditCommand::SwapMedia { + clip_id: "c".into(), + media_ref: "new".into(), + media_type: None, + source_clip_type: None, + duration_frames: None, + trim_start_frame: None, + }, + &g, + ) + .unwrap(); + assert_eq!(st.timeline.tracks[0].clips[0].media_ref, "new"); + assert!(st.can_undo()); + + // Undo via the command (undo() is pub(crate), so we route through apply). + apply(&mut st, EditCommand::Undo, &g).unwrap(); + assert_eq!(st.timeline.tracks[0].clips[0].media_ref, "asset"); // restored +} diff --git a/docs/ADVANCED-FEATURES.md b/docs/ADVANCED-FEATURES.md index eb4cd7e..52f3fec 100644 --- a/docs/ADVANCED-FEATURES.md +++ b/docs/ADVANCED-FEATURES.md @@ -10,7 +10,7 @@ OpenTake 补齐这些进阶能力,**几乎不需要新建基础设施**,全部 1. **wgpu 帧合成器(ARCHITECTURE §1 / ROADMAP Phase 3)** —— 原本是最大风险(命门),实为最大机会。 特效 / 转场 / 滤镜 / 调色 / 绿幕抠图 / 蒙版 本质都是「在合成片元着色器里对像素做数学变换」。上游被 AVFoundation 声明式合成锁死、**做不了**这些;OpenTake 自写 wgpu 合成器**天生适合**,这是反超 Palmier Pro、逼近剪映的核心窗口。 -2. **ort / candle 本地推理栈(已为 SigLIP2 语义搜索铺设,ARCHITECTURE §6)** —— AI 抠像 / 运动追踪 / 防抖 / 超分 / 消除 / 补帧 复用同一条 ONNX 推理通道(`opentake-media` 推理 worker),不必从零搭 ML 基础设施。 +2. **ort 本地推理栈(已为 SigLIP2 语义搜索铺设,ARCHITECTURE §6)** —— AI 抠像 / 运动追踪 / 防抖 / 超分 / 消除 / 补帧 复用同一条 ONNX 推理通道(`opentake-media` 推理 worker),不必从零搭 ML 基础设施。 外加三条轻量补充层:**纯逻辑(domain/ops)**、**FFmpeg 音频滤镜**、**外部生成 API(opentake-gen / BYOK)**。 @@ -40,15 +40,15 @@ OpenTake 补齐这些进阶能力,**几乎不需要新建基础设施**,全部 --- -## B 层 · ort/candle 本地 AI 推理(复用 SigLIP2 推理通道,`opentake-media` worker) +## B 层 · ort 本地 AI 推理(复用 SigLIP2 推理通道,`opentake-media` worker) | 能力 | 现状 | 难度 | 优先级 | 方案要点 | |---|---|---|---|---| -| 画质超清修复 super-res | partial(上游仅云端放大) | high | p1 | ort/candle 跑 Real-ESRGAN / SeedVR 本地;或保留 BYOK 云端双轨 | +| 画质超清修复 super-res | partial(上游仅云端放大) | high | p1 | ort 跑 Real-ESRGAN / SeedVR 本地;或保留 BYOK 云端双轨 | | AI 智能抠像 matting | missing | high | p2 | ort 跑 RVM / BiRefNet(逐帧 alpha)→ 喂合成器当 mask;无需纯色背景 | | 视频防抖 stabilization | missing | medium | p2 | 先 **FFmpeg vidstab**(零模型,快速可用);进阶用光流/特征 | | AI 运动追踪 motion tracking | missing | high | p2 | ort 跑 CoTracker/点追踪 → 输出轨迹关键帧,驱动 transform/mask 自适应形变 | -| 光流法智能补帧 | missing | blocker | p3 | ort/candle 跑 RIFE/FILM 做帧间插值;退化方案 Farneback 光流 warp 或帧混合。与曲线变速配合做丝滑慢动作 | +| 光流法智能补帧 | missing | blocker | p3 | ort 跑 RIFE/FILM 做帧间插值;退化方案 Farneback 光流 warp 或帧混合。与曲线变速配合做丝滑慢动作 | | AI 智能消除瑕疵 | missing | high | p3 | 视频 inpainting(ProPainter 等)本地重,初期建议走外部 API | 统一形态:`opentake-media` 提供 ONNX 推理 worker(GPU 加速,wgpu/CUDA/CoreML EP 按平台);AI 特性产出(alpha/轨迹/插帧/超分帧)交给 wgpu 合成器或写回媒体缓存(content-hash)。每项配 MCP 工具(`ai_matte` / `track_motion` / `stabilize` / `upscale`(已有)/ `interpolate_frames` / `remove_object`)。 diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d6cb93a..60d65e8 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -71,7 +71,7 @@ OpenTake/ │ ├── opentake-ops/ # 编辑算法:OverwriteEngine/RippleEngine/SnapEngine(纯函数)+ EditCommand::apply + UndoStack │ ├── opentake-project/ # .opentake 目录包读写(serde,#[serde(default)] 容错)+ 媒体清单/归档 │ ├── opentake-render/ # RenderPlan(Timeline→每帧合成指令,纯函数)+ wgpu 合成器 + ffmpeg 编解码后端 -│ ├── opentake-media/ # 解码/缩略图/波形/转写/语义搜索(ffmpeg-next/symphonia/whisper-rs/ort) +│ ├── opentake-media/ # 解码/缩略图/波形/转写/语义搜索(ffmpeg-sidecar/symphonia/whisper-rs/ort) │ ├── opentake-agent/ # 工具层(=ToolExecutor)+ MCP server(rmcp)+ 应用内 chat 客户端 + 短ID + 系统提示词 │ ├── opentake-gen/ # 生成后端客户端(BYOK 直连 / 自建代理),复刻 GenerationParams 联合类型 │ └── opentake-core/ # 组装:EditorState(持有 timeline+manifest)、command 路由、事件总线 @@ -134,14 +134,14 @@ struct EditResult { changed: bool, action_name: String, affected_clip_ids: Vec 更新说明：本文保留为早期 gap 评估与实现计划的历史文档。当前代码已经接通导入、时间线编辑、预览、导出 spine、全局素材库和 MCP 基础链路；现在的实现状态请以 README / CLAUDE / 代码为准。 + ## 〇、现状真话：现在能不能完成一次真实剪辑闭环？ -**结论：不能。一次也跑不通。** +**结论：这是早期 gap 评估文档；当时还不能一次跑通，现在已经有了可运行的核心闭环，但仍保留若干 1:1 对齐缺口。** 把"导入 → 预览 → 拖到时间线 → 剪 → 导出"逐段对照,OpenTake 当前是"内核很硬、四肢半残、神经没接线"的状态——底层算法层(Rust)复刻度高得惊人(ops 层 place/split/trim/move/clear_region/ripple 全部 1:1 端口,EditCommand 21 个变体,wgpu 合成器+shader+回读像素都已就绪,约 960+ 单测通过),但**凡是"用户能看见、能点、能拖"的边界粘合层几乎全是空的或假的**。 +以下分项保留为 backlog 记录；其中不少基础能力已经在后续迭代中落地，但为了保留历史决策链路，这份文档继续记录当时的差距判断。 + 逐环节体检： - **导入**：命令通路其实是通的(import_media/import_folder 已注册、emit MediaChanged、前端 refreshMedia 直刷)。但用户会觉得"点了没反应",根因有三:① 非 Tauri 环境 `openDialog()` 返回 null 后**静默 return**,毫无反馈;② ffprobe sidecar 没就绪时 probe 失败被吞成默认值,所有项 duration=0、缩略图本就是 None,网格里只有灰底图标,视觉上像没导入;③ 视频白名单只有 mov/mp4/m4v,mkv/webm/avi 被**静默丢弃**。而且 `MediaItemDto.thumbnail` 写死 `None`——**导入的视频根本没有缩略图**,尽管底层 `MediaEngine` 的缩略图能力早已实现,只是没接线。 @@ -196,4 +200,4 @@ 8. **新建落盘与自动保存必须配套设计。** `project_dir=None` 时 `save_project(None)` 报 NoProjectOpen,所以 P0-1(新建即落盘)是 P1-1(自动保存)的硬前置,二者同期落地。 -9. **全局可复用素材库(#37)是 OpenTake 新增子系统,不在本 1:1 差距清单约束内。** 上游 palmier-pro 无对应模块(grep 确认无全局收藏库)。采用 copy-on-favorite + SHA-256 内容寻址去重 + JSON manifest 原子写;后端存储层(#37-A / #104)+ Tauri 命令层(#37-B / #106)**已并入 main**,前端(#37-C / #56)待做。注意区分范畴:**#37 = 跨项目全局库;#49/#91 = 每项目媒体与文件夹浏览**,两者不同。 \ No newline at end of file +9. **全局可复用素材库(#37)是 OpenTake 新增子系统,不在本 1:1 差距清单约束内。** 上游 palmier-pro 无对应模块(grep 确认无全局收藏库)。采用 copy-on-favorite + SHA-256 内容寻址去重 + JSON manifest 原子写;后端存储层(#37-A / #104)+ Tauri 命令层(#37-B / #106)**已并入 main**,前端(#37-C / #56)待做。注意区分范畴:**#37 = 跨项目全局库;#49/#91 = 每项目媒体与文件夹浏览**,两者不同。 diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 02fb449..29655d3 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -63,7 +63,7 @@ - **进度**:`list_models` 工具已从存根接到 `opentake-gen` 内置静态 catalog(#111,`?type=` 过滤 + `{ models, loaded }`,纯本地无网络/BYOK);`generate_*`/`upscale_media` 仍待 async + ProviderRegistry + BYOK。 ## Phase 8 — 文字/字幕渲染 + 转写 + 语义搜索 -- **做**:cosmic-text + tiny-skia/Vello 文字渲染(阴影/描边/背景/对齐/换行,逐帧 opacity)接入合成器;whisper-rs 转写(word/segment 时间戳,`TranscriptionResult` 模型复用);candle/ort 跑 SigLIP2 + tokenizers 做视觉/口语搜索。 +- **做**:cosmic-text + tiny-skia/Vello 文字渲染(阴影/描边/背景/对齐/换行,逐帧 opacity)接入合成器;whisper-rs 转写(word/segment 时间戳,`TranscriptionResult` 模型复用);ort + SigLIP2 + tokenizers 做视觉/口语搜索。 - **验证**:字幕静态渲染像素对齐上游;转写时间码映射正确;`search_media` 视觉/口语命中合理。 - **进度**:SRT/VTT 字幕**导出纯逻辑**已落地(#110,`crates/opentake-domain/src/subtitle_export.rs`,按 `caption_group_id` 分组序列化,16 单测);剩接导出层 + `export_captions` agent 工具 + 前端导出对话框。 - **进阶扩展(ADVANCED-FEATURES B/C/D 层)**: diff --git a/docs/UNFINISHED-EDITING-AND-MEDIA.md b/docs/UNFINISHED-EDITING-AND-MEDIA.md new file mode 100644 index 0000000..b4b62f4 --- /dev/null +++ b/docs/UNFINISHED-EDITING-AND-MEDIA.md @@ -0,0 +1,67 @@ +# OpenTake — 音视频编辑 / 媒体库 / MCP 未完成清单 + +> 2026-06-24 整理。配合用户本轮反馈,逐项标注「已完成 / 部分完成(差什么) / 未做」,附文件路径与上游参考。 +> 状态基于实际代码核查(集成分支 `codex/integration-20260624`),非臆测。 + +--- + +## 0. 本轮(2026-06-24)已修复的阻断性问题 + +| 反馈 | 处理 | 文件 | +|---|---|---| +| **clips 删不掉** | 删除前过滤成时间线中真实存在的 clip id(单个 stale id 会让后端整批 RemoveClips 失败),并在 Tauri 下删除后主动 `forceRefresh` | `web/src/store/editActions.ts` `liveSelectedClipIds`/`deleteSelectedClips`/`rippleDeleteSelectedClips` | +| **拖入音频自动跑到末尾** | 拖放改为按鼠标释放点落点:start=落点帧、轨道=落点轨;落点轨重叠则换其他空闲同类轨,无则新建轨(不覆盖既有片段) | `TimelineContainer.tsx` `onMediaDrop` + `editActions.ts` `addMediaToTimelineAt`/`firstOpenCompatibleTrackIndex`;`TimelineRegion.tsx` 去掉重复 append | +| **选中高亮发灰、不明显** | 选中边框改醒目蓝 `rgba(56,139,253,1)` 2px(原近白色边框在片段上发灰) | `web/src/components/timeline/clipRenderer.ts` `SELECTION_BLUE` | +| **预览拖动严重卡顿/延迟** | 合成帧由 140ms 尾随防抖改为「实时帧 + ~12fps 速率闸门」:拖动即时出帧、不再等停下,仍受限不爆量 | `web/src/components/preview/Preview.tsx` `SCRUB_MIN_INTERVAL_MS` | +| **右键菜单固定在左上(0,0)** | 菜单按光标 `clientX/clientY` 定位 + 视口翻转;并修掉渲染期 `onClose()`(改 effect) | `TimelineContainer.tsx` menu state(x/y)+ `ClipContextMenu.tsx` | +| **导入文件夹只导顶层** | 前端改 `importFolder(path, true)` 递归导入整棵目录树下的媒体 | `web/src/store/mediaActions.ts` | +| **银色 Generate 按钮点了没反应** | 加 onClick → 弹「AI 生成功能即将推出」toast(后端生成仍为 stub) | `MediaPanel.tsx` + `dict.ts` `media.generateSoon` | +| **主页 macOS 红绿灯位置偏上、OpenTake 间距挤** | 红绿灯下移 `trafficLightPosition {x:18,y:24}`;`--titlebar-safe-top` 30→44、`--titlebar-safe-left` 78→82 让顶部 UI 下移留白 | `src-tauri/tauri.conf.json`、`web/src/styles/tokens.css` | + +--- + +## 1. 仍未完成 / 本轮明确延后的项(给 Codex 接力) + +### 1.1 媒体库 / 文件夹(用户很想要) +- **嵌套文件夹浏览 UI(钻取/双击进入/面包屑/拖出)** — 未做。后端 `import_folder` 已支持 `recursive`,但 DTO 缺 `folderId/folders` 的层级展示,前端无文件夹瓦片导航。需按上游 `FolderTileView` 做(对应 issue #49)。本轮仅打开了递归导入开关。 +- **不支持格式的展示** — 未做。`import_folder` 只白名单媒体;目录里的非媒体文件不会列出。需后端返回「未支持文件」占位项 + 前端灰显瓦片。 +- **My ↔ Import 切换缩略图重载** — 未修(延后)。根因:`MediaPanel.tsx:90 visibleItems = items.filter(...)` 在切 subTab 时把不匹配项**卸载**,切回时 `` 缩略图重新解码 → 视觉「重载」。推荐修法:渲染全部当前目录项、用 `display:none` 做 subTab 过滤(保持 img 挂载),或加一层缩略图缓存预加载。改动涉及网格结构,需单独小 PR 验证空状态/计数不回归。 +- **媒体面板星标迁后端** — 部分完成。面板星标仍用 `localStorage`(`favorites.ts`);后端 `library_favorite` 命令已就绪未接。 + +### 1.2 片段 / 时间线编辑 +- **片段右键菜单补全** — 部分完成。现有 Split/Delete/Link/Unlink;缺 **Swap Media / Save as Media / Extract Audio / Copy·Cut·Paste**(上游 `TimelineView.swift:741-748`)。Copy/Paste 键盘已可用,仅右键入口缺。 +- **多选 Split** — 部分完成,当前只对单片段切。 +- **Toolbar `[` / `]` / Add Track 按钮** — 未接 onClick(`Toolbar.tsx`)。 +- **Inspector 三段式(scrub 实时→防抖→单条 undo)** — 未做,现每次拖动直接发命令,产生大量 undo 历史(`Inspector.tsx`)。 +- **轨道重排序 / Solo** — 未做(`TrackHeaderColumn.tsx`)。 +- **时间线 Marker** — 未做(上游有 `TimelineMarker`)。 + +### 1.3 Inspector +- **Color Grade / Chroma Key / Mask / Effect 的 UI 面板** — 未做。后端命令 + MCP 工具均已实现,缺 Inspector tab UI。 +- **Text 完整 textStyle(字体/字号/颜色)** — 部分完成,依赖后端 `ClipProperties` 扩展。 +- **AI Edit tab** — 未渲染(类型已定义但 tabs 数组从不 push)。 + +### 1.4 音频 +- **真实音频输出(cpal)+ A/V 同步播放引擎(#53)** — 部分完成。`TimelinePlaybackLayer` 用 HTML5 媒体元素能放视频,但无 cpal 真实音频路径。最大未完成项。 +- **转录(whisper)/ 自动字幕(add_captions)** — 未做(依赖端上 whisper 接线)。 + +### 1.5 主页 macOS 圆角 +- **「用旧版 macOS 圆角而非 macOS 26 风格」** — 未做。窗口圆角由 OS 渲染,Tauri 不直接暴露旧版圆角半径;本轮只调了红绿灯位置与安全区。若要改圆角需自绘窗口/装饰层,代价大,建议另议。 + +--- + +## 2. MCP / Agent 逻辑现状 + +**服务器**:`http://127.0.0.1:19789/mcp`(rmcp Streamable-HTTP,loopback + Origin 守卫),在 `src-tauri/src/mcp.rs` `setup()` 内 `tokio::spawn` 启动,bind 失败仅记日志不影响应用。`claude mcp add --transport http opentake http://127.0.0.1:19789/mcp`。 + +**已完整接线(28)**:读取(get_timeline/get_media/list_folders/list_models/list_workflows)、片段(add/insert/move/remove/split/ripple_delete)、属性(set_clip_properties/set_keyframes/set_color_grade/chroma_key/set_mask/apply_effect)、文本(add_texts)、轨道(remove_tracks)、媒体库(rename/delete/create_folder/move_to_folder)、undo、workflow(activate/deactivate)。 + +**Stub(12,统一返回 "not yet implemented",`dispatch.rs:171-190`)**: +- **媒体读取** `inspect_media`/`get_transcript`/`search_media` — 根因 `CoreHandle` trait 未暴露 `MediaEngine`;需拓宽 `CoreHandle`(新增 `media_engine()`)。 +- **导入** `import_media` — 同上,需 `AppCore` 层新增 `import_media_path` 原子命令。 +- **字幕** `add_captions` — 依赖 `get_transcript`(whisper),`subtitle_export.rs` 纯逻辑已就绪缺桥接。 +- **生成** `generate_video/image/audio`/`upscale_media` — 需 `opentake-gen` 异步 `GenClient` + BYOK 注入 `Dispatcher`;且 `get_timeline.canGenerate` 硬编码 `false`(`dispatch.rs:124`)阻止模型主动调用,需改为 `gen_client.is_some()` 运行时判断。 +- **动效** `add_motion_graphic`/`edit_motion_graphic` — 依赖 Lottie 烘焙(#34)。 +- **时间线分析** `inspect_timeline` — 依赖 `composite_frame` 稳定后接入。 + +**局部 gap**:`create_folder`/`move_to_folder` 不支持 batch `entries`;`canGenerate` 恒 false。 diff --git a/src-tauri/src/commands.rs b/src-tauri/src/commands.rs index 787bb4c..88ac85e 100644 --- a/src-tauri/src/commands.rs +++ b/src-tauri/src/commands.rs @@ -130,94 +130,112 @@ fn msg(e: CmdError) -> String { #[derive(Debug, Deserialize)] #[serde(tag = "type", rename_all = "camelCase")] pub enum EditRequest { - AddClips { - entries: Vec, - }, + #[serde(rename_all = "camelCase")] + AddClips { entries: Vec }, + #[serde(rename_all = "camelCase")] InsertClips { track_index: usize, at_frame: i32, entries: Vec, }, - MoveClips { - moves: Vec, - }, - RemoveClips { + #[serde(rename_all = "camelCase")] + MoveClips { moves: Vec }, + #[serde(rename_all = "camelCase")] + DuplicateClips { clip_ids: Vec, + offset_frames: i32, + target_track_indexes: Vec, }, - SplitClip { - clip_id: String, - at_frame: i32, - }, - TrimClips { - edits: Vec, - }, + #[serde(rename_all = "camelCase")] + RemoveClips { clip_ids: Vec }, + #[serde(rename_all = "camelCase")] + SplitClip { clip_id: String, at_frame: i32 }, + #[serde(rename_all = "camelCase")] + TrimClips { edits: Vec }, + #[serde(rename_all = "camelCase")] SetClipProperties { clip_ids: Vec, properties: ClipPropertiesDto, }, + #[serde(rename_all = "camelCase")] SetKeyframes { clip_id: String, property: KeyframePropertyDto, payload: KeyframePayloadDto, }, + #[serde(rename_all = "camelCase")] StampKeyframe { clip_id: String, property: KeyframePropertyDto, frame: i32, }, + #[serde(rename_all = "camelCase")] RemoveKeyframe { clip_id: String, property: KeyframePropertyDto, frame: i32, }, + #[serde(rename_all = "camelCase")] MoveKeyframe { clip_id: String, property: KeyframePropertyDto, from_frame: i32, to_frame: i32, }, + #[serde(rename_all = "camelCase")] SetKeyframeInterpolation { clip_id: String, property: KeyframePropertyDto, frame: i32, interpolation: Interpolation, }, + #[serde(rename_all = "camelCase")] RippleDeleteRanges { track_index: usize, ranges: Vec, }, - RippleDeleteClips { - clip_ids: Vec, - }, - AddTexts { - entries: Vec, - }, - Link { - clip_ids: Vec, - }, - Unlink { - clip_ids: Vec, - }, - RemoveTracks { - track_indexes: Vec, - }, - InsertTrack { - kind: ClipType, - }, + #[serde(rename_all = "camelCase")] + RippleDeleteClips { clip_ids: Vec }, + #[serde(rename_all = "camelCase")] + AddTexts { entries: Vec }, + #[serde(rename_all = "camelCase")] + Link { clip_ids: Vec }, + #[serde(rename_all = "camelCase")] + Unlink { clip_ids: Vec }, + #[serde(rename_all = "camelCase")] + RemoveTracks { track_indexes: Vec }, + #[serde(rename_all = "camelCase")] + InsertTrack { kind: ClipType }, + #[serde(rename_all = "camelCase")] SetTrackProps { track_index: usize, muted: Option, hidden: Option, sync_locked: Option, }, + #[serde(rename_all = "camelCase")] CreateFolder { name: String, parent_folder_id: Option, }, + #[serde(rename_all = "camelCase")] MoveToFolder { asset_ids: Vec, folder_id: Option, }, + #[serde(rename_all = "camelCase")] + SwapMedia { + clip_id: String, + media_ref: String, + #[serde(default)] + media_type: Option, + #[serde(default)] + source_clip_type: Option, + #[serde(default)] + duration_frames: Option, + #[serde(default)] + trim_start_frame: Option, + }, } impl EditRequest { @@ -238,6 +256,15 @@ impl EditRequest { EditRequest::MoveClips { moves } => EditCommand::MoveClips { moves: moves.into_iter().map(ClipMoveDto::into_move).collect(), }, + EditRequest::DuplicateClips { + clip_ids, + offset_frames, + target_track_indexes, + } => EditCommand::DuplicateClips { + clip_ids, + offset_frames, + target_track_indexes, + }, EditRequest::RemoveClips { clip_ids } => EditCommand::RemoveClips { clip_ids }, EditRequest::SplitClip { clip_id, at_frame } => { EditCommand::SplitClip { clip_id, at_frame } @@ -345,6 +372,21 @@ impl EditRequest { asset_ids, folder_id, }, + EditRequest::SwapMedia { + clip_id, + media_ref, + media_type, + source_clip_type, + duration_frames, + trim_start_frame, + } => EditCommand::SwapMedia { + clip_id, + media_ref, + media_type, + source_clip_type, + duration_frames, + trim_start_frame, + }, }) } } @@ -450,6 +492,20 @@ pub struct ClipPropertiesDto { pub transform: Option, #[serde(default)] pub text_content: Option, + #[serde(default)] + pub crop: Option, + #[serde(default)] + pub fade_in_frames: Option, + #[serde(default)] + pub fade_out_frames: Option, + #[serde(default)] + pub fade_in_interpolation: Option, + #[serde(default)] + pub fade_out_interpolation: Option, + #[serde(default)] + pub flip_horizontal: Option, + #[serde(default)] + pub flip_vertical: Option, } impl ClipPropertiesDto { @@ -463,6 +519,13 @@ impl ClipPropertiesDto { opacity: self.opacity, transform: self.transform, text_content: self.text_content, + crop: self.crop, + fade_in_frames: self.fade_in_frames, + fade_out_frames: self.fade_out_frames, + fade_in_interpolation: self.fade_in_interpolation, + fade_out_interpolation: self.fade_out_interpolation, + flip_horizontal: self.flip_horizontal, + flip_vertical: self.flip_vertical, } } } @@ -588,3 +651,27 @@ impl KeyframePayloadDto { }) } } + +#[cfg(test)] +mod edit_request_serde_tests { + use super::EditRequest; + + // Regression: the front end sends camelCase keys (clipIds/clipId/atFrame…). + // serde's enum-level `rename_all` does NOT rename struct-variant fields, so + // each variant needs its own `rename_all`; without it RemoveClips/SplitClip/ + // … failed to deserialize ("missing field `clip_ids`") and delete/split/etc. + // silently did nothing. + #[test] + fn deserializes_camelcase_multiword_commands() { + serde_json::from_str::(r#"{"type":"removeClips","clipIds":["a"]}"#) + .expect("removeClips camelCase"); + serde_json::from_str::(r#"{"type":"splitClip","clipId":"a","atFrame":5}"#) + .expect("splitClip camelCase"); + serde_json::from_str::( + r#"{"type":"insertClips","trackIndex":0,"atFrame":0,"entries":[]}"#, + ) + .expect("insertClips camelCase"); + serde_json::from_str::(r#"{"type":"rippleDeleteClips","clipIds":["a"]}"#) + .expect("rippleDeleteClips camelCase"); + } +} diff --git a/src-tauri/src/export.rs b/src-tauri/src/export.rs index bd8f8b9..24893ec 100644 --- a/src-tauri/src/export.rs +++ b/src-tauri/src/export.rs @@ -9,11 +9,11 @@ //! Scope of this first cut (SPEC §2.4 / §8.2): //! - **H.264 / .mp4** only. The encoder already supports H.265 / ProRes presets; //! those land in a follow-up so this slice stays a clean, verifiable spine. -//! - **Linear audio mixdown**: every audio-bearing clip's source window is -//! decoded to mono f32 at the mix rate, placed at its frame-derived sample -//! offset, scaled by its `volume_at` envelope, summed, hard-limited, and mux'd -//! in by the encoder (`-c:v copy` + AAC). A timeline with no audio still -//! produces the same video-only file as before. +//! - **Linear audio mixdown**: every audio-track clip's source window is decoded +//! to mono f32 at the mix rate, placed at its frame-derived sample offset, +//! scaled by its `volume_at` envelope, summed, hard-limited, and mux'd in by +//! the encoder (`-c:v copy` + AAC). A timeline with no audio still produces +//! the same video-only file as before. //! - Export renders at the **full** export resolution //! ([`opentake_render::export_render_size`]), not the preview cap. //! - No progress callback / cancellation yet (the orchestrator runs to @@ -399,24 +399,24 @@ fn project_clip_audio( })) } -/// Decode + mix every audio-bearing clip on the timeline into one mono buffer. +/// Decode + mix every audio-track clip on the timeline into one mono buffer. /// -/// Walks audio and video clips (video clips can carry an audio track), projects -/// each through [`project_clip_audio`], and linearly mixes the lot. Returns -/// `None` when nothing contributes audio (→ the caller keeps the video-only -/// output). Errors surface decode/mix failures to the front-end. +/// Mirrors upstream's `audioMix`: visual video tracks are rendered silently, and +/// sound comes from explicit audio clips (including linked audio partners for +/// video-with-audio). Returns `None` when nothing contributes audio (→ the +/// caller keeps the video-only output). Errors surface decode/mix failures to +/// the front-end. fn mix_timeline_audio( timeline: &opentake_domain::Timeline, media: &HashMap, ) -> Result, String> { let mut clips_audio: Vec = Vec::new(); for track in &timeline.tracks { - if track.muted { + if track.kind != ClipType::Audio || track.muted { continue; } for clip in &track.clips { - // Only audio and video clips carry sound; text/image/lottie don't. - if clip.media_type != ClipType::Audio && clip.media_type != ClipType::Video { + if clip.media_type != ClipType::Audio { continue; } if let Some(ca) = project_clip_audio(clip, media, timeline.fps)? { @@ -525,7 +525,7 @@ pub fn run_export( .map_err(|e| format!("encode frame {f} failed: {e}"))?; } - // Decode + linearly mix every audio-bearing clip, then hand the mixed PCM to + // Decode + linearly mix every audio-track clip, then hand the mixed PCM to // the encoder so `finish` mux's it into the container. No audio → video-only. if let Some(pcm) = mix_timeline_audio(timeline, &media)? { encoder.push_audio(pcm); @@ -665,7 +665,7 @@ mod tests { #[test] fn mix_timeline_audio_none_when_only_text_clips() { - // A text clip carries no sound; with no audio/video clips there's nothing + // A text clip carries no sound; with no audio clips there's nothing // to decode, so the result is None without touching the media map. let mut tl = Timeline::new(); let mut track = Track::new("t1", ClipType::Text); @@ -677,6 +677,28 @@ mod tests { assert!(mix_timeline_audio(&tl, &media).expect("ok").is_none()); } + #[test] + fn mix_timeline_audio_ignores_video_track_audio_source() { + // Upstream audioMix only walks audio tracks. A video asset's sound must + // come from its linked audio clip, not the visual clip, or exports double + // the audio compared with preview playback. + let mut tl = Timeline::new(); + let mut track = Track::new("v1", ClipType::Video); + let mut clip = Clip::new("c1", "asset-1", 0, 30); + clip.media_type = ClipType::Video; + track.clips.push(clip); + tl.tracks.push(track); + let mut media: HashMap = HashMap::new(); + media.insert( + "asset-1".into(), + MediaInfo { + path: PathBuf::from("/nonexistent-with-audio.mp4"), + fps: 30.0, + }, + ); + assert!(mix_timeline_audio(&tl, &media).expect("ok").is_none()); + } + #[test] fn mix_timeline_audio_skips_muted_tracks() { // A muted audio track is excluded; with no other audio the result is None diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index 523e467..590e3e1 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -135,6 +135,7 @@ pub fn run() { media::import_media, media::relink_media, media::get_media, + media::extract_audio, media::get_waveform, render::composite_frame, export::export_video, diff --git a/src-tauri/src/media.rs b/src-tauri/src/media.rs index 62f4b2d..774c253 100644 --- a/src-tauri/src/media.rs +++ b/src-tauri/src/media.rs @@ -348,6 +348,46 @@ pub fn get_media(core: State<'_, AppCore>) -> MediaListDto { MediaListDto::from_core(&core) } +/// `extract_audio`: extract the audio track from a media asset into a +/// self-contained audio file (`.m4a` / `.mp3` / `.wav`). The output path is +/// chosen by the caller via a native save dialog; the codec falls out of the +/// extension. Used by the media panel's per-card "extract audio" action +/// (Issue #39). +/// +/// Returns the output path on success. Errors when the asset is unknown, the +/// source path cannot be resolved or found, or ffmpeg fails (missing binary, +/// non-zero exit, unsupported extension). +#[tauri::command] +pub fn extract_audio( + core: State<'_, AppCore>, + media: State<'_, MediaState>, + media_id: String, + out_path: String, +) -> Result { + let manifest = core.media(); + let entry = manifest + .entries + .iter() + .find(|e| e.id == media_id) + .ok_or_else(|| format!("unknown media id: {media_id}"))?; + let input = match &entry.source { + MediaSource::External { absolute_path } => PathBuf::from(absolute_path), + MediaSource::Project { relative_path } => match core.project_dir() { + Some(base) => base.join(relative_path), + None => return Err("project not saved; cannot resolve media path".into()), + }, + }; + if !input.is_file() { + return Err(format!("source file not found: {}", input.display())); + } + let output = PathBuf::from(&out_path); + media + .engine() + .extract_audio(&input, &output) + .map(|p| p.to_string_lossy().into_owned()) + .map_err(|e| e.to_string()) +} + /// `relink_media`: point a missing/offline asset at a newly chosen file, KEEPING /// the same asset id so every clip that references it recovers in place. This is /// the fix for "lost media stays red after re-selecting the path": the old flow diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json index c134255..5c824a0 100644 --- a/src-tauri/tauri.conf.json +++ b/src-tauri/tauri.conf.json @@ -26,6 +26,7 @@ "minHeight": 480, "titleBarStyle": "Overlay", "hiddenTitle": true, + "trafficLightPosition": { "x": 18, "y": 24 }, "backgroundColor": "#0A0A0A", "dragDropEnabled": false } diff --git a/src-tauri/tests/export_integration.rs b/src-tauri/tests/export_integration.rs index 217f80b..ade44ab 100644 --- a/src-tauri/tests/export_integration.rs +++ b/src-tauri/tests/export_integration.rs @@ -161,6 +161,17 @@ fn build_timeline(frames: i32, src_w: i32, src_h: i32, src_fps: f64) -> Timeline tl } +/// Add the audio half of the same asset, matching the UI/core linked-audio +/// placement for a video file that carries sound. +fn add_linked_audio_track(tl: &mut Timeline, frames: i32) { + let mut track = Track::new("a1", ClipType::Audio); + let mut clip = Clip::new("clip-1-audio", "asset-1", 0, frames); + clip.media_type = ClipType::Audio; + clip.source_clip_type = ClipType::Video; + track.clips.push(clip); + tl.tracks.push(track); +} + /// Build a manifest with one external video asset pointing at `media_path`. fn build_manifest(media_path: &Path, src_w: i32, src_h: i32, src_fps: f64) -> MediaManifest { build_manifest_with_audio(media_path, src_w, src_h, src_fps, false) @@ -299,7 +310,8 @@ fn export_with_audio_clip_mux_aac_stream() { return; } - let timeline = build_timeline(frames as i32, sw as i32, sh as i32, sfps as f64); + let mut timeline = build_timeline(frames as i32, sw as i32, sh as i32, sfps as f64); + add_linked_audio_track(&mut timeline, frames as i32); let manifest = build_manifest_with_audio(&src, sw as i32, sh as i32, sfps as f64, true); let req = ExportRequest { diff --git a/web/src/App.tsx b/web/src/App.tsx index cbc8f69..a6990cd 100644 --- a/web/src/App.tsx +++ b/web/src/App.tsx @@ -14,6 +14,38 @@ import { initI18n } from "./i18n"; import { initTheme } from "./store/settingsStore"; import { onGoHome } from "./lib/api"; +function Toast() { + const toast = useEditorUiStore((s) => s.toast); + const clearToast = useEditorUiStore((s) => s.clearToast); + useEffect(() => { + if (!toast) return; + const timer = setTimeout(clearToast, 2000); + return () => clearTimeout(timer); + }, [toast, clearToast]); + if (!toast) return null; + return ( +
+ {toast.message} +
+ ); +} + export default function App() { // Editor-only hooks are safe to keep mounted across views: they only act on // editor state/events and the keyboard handler is a no-op until the editor is @@ -65,6 +97,7 @@ export default function App() {
+ ); } diff --git a/web/src/components/home/HomeView.tsx b/web/src/components/home/HomeView.tsx index 3ef7477..f01c807 100644 --- a/web/src/components/home/HomeView.tsx +++ b/web/src/components/home/HomeView.tsx @@ -10,7 +10,7 @@ import { useState } from "react"; import { Plus, FolderOpen, Settings as SettingsIcon, Film, Trash2, Library } from "lucide-react"; import { Icon } from "../ui/Icon"; -import { useT } from "../../i18n"; +import { useT, type TFunction } from "../../i18n"; import { useEditorUiStore } from "../../store/uiStore"; import { useRecentStore, type RecentProject } from "../../store/recentStore"; import { @@ -235,7 +235,7 @@ function NewProjectCard({ onClick }: { onClick: () => void }) { textAlign: "left", position: "relative", zIndex: hovered ? 2 : 1, - transform: hovered ? "scale(1.02)" : "scale(1)", + transform: hovered ? "scale(1.03)" : "scale(1)", transition: "transform var(--anim-transition) var(--ease-out)", }} > @@ -269,6 +269,23 @@ function NewProjectCard({ onClick }: { onClick: () => void }) { ); } +/** Format `openedAt` (epoch ms) as a relative time string — today / yesterday / + * N days ago / N weeks ago / N months ago. Mirrors upstream's + * `RelativeDateTimeFormatter` output. */ +function relativeTime(openedAt: number, t: TFunction): string { + const now = Date.now(); + const diffMs = now - openedAt; + const dayMs = 86_400_000; + const days = Math.floor(diffMs / dayMs); + if (days <= 0) return t("home.relative.today"); + if (days === 1) return t("home.relative.yesterday"); + if (days < 7) return t("home.relative.daysAgo", { count: days }); + const weeks = Math.floor(days / 7); + if (weeks < 5) return t("home.relative.weeksAgo", { count: weeks }); + const months = Math.floor(days / 30); + return t("home.relative.monthsAgo", { count: months }); +} + function ProjectCard({ entry }: { entry: RecentProject }) { const t = useT(); const remove = useRecentStore((s) => s.remove); @@ -281,7 +298,7 @@ function ProjectCard({ entry }: { entry: RecentProject }) { style={{ position: "relative", zIndex: hovered ? 2 : 1, - transform: hovered ? "scale(1.02)" : "scale(1)", + transform: hovered ? "scale(1.03)" : "scale(1)", transition: "transform var(--anim-transition) var(--ease-out)", }} > @@ -306,30 +323,47 @@ function ProjectCard({ entry }: { entry: RecentProject }) { }} > - -
- {entry.name} + {/* Bottom gradient + name overlay (mirrors upstream ProjectCard's + 60pt black gradient + white title). Keeps the title inside the + thumbnail so the card footprint matches upstream. */} +
+ + {entry.name} + +
- {entry.path} + {relativeTime(entry.openedAt, t)}
@@ -349,7 +383,7 @@ function ProjectCard({ entry }: { entry: RecentProject }) { display: "inline-flex", alignItems: "center", justifyContent: "center", - borderRadius: "var(--radius-sm)", + borderRadius: "50%", background: "rgba(0,0,0,0.55)", color: "var(--status-error)", }} diff --git a/web/src/components/inspector/Inspector.tsx b/web/src/components/inspector/Inspector.tsx index 066ed38..4b4b125 100644 --- a/web/src/components/inspector/Inspector.tsx +++ b/web/src/components/inspector/Inspector.tsx @@ -5,7 +5,8 @@ * Text/AI-Edit tabs are scaffolded (TODO: full parity in a later pass). */ -import { Info, SlidersHorizontal, Diamond } from "lucide-react"; +import { useState } from "react"; +import { Info, SlidersHorizontal, Diamond, RefreshCw } from "lucide-react"; import { PanelHeaderBar } from "../ui/PanelShell"; import { Icon } from "../ui/Icon"; import { ScrubbableNumberField } from "./ScrubbableNumberField"; @@ -13,10 +14,19 @@ import { TextTab } from "./TextTab"; import { KeyframesPanel } from "./KeyframesPanel"; import { useProjectStore } from "../../store/projectStore"; import { useEditorUiStore } from "../../store/uiStore"; +import { useMediaStore } from "../../store/mediaStore"; import * as edit from "../../store/editActions"; import { formatTimecode } from "../../lib/geometry"; +import { + cropAt, + opacityAt, + rotationAt, + sizeAt, + topLeftAt, + volumeAt, +} from "../../lib/clip"; import { useT, type TFunction } from "../../i18n"; -import type { Clip, Timeline } from "../../lib/types"; +import type { Clip, Crop, Interpolation, MediaItem, Timeline } from "../../lib/types"; function gcd(a: number, b: number): number { return b === 0 ? a : gcd(b, a % b); @@ -127,6 +137,79 @@ function Row({ label, children }: { label: string; children: React.ReactNode }) ); } +/** A non-interactive numeric value shown when a property is keyframe-animated. + * Mirrors ScrubbableNumberField's typography but without drag/click handlers. */ +function ReadOnlyValue({ text, width = 56 }: { text: string; width?: number }) { + return ( + + {text} + + ); +} + +/** Inline hint shown beside a read-only field when a property is animated. */ +function AnimatedHint({ t }: { t: TFunction }) { + return ( + + {t("inspector.animatedHint")} + + ); +} + +const INTERPOLATION_KEYS: Record = { + linear: "inspector.interpolation.linear", + hold: "inspector.interpolation.hold", + smooth: "inspector.interpolation.smooth", +}; + +/** A compact native ` onChange(e.target.value as Interpolation)} + style={{ + fontSize: "var(--fs-sm)", + color: "var(--accent-primary)", + background: "var(--bg-raised)", + border: "var(--bw-thin) solid var(--border-primary)", + borderRadius: "var(--radius-xs)", + padding: "1px 4px", + }} + > + {(Object.keys(INTERPOLATION_KEYS) as Interpolation[]).map((k) => ( + + ))} + + ); +} + const TAB_LABEL_KEY: Record<"text" | "video" | "audio" | "aiEdit", string> = { text: "inspector.tab.text", video: "inspector.tab.video", @@ -134,6 +217,141 @@ const TAB_LABEL_KEY: Record<"text" | "video" | "audio" | "aiEdit", string> = { aiEdit: "inspector.tab.aiEdit", }; +/** A compact media-type badge label. */ +function mediaTypeLabel(type: MediaItem["type"]): string { + switch (type) { + case "video": + return "Video"; + case "audio": + return "Audio"; + case "image": + return "Image"; + case "text": + return "Text"; + case "lottie": + return "Lottie"; + } +} + +/** "替换媒体" section: opens an inline media picker that lists every library + * asset except the clip's current `mediaRef`. Selecting one fires + * `edit.swapMedia`, which preserves all editing attributes and truncates the + * duration when the new media is shorter. Text clips don't render this section + * (they have no source media to swap). */ +function SwapMediaSection({ clip, t }: { clip: Clip; t: TFunction }) { + const [open, setOpen] = useState(false); + const items = useMediaStore((s) => s.items); + + // Exclude the current media source; text items aren't swappable targets. + const candidates = items.filter((m) => m.id !== clip.mediaRef && m.type !== "text"); + + const handlePick = (item: MediaItem) => { + void edit.swapMedia(clip.id, item.id, { mediaType: item.type }); + setOpen(false); + }; + + return ( +
+ + + {open && ( +
+
+ {t("inspector.swapMediaTitle")} +
+ {candidates.length === 0 ? ( +
+ {t("inspector.swapMediaEmpty")} +
+ ) : ( + candidates.map((item) => ( + + )) + )} +
+ )} +
+ ); +} + function ClipInspector({ clip, tab, @@ -159,9 +377,29 @@ function ClipInspector({ const activeTab = tabs.includes(tab as never) ? tab : tabs[0]; + // Live sampling: read the current playhead frame so every numeric field shows + // the value at the playhead (upstream `InspectorView.livePreview`). + const activeFrame = useEditorUiStore((s) => s.activeFrame); + const commit = (props: Parameters[1]) => edit.setClipProperties([clip.id], props); + // Track-active checks (a track is active iff it holds ≥1 keyframe). + const opacityAnimated = !!clip.opacityTrack && clip.opacityTrack.keyframes.length > 0; + const volumeAnimated = !!clip.volumeTrack && clip.volumeTrack.keyframes.length > 0; + const rotationAnimated = !!clip.rotationTrack && clip.rotationTrack.keyframes.length > 0; + const scaleAnimated = !!clip.scaleTrack && clip.scaleTrack.keyframes.length > 0; + const positionAnimated = !!clip.positionTrack && clip.positionTrack.keyframes.length > 0; + const cropAnimated = !!clip.cropTrack && clip.cropTrack.keyframes.length > 0; + + // Sampled values at the playhead. + const sampledOpacity = opacityAt(clip, activeFrame); + const sampledVolume = volumeAt(clip, activeFrame); + const sampledRotation = rotationAt(clip, activeFrame); + const sampledScale = sizeAt(clip, activeFrame)[0]; + const sampledTopLeft = topLeftAt(clip, activeFrame); + const sampledCrop = cropAt(clip, activeFrame); + return (
{tabs.length > 1 && ( @@ -192,24 +430,35 @@ function ClipInspector({ )}
+ {clip.mediaType !== "text" && } {activeTab === "text" ? ( ) : activeTab === "audio" ? (
- (20 * Math.log10(Math.max(1e-6, v))).toFixed(1)} - suffix=" dB" - width={56} - displayTextOverride={(v) => (v <= 0 ? "-∞ dB" : null)} - onCommit={(v) => commit({ volume: v })} - /> + {volumeAnimated ? ( + <> + + + + ) : ( + (20 * Math.log10(Math.max(1e-6, v))).toFixed(1)} + suffix=" dB" + width={56} + displayTextOverride={(v) => (v <= 0 ? "-∞ dB" : null)} + onCommit={(v) => commit({ volume: v })} + /> + )} +
) : ( <> @@ -218,45 +467,86 @@ function ClipInspector({
- Math.round(v * 100).toString()} - suffix="%" - width={56} - onCommit={(v) => - commit({ transform: { ...clip.transform, width: v, height: v } }) - } - /> + {scaleAnimated ? ( + <> + + + + ) : ( + Math.round(v * 100).toString()} + suffix="%" + width={56} + onCommit={(v) => + commit({ transform: { ...clip.transform, width: v, height: v } }) + } + /> + )} - v.toFixed(0)} - suffix="°" - width={56} - onCommit={(v) => commit({ transform: { ...clip.transform, rotation: v } })} - /> + {rotationAnimated ? ( + <> + + + + ) : ( + v.toFixed(0)} + suffix="°" + width={56} + onCommit={(v) => commit({ transform: { ...clip.transform, rotation: v } })} + /> + )} - Math.round(v * 100).toString()} - suffix="%" - width={56} - onCommit={(v) => commit({ opacity: v })} - /> + {opacityAnimated ? ( + <> + + + + ) : ( + Math.round(v * 100).toString()} + suffix="%" + width={56} + onCommit={(v) => commit({ opacity: v })} + /> + )} + + + + + + + +
@@ -305,6 +595,213 @@ function ClipInspector({ ); } +// MARK: - Position section (top-left x/y) + +function PositionSection({ + clip, + sampledTopLeft, + animated, + commit, + t, +}: { + clip: Clip; + sampledTopLeft: { x: number; y: number }; + animated: boolean; + commit: (props: Parameters[1]) => void; + t: TFunction; +}) { + // Editing top-left x/y writes back through `transform.centerX/centerY`. The + // size is preserved from the current transform (scale track writes via scale). + const [w, h] = [clip.transform.width, clip.transform.height]; + return ( +
+ + + {animated ? ( + <> + + + + ) : ( + v.toFixed(3)} + width={56} + onCommit={(v) => + commit({ transform: { ...clip.transform, centerX: v + w / 2 } }) + } + /> + )} + + + {animated ? ( + <> + + + + ) : ( + v.toFixed(3)} + width={56} + onCommit={(v) => + commit({ transform: { ...clip.transform, centerY: v + h / 2 } }) + } + /> + )} + +
+ ); +} + +// MARK: - Crop section (4 edge insets, 0–1) + +function CropSection({ + clip, + sampledCrop, + animated, + commit, + t, +}: { + clip: Clip; + sampledCrop: Crop; + animated: boolean; + commit: (props: Parameters[1]) => void; + t: TFunction; +}) { + const commitEdge = (edge: keyof Crop, v: number) => { + const next: Crop = { ...clip.crop, [edge]: v }; + commit({ crop: next }); + }; + const renderEdge = (label: string, edge: keyof Crop, value: number) => ( + + {animated ? ( + <> + + + + ) : ( + v.toFixed(3)} + width={56} + onCommit={(v) => commitEdge(edge, v)} + /> + )} + + ); + return ( +
+ + {renderEdge(t("inspector.field.cropLeft"), "left", sampledCrop.left)} + {renderEdge(t("inspector.field.cropTop"), "top", sampledCrop.top)} + {renderEdge(t("inspector.field.cropRight"), "right", sampledCrop.right)} + {renderEdge(t("inspector.field.cropBottom"), "bottom", sampledCrop.bottom)} +
+ ); +} + +// MARK: - Flip section (horizontal / vertical checkboxes) + +function FlipSection({ + clip, + commit, + t, +}: { + clip: Clip; + commit: (props: Parameters[1]) => void; + t: TFunction; +}) { + const checkboxStyle: React.CSSProperties = { + accentColor: "var(--accent-primary)", + cursor: "pointer", + }; + return ( +
+ + + commit({ flipHorizontal: e.target.checked })} + /> + + + commit({ flipVertical: e.target.checked })} + /> + +
+ ); +} + +// MARK: - Fade section (fade in/out frames + interpolation) + +function FadeSection({ + clip, + commit, + t, +}: { + clip: Clip; + commit: (props: Parameters[1]) => void; + t: TFunction; +}) { + return ( +
+ + + v.toFixed(0)} + width={56} + onCommit={(v) => commit({ fadeInFrames: Math.round(v) })} + /> + + + commit({ fadeInInterpolation: v })} + t={t} + /> + + + v.toFixed(0)} + width={56} + onCommit={(v) => commit({ fadeOutFrames: Math.round(v) })} + /> + + + commit({ fadeOutInterpolation: v })} + t={t} + /> + +
+ ); +} + function ProjectMetadata({ timeline, t }: { timeline: Timeline; t: TFunction }) { const g = gcd(timeline.width, timeline.height) || 1; const total = timeline.tracks.reduce( diff --git a/web/src/components/media/MediaPanel.tsx b/web/src/components/media/MediaPanel.tsx index c242ba9..756ef75 100644 --- a/web/src/components/media/MediaPanel.tsx +++ b/web/src/components/media/MediaPanel.tsx @@ -14,11 +14,14 @@ import { Filter, ArrowUpDown, LayoutGrid, + Folder, FolderOpen, FileVideo, FileAudio, Image as ImageIcon, Type as TypeIcon, + ChevronRight, + FolderPlus, AlertTriangle, Star, } from "lucide-react"; @@ -35,8 +38,10 @@ import { useT } from "../../i18n"; import { formatTimecode } from "../../lib/geometry"; import { assetUrl } from "../../lib/asset"; import { useProjectStore } from "../../store/projectStore"; -import { addMediaToTimeline } from "../../store/editActions"; -import type { MediaItem } from "../../lib/types"; +import { addMediaToTimeline, createFolder, moveToFolder } from "../../store/editActions"; +import { extractAudio } from "../../lib/api"; +import { saveDialog } from "../../lib/dialog"; +import type { MediaFolder, MediaItem } from "../../lib/types"; import { MediaTabBar, MediaSubTabBar } from "./MediaTabBar"; import { useFavoritesStore, useIsFavorite } from "./favorites"; @@ -73,22 +78,27 @@ export function MediaPanel() { function MediaTab({ kind }: { kind: MediaTabKind }) { const t = useT(); const items = useMediaStore((s) => s.items); + const folders = useMediaStore((s) => s.folders); const importing = useMediaStore((s) => s.importing); const error = useMediaStore((s) => s.error); + const currentFolderId = useEditorUiStore((s) => s.mediaPanelCurrentFolderId); + const setCurrentFolderId = useEditorUiStore((s) => s.setMediaPanelCurrentFolderId); const subTab = useEditorUiStore((s) => s.mediaSubTab); const setSubTab = useEditorUiStore((s) => s.setMediaSubTab); const favoriteIds = useFavoritesStore((s) => s.ids); - // 过滤管线（全部不可变 filter，不改 store）： - // 1) 按主标签——「音频」仅纯音频素材（严格 type==='audio'，不含有声视频，匹配剪映）； - // 若日后需含「有音轨的视频」，改为 `item.type === "audio" || item.hasAudio`。 - // 「素材」显示全部类型。 - // 2) 按二级标签——「我的」仅星标收藏（命中当前已加载 items）；「导入」显示全部。 - const filtered = items.filter((item) => { + const visibleItems = items.filter((item) => { + if ((item.folderId ?? null) !== currentFolderId) return false; if (kind === "audio" && item.type !== "audio") return false; if (subTab === "mine" && !favoriteIds.has(item.id)) return false; return true; }); + const visibleFolders = + kind === "material" && subTab === "import" + ? folders.filter((folder) => (folder.parentFolderId ?? null) === currentFolderId) + : []; + const breadcrumb = buildBreadcrumb(folders, currentFolderId); + const totalCount = visibleFolders.length + visibleItems.length; return ( <> @@ -106,8 +116,31 @@ function MediaTab({ kind }: { kind: MediaTabKind }) { {/* actionsRow */}
+ {kind === "material" && subTab === "import" && ( + + )}
+ {/* Breadcrumb */} + {/* contextBar */}
{t("media.library")} - {importing ? t("media.importing") : t("media.itemCount", { count: filtered.length })} + {importing ? t("media.importing") : t("media.itemCount", { count: totalCount })}
{error && (
@@ -174,11 +214,113 @@ function MediaTab({ kind }: { kind: MediaTabKind }) { )}
- {filtered.length === 0 ? : } + {totalCount === 0 ? ( + + ) : ( + + )} ); } +/** Build the breadcrumb path from root to `folderId` (inclusive). Root is + * represented as a synthetic entry with id=null. */ +function buildBreadcrumb( + folders: MediaFolder[], + folderId: string | null, +): Array<{ id: string | null; name: string }> { + const path: Array<{ id: string | null; name: string }> = []; + let cur = folderId; + const guard = new Set(); + while (cur !== null && !guard.has(cur)) { + guard.add(cur); + const f = folders.find((x) => x.id === cur); + if (!f) break; + path.unshift({ id: f.id, name: f.name }); + cur = f.parentFolderId ?? null; + } + return path; +} + +function Breadcrumb({ + path, + currentFolderId, + onNavigate, +}: { + path: Array<{ id: string | null; name: string }>; + currentFolderId: string | null; + onNavigate: (id: string | null) => void; +}) { + const t = useT(); + return ( +
+ onNavigate(null)} + /> + {path.map((p) => ( + + + onNavigate(p.id)} + /> + + ))} +
+ ); +} + +function BreadcrumbCrumb({ + label, + active, + onClick, +}: { + label: string; + active: boolean; + onClick: () => void; +}) { + return ( + + ); +} + /** Import button with a small folder/files menu (CapCut-style folder import). */ function ImportMenu() { const t = useT(); @@ -300,7 +442,15 @@ const TYPE_ICON: Record = { lottie: Sparkles, }; -function MediaGrid({ items }: { items: MediaItem[] }) { +function MediaGrid({ + items, + folders, + allItems, +}: { + items: MediaItem[]; + folders: MediaFolder[]; + allItems: MediaItem[]; +}) { return (
+ {folders.map((folder) => ( + + ))} {items.map((item) => ( ))} @@ -320,6 +477,115 @@ function MediaGrid({ items }: { items: MediaItem[] }) { ); } +/** Count items whose folderId is `folderId` (direct children only — matches the + * panel's single-level navigation). Used for the FolderTile badge. */ +function countDescendants(items: MediaItem[], folderId: string): number { + return items.reduce((n, it) => (it.folderId === folderId ? n + 1 : n), 0); +} + +/** A folder tile in the media grid. Double-click opens it (sets the current + * folder id in the UI store); single-click selects. Drag-over highlights and + * accepts dropped media items (reparents them via `moveToFolder`). Mirrors + * upstream `FolderTileView` (onTap / onOpen / drop target). */ +function FolderTile({ + folder, + itemCount, +}: { + folder: MediaFolder; + itemCount: number; +}) { + const t = useT(); + const setCurrentFolderId = useEditorUiStore((s) => s.setMediaPanelCurrentFolderId); + const [dragOver, setDragOver] = useState(false); + + const onDragOver = (e: React.DragEvent) => { + if (e.dataTransfer.types.includes(MEDIA_DND_TYPE)) { + e.preventDefault(); + e.dataTransfer.dropEffect = "move"; + setDragOver(true); + } + }; + const onDragLeave = () => setDragOver(false); + const onDrop = (e: React.DragEvent) => { + e.preventDefault(); + setDragOver(false); + const assetId = e.dataTransfer.getData(MEDIA_DND_TYPE); + if (assetId && assetId !== folder.id) { + void moveToFolder([assetId], folder.id); + } + }; + + return ( +
setCurrentFolderId(folder.id)} + onDragOver={onDragOver} + onDragLeave={onDragLeave} + onDrop={onDrop} + title={folder.name} + style={{ + display: "flex", + flexDirection: "column", + gap: 4, + cursor: "pointer", + }} + > +
+ + {dragOver && ( + + {t("media.folder.dropHere")} + + )} +
+ + {folder.name} + + + {t("media.folder.itemCount", { count: itemCount })} + +
+ ); +} + function MediaCard({ item }: { item: MediaItem }) { const t = useT(); const fps = useProjectStore((s) => s.timeline.fps); @@ -331,18 +597,53 @@ function MediaCard({ item }: { item: MediaItem }) { const toggleFavorite = useFavoritesStore((s) => s.toggle); // Offline assets shouldn't try to load a (now-missing) thumbnail. const thumb = item.missing ? null : assetUrl(item.path); + const [hovered, setHovered] = useState(false); + const [feedback, setFeedback] = useState(null); const onDragStart = (e: React.DragEvent) => { e.dataTransfer.setData(MEDIA_DND_TYPE, item.id); e.dataTransfer.effectAllowed = "copy"; }; + /** Extract the audio track into a standalone file via ffmpeg. Opens a native + * save dialog (m4a/mp3/wav), then calls the `extract_audio` Tauri command. + * Only shown for video assets that carry audio (Issue #39). */ + const onExtractAudio = async (e: React.MouseEvent) => { + e.stopPropagation(); + e.preventDefault(); + const save = await saveDialog(); + if (!save) return; // non-Tauri / dialog unavailable + const chosen = await save({ + title: t("media.extractAudio"), + defaultPath: `${item.name}.m4a`, + filters: [ + { name: "Audio (M4A)", extensions: ["m4a"] }, + { name: "Audio (MP3)", extensions: ["mp3"] }, + { name: "Audio (WAV)", extensions: ["wav"] }, + ], + }); + if (typeof chosen !== "string") return; // user cancelled + setFeedback(null); + try { + const out = await extractAudio(item.id, chosen); + setFeedback(t("media.extractAudioSuccess", { path: out })); + } catch (err) { + setFeedback(t("media.extractAudioFailed", { error: String(err) })); + } + setTimeout(() => setFeedback(null), 4000); + }; + + // Only local, present video assets with an audio track can be extracted. + const canExtractAudio = item.type === "video" && item.hasAudio && !item.missing; + return (
setPreviewMedia(item.id)} onDoubleClick={() => void addMediaToTimeline(item)} + onMouseEnter={() => setHovered(true)} + onMouseLeave={() => setHovered(false)} title={item.name} style={{ display: "flex", flexDirection: "column", gap: 4, cursor: "grab" }} > @@ -475,6 +776,32 @@ function MediaCard({ item }: { item: MediaItem }) { > + {canExtractAudio && hovered && ( + + )}
{item.name} + {feedback && ( + + {feedback} + + )}
); } diff --git a/web/src/components/preview/Preview.tsx b/web/src/components/preview/Preview.tsx index bf83a80..0ddac07 100644 --- a/web/src/components/preview/Preview.tsx +++ b/web/src/components/preview/Preview.tsx @@ -3,6 +3,12 @@ * bar with project-setting badges. The canvas displays Rust composite frames via * the `preview_frame` event (SPEC §11.2) — not yet wired, so it shows the canvas * background + a centered placeholder. Transport drives the local playhead. + * + * ⚠️ The TIMELINE path's composite ⇄
)} + {/* Layer: TimelinePlayback stays mounted when paused so media elements + survive the pause→play transition (upstream VideoEngine model). */} + {!previewItem && timelineHasContent && ( + + )} {/* The app's scrub + transport are the single control surface — they drive @@ -372,7 +384,7 @@ function PreviewTabs({ item }: { item: MediaItem | null }) { function ScrubBar({ frame, total, onSeek }: { frame: number; total: number; onSeek: (f: number) => void }) { const ref = useRef(null); const [hover, setHover] = useState(false); - const progress = total > 0 ? frame / total : 0; + const progress = total > 0 ? Math.max(0, Math.min(1, frame / total)) : 0; const seekFromEvent = (clientX: number) => { const el = ref.current; @@ -388,7 +400,7 @@ function ScrubBar({ frame, total, onSeek }: { frame: number; total: number; onSe onMouseEnter={() => setHover(true)} onMouseLeave={() => setHover(false)} onPointerDown={(e) => { - (e.target as HTMLElement).setPointerCapture(e.pointerId); + e.currentTarget.setPointerCapture(e.pointerId); seekFromEvent(e.clientX); }} onPointerMove={(e) => { @@ -408,6 +420,7 @@ function ScrubBar({ frame, total, onSeek }: { frame: number; total: number; onSe style={{ flex: 1, height: hover ? 4 : 3, + position: "relative", background: "rgba(255,255,255,0.1)", borderRadius: 2, }} diff --git a/web/src/components/preview/TimelinePlaybackLayer.tsx b/web/src/components/preview/TimelinePlaybackLayer.tsx index bcf9d86..28aa998 100644 --- a/web/src/components/preview/TimelinePlaybackLayer.tsx +++ b/web/src/components/preview/TimelinePlaybackLayer.tsx @@ -1,4 +1,11 @@ /** + * ⚠️ REWRITE-PER-UPSTREAM (issue #142): this two-surface model (DOM