Skip to content

feat(media): extract audio track to local file (#39)#79

Closed
cuic19053-hue wants to merge 3 commits into
appergb:mainfrom
cuic19053-hue:feat-39-extract-audio
Closed

feat(media): extract audio track to local file (#39)#79
cuic19053-hue wants to merge 3 commits into
appergb:mainfrom
cuic19053-hue:feat-39-extract-audio

Conversation

@cuic19053-hue

Copy link
Copy Markdown
Contributor

Summary

End-to-end "extract audio" path so users can save a video's soundtrack as a standalone audio file from the media panel. Closes #39.

Changes

Backend (Rust)

  • crates/opentake-media/src/lib.rsMediaEngine::extract_audio + extract_audio_file helper drive ffmpeg via the existing ffmpeg_path() CLI wrapper. Invokes -y -i <in> -vn plus codec args picked by output extension:
    • .m4a / .aac → AAC 192k
    • .mp3 → libmp3lame 192k
    • .wav → pcm_s16le
  • src-tauri/src/media.rs — new extract_audio Tauri command resolves a media id to its MediaSource::External absolute path, validates the file exists, then delegates to the engine. Returns the output path.
  • src-tauri/src/lib.rs — register media::extract_audio in generate_handler!.

Frontend (React/TS)

  • web/src/lib/api.tsextractAudio(mediaId, outPath) wrapper; rejects outside Tauri (no ffmpeg available).
  • web/src/components/media/MediaPanel.tsxMediaCard gains a star-shaped "Extract Audio" button on the top-left, shown only when hovering a video that carries an audio track. Click opens a native save dialog (m4a/mp3/wav filters), invokes extract_audio, and surfaces a transient success/failure feedback message. stopPropagation+preventDefault keep the click from selecting the card.
  • web/src/i18n/dict.ts — 6 new keys (zh-CN + en) for the button title, hint, success, failure, and no-audio messages.

Verification

  • pnpm tsc --noEmit
  • pnpm build ✅ (vite production build, 1639 modules)
  • Rust changes follow existing patterns (mirror MediaEngine method + Tauri command); CI will verify cargo build.

User flow

  1. Hover a video asset in the media panel that has an audio track.
  2. Click the ★ star icon (top-left of the card).
  3. Native save dialog opens with .m4a / .mp3 / .wav filters.
  4. ffmpeg muxes the audio track to the chosen path.
  5. A transient toast confirms the saved path (or shows the error).

Notes

  • Only MediaSource::External assets are supported (project-relative assets return an error). This matches the existing import_folder / import_media path which also requires absolute external files.
  • Output codec is selected by file extension so the user can pick the format they need without an extra UI control.

…card) (appergb#39)

Adds an end-to-end "extract audio" path so users can save a video's
soundtrack as a standalone audio file from the media panel.

Backend (Rust):
- opentake-media: `MediaEngine::extract_audio` + `extract_audio_file`
  helper drive ffmpeg via the existing `ffmpeg_path()` CLI wrapper.
  `-y -i <in> -vn` plus codec args picked by output extension:
  .m4a/.aac → AAC 192k, .mp3 → libmp3lame 192k, .wav → pcm_s16le.
- src-tauri/media: new `extract_audio` Tauri command resolves a media
  id to its `MediaSource::External` absolute path, validates the file
  exists, then delegates to the engine. Returns the output path.
- src-tauri/lib: register `media::extract_audio` in `generate_handler!`.

Frontend (React/TS):
- api.ts: `extractAudio(mediaId, outPath)` wrapper; rejects outside
  Tauri (no ffmpeg available).
- MediaPanel.tsx: MediaCard gains a star-shaped "Extract Audio" button
  on the top-left, shown only when hovering a video that carries an
  audio track. Click opens a native save dialog (m4a/mp3/wav filters),
  invokes `extract_audio`, and surfaces a transient success/failure
  feedback message. `stopPropagation`+`preventDefault` keep the click
  from selecting the card.
- i18n dict.ts: 6 new keys (zh-CN + en) for the button title, hint,
  success, failure, and no-audio messages.

Closes appergb#39.

@appergb appergb left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cuic19053-hue 自动审核结论:请修改(REQUEST_CHANGES)。Rust 抽音轨逻辑结构正确,但无法直接合并:

阻塞项:

  1. 冲突,需 rebase:diff 基于 PR#102 之前的旧版。src-tauri/src/lib.rs 上下文缺 media::get_waveform 行 → patch 应用失败,extract_audio 命令注册不上;MediaPanel.tsx 也是旧版上下文。请 rebase 到最新 main。
  2. UI 叠层冲突(严重):main 的 MediaCard 已有收藏星标固定在 position:absolute left:4 top:4;本 PR 在同坐标再放一个 Star 做"提取音频",两按钮叠加、点击错乱。上游抽音频走右键菜单 Save as Media(EditorViewModel+SaveAsMedia.swift),不用星标。请换图标/位置或改走右键菜单(与 #93/#101 协调)。
  3. 缺 issue #39 验收测试:提取后文件存在 / 时长匹配 / 无视频流 三项测试缺失,请补集成测试。
  4. out_path 无后端路径边界校验(任意 ../ 可写)。

注:本 PR 触碰 media.rs/MediaPanel,属 #91 重写范围,建议与 #91/#101 协调以免白做。请 rebase + 重设计 UI + 补测试后重提。

# Conflicts:
#	src-tauri/src/lib.rs
#	src-tauri/src/media.rs
#	web/src/components/media/MediaPanel.tsx
#	web/src/i18n/dict.ts
#	web/src/lib/api.ts
 review #3 #4)

Review #3 (缺验收测试):
- Extract audio_codec_args() as pure function for unit testing
- Add 4 unit tests covering m4a/mp3/wav/unknown extension selection
- Add #[ignore] integration test verifying issue appergb#39 acceptance criteria:
  1. output file exists after extraction
  2. duration matches input (within 0.5s)
  3. no video stream in output
  Auto-skips when ffmpeg/ffprobe unavailable; run with --ignored

Review #4 (out_path 无路径边界校验):
- Add validate_extract_output() enforcing:
  - reject null bytes (OS API truncation risk)
  - require absolute path (save dialog always returns absolute)
  - extension whitelist (m4a/m4r/aac/mp3/wav)
- extract_audio command calls validator before touching manifest/ffmpeg
- 5 unit tests covering each rejection path + whitelist acceptance

No behavioral change to the extraction logic itself; the codec table
and ffmpeg invocation are unchanged.
@cuic19053-hue

Copy link
Copy Markdown
Contributor Author

@appergb 请求重新审查。R2 四个阻塞项已逐条核对/补修,CI 双绿(commit 74af487,基于 2d655b9 rebase+UI 修复):

1. 冲突需 rebase ✅

  • 2d655b9 已 merge upstream/mainsrc-tauri/src/lib.rs:138 media::extract_audio 注册在 media::get_waveform 之前(上下文已对齐最新 main)
  • MediaPanel.tsx hunk 冲突已解决,diff 基于 upstream/main 最新版本

2. UI 叠层冲突 ✅

3. 缺 issue #39 验收测试 ✅(commit 74af487

  • crates/opentake-media/src/lib.rs 提取 audio_codec_args() 为纯函数 + 4 个单元测试覆盖 m4a/m4r/aac/mp3/wav/unknown extension
  • crates/opentake-media/src/lib.rs 新增 #[ignore] 集成测试 extract_audio_file_produces_audio_only_output_matching_input_duration,验证 issue 提取视频音频/音乐保存本地(媒体项星标导出) #39 三项验收标准:
    1. 提取后文件存在assert!(output.is_file())
    2. 时长匹配:ffprobe 对比 input/output duration,abs(dur_in - dur_out) < 0.5s
    3. 无视频流ffprobe -select_streams v 返回空
    • ffmpeg -f lavfi sine + color 生成 2s 测试素材
    • 自动跳过(ffmpeg/ffprobe 不可用时 return),CI Linux 有 ffmpeg 可 cargo test --ignored
  • src-tauri/src/media.rs validate_extract_output 5 个单元测试

4. out_path 路径边界校验 ✅(commit 74af487

  • src-tauri/src/media.rs:367 新增 validate_extract_output()extract_audio command 在触及 manifest/ffmpeg 前先调用校验:
    • 拒绝 null 字节(out_path.contains('\0') → 防 OS API 截断)
    • 必须绝对路径(output.is_absolute() → save dialog 本就返回绝对路径,但 command 也可被直接调用)
    • 扩展名白名单(m4a/m4r/aac/mp3/wav,与 codec 表 + save dialog filters 一致)
  • 5 个单元测试覆盖每条拒绝路径 + 白名单接受

本地验证

  • cargo fmt --check
  • cargo clippy -p opentake-media --lib --tests -- -D warnings
  • cargo test -p opentake-media --lib ✅ 234 passed; 1 ignored
  • cargo check -p opentake-tauri --lib --tests
  • pnpm run build(tsc -b + vite build)✅ 1650 modules in 3.63s
  • 注:cargo test -p opentake-tauri 本地因 muda v0.19.3 Windows 编译问题(AcceleratorParseError doesn't implement Display)无法跑,依赖 CI Linux 验证;validate_extract_output 是简单纯函数,cargo check 已证明类型正确

@appergb

appergb commented Jun 26, 2026

Copy link
Copy Markdown
Owner

Closing as superseded by #155, which has been merged into main and includes this work in the integrated codebase. Keeping this PR closed avoids duplicate/conflicting review paths.

@appergb

appergb commented Jun 26, 2026

Copy link
Copy Markdown
Owner

Superseded by merged integration PR #155.

@appergb appergb closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

提取视频音频/音乐保存本地(媒体项星标导出)

2 participants