Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
4b27a47
feat(tts): 视频管线新增「激情」配音风格预设与官方站点信源补充规范;
ThreeFish-AI Aug 17, 2026
bb0ffd6
feat(video): 《会写代码的 AI,开始给自己写代码》论文深读校准与动画升级(v3);
ThreeFish-AI Aug 17, 2026
aa7304c
feat(video): 《AI 如何自己变强?》官方站点信源补充与收尾活地图镜(v3);
ThreeFish-AI Aug 17, 2026
8805194
feat(video): 《上线之后,AI 才开始上学》经验编译器闭环与三代史升级(v2);
ThreeFish-AI Aug 17, 2026
fe3af2f
feat(tts): IndexTTS 推理新增束搜索宽度旋钮并将管线默认降为 1 束(MPS 长跑提速);
ThreeFish-AI Aug 17, 2026
11f5be7
feat(video): 三集 Remotion 场景新增信源镜与动画升级(站点概念代码化重建);
ThreeFish-AI Aug 17, 2026
8be6bb1
feat(video): 《AI 如何自己变强?》出题循环图升级流光巡游与游标辉光;
ThreeFish-AI Aug 17, 2026
04bf703
docs(video): 三集克隆配音说明与知识地图同步;
ThreeFish-AI Aug 17, 2026
89b239e
fix(video): 《上线之后,AI 才开始上学》1-G beat 窗口引用笔误修正(p1-26a→p1-26);
ThreeFish-AI Aug 17, 2026
3250a63
fix(tts): IndexTTS 缓存摘要纳入 num_beams 且实测 RTF 口径统一;
ThreeFish-AI Aug 18, 2026
13b90a7
fix(video): 三集升级分支六条评审意见修复(坐标/笔误/死代码);
ThreeFish-AI Aug 18, 2026
9f7890a
chore(video): 合入 feature/1.x.x 解决 squash 后分支与基线冲突;
ThreeFish-AI Aug 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion media/experience-era-agents-video/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> 基于 Che Jiang, Jincheng Zhong, Yu Fu, *et al.*, "Self-Improving Agents in the Era of Experience: A Survey of Self- to Meta-Evolution," *Frontis.AI / Tsinghua University*, Jun. 2026(88 页综述,无公开 arXiv 号)的动效图解式科普视频(B 站/YouTube,约 13.6 分钟)。
> 形态:**本人音色克隆配音**(IndexTTS-2.5,passionate 激情风格)+ Remotion 代码动画,无真人出镜。
> (v3 前为 edge-tts 预置音色,两种引擎 manifest 契约一致,可随时切回。)
> (v2 前为 edge-tts 预置音色,两种引擎 manifest 契约一致,可随时切回。)
> 与上一集《AI 如何自己变强?》([../self-improving-agents-video/](../self-improving-agents-video/README.md),Schmidhuber 团队综述)互补:上集讲「自我进化改什么」,本集讲「部署之后经验怎么攒」;片尾互相引用。

## 目录结构
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,4 @@ RISKY = 0,REWRITE = 0。✓

- `build_narration.py` 重建通过(实测见执行日志)。
- 未触碰句 text 字节稳定:build 前后公共 id(172 个)text 全等 ✓。
- storyboard 引用 id ⊆ narration id 集 ✓(1-G 扩 ..26a;P6 重排为 6-B/6-B2/6-C/6-D/6-E/6-F/6-G)。
- storyboard 引用 id ⊆ narration id 集 ✓(1-G 扩至 p1-26、新增句为 p1-25a;P6 重排为 6-B/6-B2/6-C/6-D/6-E/6-F/6-G)。
2 changes: 1 addition & 1 deletion media/experience-era-agents-video/script/storyboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
| 1-D 便宜洞察 | p1-10..12 | 分屏对比:左侧改大脑=重型机械(慢/贵/锁定),右侧改工位=乐高快拆(青) | 分屏滑入 |
| 1-E 原油汽油 | p1-13..18 | 金色提纯漏斗:左侧黑浊「trace 原始流水」倒入,四道工序环(过滤/压缩/归因/验证),滴出金色「经验 z」;角标 z_i=H(τ_i) | 漏斗流动 + 工序环逐个点亮 |
| 1-F 快慢去路 | p1-19..21 | 金色经验球到分岔:上路青色箭头「改工位·快」直通工位;下路紫色箭头「写大脑·慢」缓慢沉入大脑 | 分岔流动 + 差速 |
| 1-G 三代史 | p1-22..26a | 横向时间轴 2021→2026:Gen1 任务环(图标:单环)→ Gen2 复用(图标:书架)→ Gen3 运行时(图标:齿轮房);p1-25a 时 Gen3 齿轮房上的「可改接口」小面板逐个亮起(适应面自进化);时间轴下方里程碑名(ReAct/Voyager/Claude Code) | 时间轴推进 + 图标弹跳 + Gen3 定格放大 + 接口面板点亮 |
| 1-G 三代史 | p1-22..26 | 横向时间轴 2021→2026:Gen1 任务环(图标:单环)→ Gen2 复用(图标:书架)→ Gen3 运行时(图标:齿轮房);p1-25a 时 Gen3 齿轮房上的「可改接口」小面板逐个亮起(适应面自进化);时间轴下方里程碑名(ReAct/Voyager/Claude Code) | 时间轴推进 + 图标弹跳 + Gen3 定格放大 + 接口面板点亮 |
| 1-H 转场钩子 | p1-27..28 | 四条管道分叉预告图(金流分四路) | 管道延伸 |

## P2 经验的四个去处(约 4.5 分钟)
Expand Down
4 changes: 2 additions & 2 deletions media/pipeline/VOICE-CLONING.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ uv run --no-project --with soundfile --with numpy \

### 4.3b 束搜索宽度(--num-beams,速度主旋钮)

GPT 声码段的束搜索宽度,默认 **1**(上游库内部默认 3)。采样生成(`do_sample=True`)下 1 与 3 的听感差异可忽略,但 GPT 段耗时约按束宽线性放大。**MPS fp32 实测**(2026-08-18,M3 系列):每句墙钟 = GPT 束搜索 + 扩散声码 + BigVGAN,RTF(耗时/音频时长)约 40–58(beams=3)与 20–30(beams=1;整集(约 180–230 句)预计数小时,按句缓存可断点续跑。质量敏感的单句可 `--num-beams 3` 单独重合成。
GPT 声码段的束搜索宽度,默认 **1**(上游库内部默认 3)。采样生成(`do_sample=True`)下 1 与 3 的听感差异可忽略,但 GPT 段耗时约按束宽线性放大。**MPS fp32 实测**(2026-08-18,M3 系列):每句墙钟 = GPT 束搜索 + 扩散声码 + BigVGAN,RTF(耗时/音频时长)约 40–58(beams=3)beams=1 下三集全量连续跑(596 句 / 40.2 分钟纯语音 / 8.5 小时墙钟)折算整集 RTF≈12–14;整集(约 180–230 句)约 2.5–3.5 小时,按句缓存可断点续跑。质量敏感的单句可 `--num-beams 3` 单独重合成。

### 4.4 调参建议

Expand All @@ -159,7 +159,7 @@ cd video && pnpm run render:draft && pnpm run render
```

- 服务启动一次可服务多集;管线客户端不常驻模型;
- 每句墙钟与文本长度及束宽相关:MPS fp32 实测 RTF≈40–58(`--num-beams 3`)/ ≈20–30(默认 `--num-beams 1`),即 5 秒的句子约需 2–5 分钟;整集(约 180–230 句)预计 6–10 小时,建议 `nohup` 挂后台跑、按句缓存断点续跑(见 §六);长篇管线**不要**用默认 3 束逐句等待;
- 每句墙钟与文本长度及束宽相关:MPS fp32 实测 RTF≈40–58(`--num-beams 3`)/ ≈12–14(默认 `--num-beams 1`),即 5 秒的句子约需 1–1.5 分钟;整集(约 180–230 句)约 2.5–3.5 小时,建议 `nohup` 挂后台跑、按句缓存断点续跑(见 §六);长篇管线**不要**用默认 3 束逐句等待;
- 引擎/风格/样本任一变化都会改写时长,合成后**必须重跑草渲**让时间轴重算;
- 超长句(>120 token)服务端内部自动分段;极端长句推理可达数分钟。客户端并发为 1(与服务端串行推理对齐,避免排队时间计入超时),HTTP 超时 600s;万一超时——重跑即续传,无需干预。

Expand Down
7 changes: 5 additions & 2 deletions media/pipeline/scripts/tts.py
Original file line number Diff line number Diff line change
Expand Up @@ -282,10 +282,13 @@ def digest_indextts(
lang: str,
engine_tag: str,
text: str,
num_beams: int = 1,
) -> str:
vec_str = ",".join(repr(x) for x in vec) if vec else "none"
# 束宽改变合成结果,须入键;默认 1 时省略字段——沿用历史摘要格式,存量缓存不失效
beams_part = "" if num_beams == 1 else f"|beams={num_beams}"
return hashlib.sha1(
f"indextts|{engine_tag}|{ref_sha1}|{lang}|{style}|{vec_str}|{alpha!r}|{df!r}|{text}".encode()
f"indextts|{engine_tag}|{ref_sha1}|{lang}|{style}|{vec_str}|{alpha!r}|{df!r}|{text}{beams_part}".encode()
).hexdigest()


Expand All @@ -308,7 +311,7 @@ async def synth_indextts(
sid, text = item["id"], item["text"]
mp3 = out_dir / f"{sid}.mp3"
meta = out_dir / f"{sid}.sha"
digest = digest_indextts(ref_sha1, style, vec, alpha, df, lang, engine_tag, text)
digest = digest_indextts(ref_sha1, style, vec, alpha, df, lang, engine_tag, text, num_beams)

if (
not force
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -331,8 +331,8 @@ const Matrix3x3: React.FC<{diagonalFrom: number}> = ({diagonalFrom}) => {
<line
x1={180 + 10 + 250 * 0.5}
y1={70 + 10 + 170 * 0.5}
x2={180 + 10 + 250 * 2.5}
y2={70 + 10 + 170 * 2.5}
x2={180 + 10 + 2 * (250 + 10) + 250 * 0.5}
y2={70 + 10 + 2 * (170 + 10) + 170 * 0.5}
stroke={theme.ok}
strokeWidth={10}
strokeLinecap="round"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -987,9 +987,9 @@ const LivingMap: React.FC<{countAt: number; ringAt: number; insightAt: number}>
const frame = useCurrentFrame();
const {fps} = useVideoConfig();
const bars = [
{label: '改大脑 θ', en: 'FM improvement', n: 77, color: theme.brain, deep: theme.brainDeep},
{label: '改装备 Σ', en: 'Scaffolding improvement', n: 176, color: theme.gear, deep: theme.gearDeep},
{label: '评测', en: 'Evaluation & Benchmarking', n: 59, color: theme.dim, deep: '#2A2F3A'},
{label: '改大脑 θ', en: 'FM improvement', n: 77, color: theme.brain},
{label: '改装备 Σ', en: 'Scaffolding improvement', n: 176, color: theme.gear},
{label: '评测', en: 'Evaluation & Benchmarking', n: 59, color: theme.dim},
];
const total = 312;
const grow = ci(frame, countAt, countAt + 42);
Expand All @@ -998,7 +998,6 @@ const LivingMap: React.FC<{countAt: number; ringAt: number; insightAt: number}>
const orangePulse = insight > 0 ? 0.85 + 0.15 * Math.sin(frame * 0.1) : 1;
// 环形图几何(viewBox 360x360,中心 180,180,半径 110;蓝从顶部顺时针、橙紧随、灰收尾)
const R = 110;
const C = 2 * Math.PI * R;
const segs = [77 / total, 176 / total, 59 / total];
const offsets = [0, segs[0], segs[0] + segs[1]];
const enters = spring({frame, fps, config: {damping: 200}});
Expand Down