Framework 开源框架 @@ -162,8 +163,8 @@
Framework 开源框架
-RSIHub
++ +
++ Evolve your Codex on Terminal-Bench 2. + 在 Terminal-Bench 2 上进化你的 Codex。 +
Build agents that improve — and keep the evidence. 让智能体持续改进,并保留证据。 @@ -183,17 +216,6 @@
From e243abb869d158643c1d471fd952f16fd777d60c Mon Sep 17 00:00:00 2001
From: Octane
Framework
开源框架
@@ -162,8 +163,8 @@
RSIHub
让智能体持续改进,并保留证据。
- A file-based framework for evaluator-driven evolution, reproducible candidate lineage, and controlled self-modification. - 一个基于文件的框架,支持评估器驱动的进化、可复现的候选谱系,以及受控的自我修改。 + A file-based evolution framework for evaluator-driven learning, reproducible candidate lineage, and controllable modification. + 一个基于文件的进化框架,支持评估器驱动的学习、可复现的候选谱系,以及可控的修改。
+ The public composition model has four parts: + 公开的组合模型由四部分构成: +
+select, analyze, or mutate;
+ stage(阶段)是固定的生命周期槽位,例如 select、analyze、mutate;
+ library/<stage>/<name>.py;
+ operator(操作符)是位于 library/<stage>/<name>.py 的可复用实现;
+ + Add an operator to a source checkout, validate it, and compose it without a registry edit: + 在源码检出中新增一个操作符、验证它并完成组合,全程无需修改注册表: +
+uv run --frozen evolve operator new mutate my_operator
+uv run --frozen evolve operator describe mutate/my_operator
+uv run --frozen evolve operator check mutate/my_operator --config '{}'
+uv run --frozen evolve operator list mutate
+uv run --frozen evolve recipe check /path/to/my-recipe/evolve.yaml
+ + See the operator guide for the complete workflow. + 完整工作流见 operator 指南。 +
- Across the four-paper showcase, the deterministic completion pass rate moved from 1/4 at Gen 0 to 4/4 at Gen 2. The trials ran concurrently through Harbor’s local environment without Docker and retained ATIF trajectories plus evaluator-owned visual feedback. This is a representative evolution run rather than a broad benchmark. - 在四篇论文的 showcase 中,确定性完成通过率从 Gen 0 的 1/4 提升到 Gen 2 的 4/4。这些试验通过 Harbor 的本地环境并发运行(无需 Docker),并保留了 ATIF 轨迹与评估器持有的视觉反馈。这是一次代表性的进化运行,而非广泛的基准测试。 + Across the four-paper showcase, the deterministic completion pass rate moved from 1/4 at Gen 0 to 4/4 at Gen 2. The trials ran concurrently through Harbor’s local environment without Docker and retained ATIF trajectories plus evaluator-owned visual feedback. This is a representative evolution run rather than a broad benchmark; see the result snapshot, frozen rubric, and minimal seed Skill. + 在四篇论文的 showcase 中,确定性完成通过率从 Gen 0 的 1/4 提升到 Gen 2 的 4/4。这些试验通过 Harbor 的本地环境并发运行(无需 Docker),并保留了 ATIF 轨迹与评估器持有的视觉反馈。这是一次代表性的进化运行,而非广泛的基准测试;另见结果快照、冻结的评分标准与最小种子 Skill。
- Scores are shown as seed → best, with the absolute change underneath. All runs use a GPT-5.4-high target model and a GPT-5.4-xhigh Codex meta-agent. - 分数以 seed → best 呈现,括号中为绝对变化。所有运行使用 GPT-5.4-high 目标模型与 GPT-5.4-xhigh Codex meta-agent。 + Scores are percentages shown as seed → evolved agent, with the absolute change underneath. The train score is measured on the recipe’s training split; the full benchmark score is measured across the complete benchmark. All runs use a GPT-5.4-high target model and a GPT-5.4-xhigh Codex mutate operator. + 分数为百分比,以 seed → 进化后的智能体 呈现,括号中为绝对变化。Train 分数在 recipe 的训练划分上测得;完整基准分数在整个基准上测得。所有运行使用 GPT-5.4-high 目标模型与 GPT-5.4-xhigh 的 Codex mutate 操作符。
- Split: 50 train / 19 gate / 20 sealed. - 数据划分:50 train / 19 gate / 20 sealed。 -
| + Benchmark + 基准 + |
Target agent
目标智能体
@@ -478,156 +513,102 @@ Terminal Bench 2Method 方法 |
- Train | -Gate | -Sealed | -Overall | ++ Train score + Train 分数 + | ++ Full benchmark score + 完整基准分数 + | ||
|---|---|---|---|---|---|---|---|---|---|
| MiniSWE Agent | -AHE | -58.0% → 74.0% (+16.0%) |
- 57.9% → 68.4% (+10.5%) |
- 70.0% → 70.0% (+0.0%) |
- 60.7% → 71.9% (+11.2%) |
- ||||
| Hyperagents | -58.0% → 68.0% (+10.0%) |
- 57.9% → 73.7% (+15.8%) |
- 70.0% → 70.0% (+0.0%) |
- 60.7% → 69.7% (+9.0%) |
+ Terminal-Bench 2 50 train / 19 gate / 20 sealed |
+ MiniSWE | +AHE | +70.0% → 74.0% (+4.0%) |
+ 55.1% → 56.2% (+1.1%) |
| A Evolve | -58.0% → 68.0% (+10.0%) |
- 57.9% → 78.9% (+21.0%) |
- 70.0% → 65.0% (−5.0%) |
- 60.7% → 69.7% (+9.0%) |
+ Hyperagents | +58.0% → 68.0% (+10.0%) |
+ 55.1% → 68.5% (+13.4%) |
||
| GEPA | -58.0% → 68.0% (+10.0%) |
- 57.9% → 68.4% (+10.5%) |
- 70.0% → 75.0% (+5.0%) |
- 60.7% → 69.7% (+9.0%) |
+ A-Evolve | +66.0% → 68.0% (+2.0%) |
+ 55.1% → 65.2% (+10.1%) |
||
| Codex | -AHE | -58.0% → 74.0% (+16.0%) |
- 52.6% → 47.4% (−5.2%) |
- 65.0% → 70.0% (+5.0%) |
- 58.4% → 67.4% (+9.0%) |
+ GEPA | +58.0% → 68.0% (+10.0%) |
+ 55.1% → 59.6% (+4.5%) |
|
| Hyperagents | -58.0% → 72.0% (+14.0%) |
- 52.6% → 57.9% (+5.3%) |
- 65.0% → 75.0% (+10.0%) |
- 58.4% → 69.7% (+11.3%) |
+ Codex | +AHE | +60.0% → 74.0% (+14.0%) |
+ 60.7% → 66.3% (+5.6%) |
|
| A Evolve | -58.0% → 58.0% (+0.0%) |
- 52.6% → 52.6% (+0.0%) |
- 65.0% → 65.0% (+0.0%) |
- 58.4% → 58.4% (+0.0%) |
+ Hyperagents | +58.0% → 72.0% (+14.0%) |
+ 60.7% → 70.8% (+10.1%) |
||
| GEPA | -58.0% → 58.0% (+0.0%) |
- 52.6% → 52.6% (+0.0%) |
- 65.0% → 65.0% (+0.0%) |
- 58.4% → 58.4% (+0.0%) |
+ A-Evolve | +62.0% → 62.0% (0.0%) |
+ 60.7% → 64.0% (+3.3%) |
- Split: 50 train / 20 gate / 27 sealed. - 数据划分:50 train / 20 gate / 27 sealed。 -
-| - Target agent - 目标智能体 - | -- Method - 方法 - | -Train | -Gate | -Sealed | -Overall | +GEPA | +64.0% → 64.0% (0.0%) |
+ 60.7% → 60.7% (0.0%) |
||
|---|---|---|---|---|---|---|---|---|---|---|
| MiniSWE Agent | -AHE | -30.0% → 36.0% (+6.0%) |
- 35.0% → 35.0% (+0.0%) |
- 18.5% → 25.9% (+7.4%) |
- 27.8% → 33.0% (+5.2%) |
+ Tau³ Banking 50 train / 20 gate / 27 sealed |
+ MiniSWE | +AHE | +34.0% → 36.0% (+2.0%) |
+ 12.4% → 22.7% (+10.3%) |
| Hyperagents | -30.0% → 38.0% (+8.0%) |
- 35.0% → 45.0% (+10.0%) |
- 18.5% → 37.0% (+18.5%) |
- 27.8% → 39.2% (+11.4%) |
+ Hyperagents | +30.0% → 38.0% (+8.0%) |
+ 12.4% → 28.9% (+16.5%) |
|||
| A Evolve | -30.0% → 34.0% (+4.0%) |
- 35.0% → 45.0% (+10.0%) |
- 18.5% → 29.6% (+11.1%) |
- 27.8% → 35.1% (+7.3%) |
+ A-Evolve | +30.0% → 34.0% (+4.0%) |
+ 12.4% → 24.7% (+12.3%) |
|||
| GEPA | -30.0% → 32.0% (+2.0%) |
- 35.0% → 45.0% (+10.0%) |
- 18.5% → 25.9% (+7.4%) |
- 27.8% → 33.0% (+5.2%) |
+ GEPA | +30.0% → 32.0% (+2.0%) |
+ 12.4% → 22.7% (+10.3%) |
|||
| Codex | -AHE | -30.0% → 36.0% (+6.0%) |
- 30.0% → 45.0% (+15.0%) |
- 7.4% → 14.8% (+7.4%) |
- 23.7% → 32.0% (+8.3%) |
+ Codex | +AHE | +32.0% → 36.0% (+4.0%) |
+ 11.3% → 26.8% (+15.5%) |
|
| Hyperagents | -30.0% → 36.0% (+6.0%) |
- 30.0% → 50.0% (+20.0%) |
- 7.4% → 48.1% (+40.7%) |
- 23.7% → 42.3% (+18.6%) |
+ Hyperagents | +34.0% → 36.0% (+2.0%) |
+ 11.3% → 39.2% (+27.9%) |
|||
| A Evolve | -30.0% → 38.0% (+8.0%) |
- 30.0% → 45.0% (+15.0%) |
- 7.4% → 18.5% (+11.1%) |
- 23.7% → 34.0% (+10.3%) |
+ A-Evolve | +30.0% → 38.0% (+8.0%) |
+ 11.3% → 13.4% (+2.1%) |
|||
| GEPA | -30.0% → 36.0% (+6.0%) |
- 30.0% → 35.0% (+5.0%) |
- 7.4% → 14.8% (+7.4%) |
- 23.7% → 29.9% (+6.2%) |
+ GEPA | +10.0% → 16.0% (+6.0%) |
+ 11.3% → 15.5% (+4.2%) |
Framework 开源框架
-+ Evolve your Codex on Terminal-Bench 2. + 在 Terminal-Bench 2 上进化你的 Codex。 +
Build agents that improve — and keep the evidence. 让智能体持续改进,并保留证据。 @@ -183,17 +216,6 @@