Skip to content

Rename eval_runner.py to experiment_runner.py#896

Merged
cvolkcvolk merged 3 commits into
mainfrom
cvolk/refactor/arena-experiment-runner
Jul 14, 2026
Merged

Rename eval_runner.py to experiment_runner.py#896
cvolkcvolk merged 3 commits into
mainfrom
cvolk/refactor/arena-experiment-runner

Conversation

@cvolkcvolk

@cvolkcvolk cvolkcvolk commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Align runner naming with Experiments. Renames eval_runner.py to experiment_runner.py because its unit of work is one ArenaExperiment: an ordered list[ArenaRunCfg].

Detailed description

  • Uses Experiment Runner for the already-namespaced component while retaining the Arena prefix on public domain types.
  • Updates Python APIs, helpers, tests, and documentation without retaining a compatibility wrapper or changing execution behavior.

Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk cvolkcvolk changed the title Rename Eval Runner to Arena Experiment Runner Rename eval_runner.py to arena_experiment_runner.py Jul 13, 2026
@cvolkcvolk
cvolkcvolk marked this pull request as ready for review July 13, 2026 18:02
@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR renames the batch evaluation runner to match Arena Experiment terminology. The main changes are:

  • eval_runner.py and related helper modules renamed to arena_experiment_runner.py names.
  • Runner imports, CLI parser names, tests, and subprocess helpers updated to the new names.
  • Documentation examples and Sphinx references updated to use arena_experiment_runner.py.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
isaaclab_arena/evaluation/arena_experiment_runner.py Renamed the main runner module and updated imports for the renamed CLI and legacy helpers.
isaaclab_arena/evaluation/arena_experiment_runner_cli.py Renamed the runner-specific CLI helpers while keeping the argument behavior unchanged.
isaaclab_arena/evaluation/legacy_arena_experiment_runner.py Renamed the legacy JSON chunking helper and updated its runner-facing log text.
isaaclab_arena/tests/test_arena_experiment_runner.py Updated runner tests and helpers to execute the renamed script path.
docs/pages/concepts/policy/concept_evaluation_types.rst Updated the concept docs, runner filename references, and batch-runner Sphinx label.

Reviews (1): Last reviewed commit: "Rename Eval Runner to Arena Experiment R..." | Re-trigger Greptile

@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

This PR renames eval_runner.pyarena_experiment_runner.py (plus its _cli and legacy_ helpers, their functions, tests, and docs) to align naming with the Arena Experiment concept — one experiment being an ordered list[ArenaRunCfg]. It is a pure rename with no behavior change.

I fetched the full tree at the PR head and verified the rename is complete and consistent: every old symbol (parse_eval_runner_args, add_eval_runner_arguments, run_eval_runner, legacy_eval_runner, test_eval_runner) and the old evaluation/eval_runner.py script path is gone, with no dangling references anywhere in code, docs, CI, or Docker. The renamed RST cross-reference anchor (sequential-batch-eval-runnersequential-batch-arena-experiment-runner) is updated at both its definition and its single :ref:, so the docs build stays intact. The unrelated legacy_eval_config.py / legacy_environment_cli_args.py modules are correctly left untouched.

Findings

None. The renamed files are git renames of pre-existing files, so their 2025-2026 copyright headers are correct.

Test Coverage

Tests were renamed alongside the module and keep their @pytest.mark.with_subprocess markers; test_all_environments.py imports the renamed run_arena_experiment_runner / helpers correctly, and test_arena_experiment_config_loader.py monkeypatches the renamed arena_experiment_runner module. No new logic is introduced, so no new tests are warranted.

Verdict

Ship it.

Comment thread docs/pages/concepts/policy/concept_evaluation_types.rst Outdated
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk cvolkcvolk changed the title Rename eval_runner.py to arena_experiment_runner.py Rename eval_runner.py to experiment_runner.py Jul 14, 2026
Signed-off-by: Clemens Volk <cvolk@nvidia.com>

@alexmillane alexmillane left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes a ton of sense. LGTM

@cvolkcvolk
cvolkcvolk merged commit e34c66a into main Jul 14, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants