From ff427a2badacb60c7b44b12883b77eb9337082bb Mon Sep 17 00:00:00 2001 From: liujiyuan Date: Tue, 4 Aug 2026 00:34:02 +0800 Subject: [PATCH] docs: update dashboard documentation --- README.md | 5 +- README.zh-CN.md | 5 +- .../rst_source/development/add_primitive.rst | 3 +- .../rst_source/development/add_robot.rst | 26 ++++++---- .../rst_source/development/architecture.rst | 52 +++++++++---------- .../rst_source/development/interfaces.rst | 9 ++-- docs/source-en/rst_source/quickstart.rst | 13 ++--- .../rst_source/usage/advanced_deployment.rst | 4 ++ docs/source-en/rst_source/usage/libero.rst | 27 +++++++--- .../rst_source/development/add_primitive.rst | 3 +- .../rst_source/development/add_robot.rst | 29 ++++++----- .../rst_source/development/architecture.rst | 39 +++++++------- .../rst_source/development/interfaces.rst | 10 ++-- docs/source-zh/rst_source/quickstart.rst | 8 ++- .../rst_source/usage/advanced_deployment.rst | 3 ++ docs/source-zh/rst_source/usage/libero.rst | 23 ++++++-- 16 files changed, 157 insertions(+), 102 deletions(-) diff --git a/README.md b/README.md index 1107daee..83c23b5e 100644 --- a/README.md +++ b/README.md @@ -139,11 +139,10 @@ rpent --env libero --suite libero_object_swap --task 2 --seed 0 \ ### Live Dashboard -Add `--dashboard` to start a local dashboard server. The command prints the URL in the terminal; open it to confirm the configuration on the launcher screen. Once the run starts, the page streams agent reasoning, camera and Pi0 views, the action timeline, and clip replays. Use `--dashboard-language zh-cn` for the Chinese UI. +Add `--dashboard` to start a local Dashboard and print its URL in the terminal. Open the URL and confirm the configuration; once the services are ready, start a task with `/rpent-task `. The page streams agent reasoning, camera views, and the action timeline, and you can submit another task after the current one finishes. Use `--dashboard-language zh-cn` for the Chinese UI. ```bash rpent --env libero --dashboard --dashboard-language zh-cn \ - --suite libero_goal_task --task 1 --seed 0 \ --planner claude_code --model claude-opus-4-8 ``` @@ -172,7 +171,7 @@ For more detailed documentation, see the [RPent documentation](https://rpent.rea --max-episode-steps10000Max env steps --libero-typeLIBERO_TYPE or proLIBERO variant: standard | pro | plus --cuda-deviceinheritedGPU device exposed to the env / VLA / SAM3 servers - --dashboardoffStart the local dashboard for this run + --dashboardoffStart a local Dashboard --dashboard-languageenDashboard UI language: en | zh-cn --env-endpoint— (spawn)[protocol://]host:port of an existing env_server (protocol=http|socket, default http). If unset, one is spawned locally. --vla-endpoint— (spawn)[protocol://]host:port of an existing vla_server (same rules). If unset, one is spawned locally. diff --git a/README.zh-CN.md b/README.zh-CN.md index a8e30efa..e9cbe0d0 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -140,11 +140,10 @@ rpent --env libero --suite libero_object_swap --task 2 --seed 0 \ ### 实时 Dashboard -加上 `--dashboard` 后，会启动本地监控服务，并在终端输出访问地址。打开该地址后，可以在启动页面确认配置；运行开始后，页面会实时显示智能体的推理过程、相机画面和动作时间线。使用 `--dashboard-language zh-cn` 可切换到中文界面。 +加上 `--dashboard` 后，会启动本地 Dashboard，并在终端输出访问地址。打开该地址并确认配置；服务就绪后，通过 `/rpent-task ` 启动任务。页面会实时显示智能体的推理过程、相机画面和动作时间线，任务结束后可以继续提交下一任务。使用 `--dashboard-language zh-cn` 可切换到中文界面。 ```bash rpent --env libero --dashboard --dashboard-language zh-cn \ - --suite libero_goal_task --task 1 --seed 0 \ --planner claude_code --model claude-opus-4-8 ``` @@ -184,7 +183,7 @@ bash scripts/run_robocasa.sh PickPlaceCounterToCabinet 0 0 # <任务> < --max-episode-steps10000环境最大步数 --libero-typeLIBERO_TYPE 或 proLIBERO 类型：standard | pro | plus --cuda-device继承当前环境env_server、vla_server 和 sam3_server 可见的 GPU 设备 - --dashboard关为本次运行启动本地 Dashboard + --dashboard关启动本地 Dashboard --dashboard-languageenDashboard 界面语言：en | zh-cn --env-endpoint—（自动启动）已在运行的 env_server 的 [protocol://]host:port（protocol=http|socket，默认 http）。留空时自动启动本地实例。 --vla-endpoint—（自动启动）已在运行的 vla_server 的 [protocol://]host:port（同上）。留空时自动启动本地实例。 diff --git a/docs/source-en/rst_source/development/add_primitive.rst b/docs/source-en/rst_source/development/add_primitive.rst index d65d920d..03ce0a97 100644 --- a/docs/source-en/rst_source/development/add_primitive.rst +++ b/docs/source-en/rst_source/development/add_primitive.rst @@ -128,10 +128,11 @@ primitive requires a few additional components: .. code-block:: python - def get_toolkit(*, primitives_kwargs, video_path=None): + def get_toolkit(*, primitives_kwargs, dashboard_events, video_path=None): from robots.myrobot.toolkit import MyRobotToolkit return MyRobotToolkit( primitives_kwargs=primitives_kwargs, + dashboard_events=dashboard_events, video_path=video_path, ) diff --git a/docs/source-en/rst_source/development/add_robot.rst b/docs/source-en/rst_source/development/add_robot.rst index 92342076..d9fd2b90 100644 --- a/docs/source-en/rst_source/development/add_robot.rst +++ b/docs/source-en/rst_source/development/add_robot.rst @@ -49,6 +49,7 @@ two factory functions: .. code-block:: python # robots/myenv/__init__.py + from rpent.dashboard.events import DashboardEventSink from rpent.envs.env_spec import EnvSpec, RunConfig from rpent.envs.prompt_bundle import PromptBundle from robots.myenv.prompt_bundle import system_prompt, user_prompt @@ -62,9 +63,13 @@ two factory functions: init_runtime=_init_runtime, ) - def get_toolkit(*, primitives_kwargs, video_path=None): + def get_toolkit(*, primitives_kwargs, dashboard_events: DashboardEventSink, video_path=None): from robots.myenv.toolkit import MyEnvToolkit - return MyEnvToolkit(primitives_kwargs=primitives_kwargs, video_path=video_path) + return MyEnvToolkit( + primitives_kwargs=primitives_kwargs, + dashboard_events=dashboard_events, + video_path=video_path, + ) def _add_cli_args(parser, use_dashboard) -> None: """Register env flags on the shared parser. See §4.""" @@ -74,7 +79,7 @@ two factory functions: """Validate final `args`, return a RunConfig. See §4.""" ... - def _init_runtime(args, output_dir): + def _init_runtime(args, output_dir, dashboard_events: DashboardEventSink): """Spawn env_server, vla_server, and any supporting services. Returns (daemons, primitives_kwargs). See §5. @@ -268,13 +273,15 @@ hooks and participate in the final argparse pass: **``_add_cli_args(parser, use_dashboard) -> None``.** Register the environment's arguments on the shared parser created by main.py. ``use_dashboard`` determines whether normally required arguments remain -optional for the dashboard launcher to fill in later. main.py calls this hook +optional. For each Dashboard TaskRun, the ``/rpent-task`` command supplies +``suite`` and ``task`` before ``parse_config`` runs. main.py calls this hook before ``parser.parse_args()``, so argparse's usage and error output includes the environment arguments. -**``_parse_config(args) -> RunConfig``.** Called after ``parser.parse_args()`` -and, if applicable, the dashboard launcher. Enforces any dashboard-only -optional flags are now populated and returns a +**``_parse_config(args) -> RunConfig``.** In normal CLI mode, this is called +after ``parser.parse_args()``. In Dashboard mode, it is called for each +TaskRun after the ``/rpent-task`` command's ``suite`` and ``task`` have been +copied to the task arguments. It validates those fields and returns a :class:`~rpent.envs.RunConfig`: - ``recipe_tag`` — env's per-run tag, used in transcript filenames / recipe @@ -283,8 +290,6 @@ optional flags are now populated and returns a calls ``init_output_dir`` to create it and configure logging). - ``prompt_vars`` — dict passed to ``PromptBundle.render`` (typically the run identifiers plus anything else the prompts reference). -- ``dashboard_state`` — a :class:`~rpent.dashboard.state.State` when - ``args.dashboard`` is set, else ``None``. - ``task_desc`` — env-specific dict of task-identifying fields, written into the transcript JSON record verbatim (LIBERO: ``{"suite": ..., "task": ..., "seed": ...}``). @@ -299,12 +304,11 @@ optional flags are now populated and returns a def _parse_config(args) -> RunConfig: if not args.suite: raise ValueError("--suite is required") - # ... derive recipe_tag, output_dir, prompt_vars, dashboard_state ... + # ... derive recipe_tag, output_dir, and prompt_vars ... return RunConfig( recipe_tag=recipe_tag, output_dir=output_dir, prompt_vars=prompt_vars, - dashboard_state=dashboard_state, task_desc={"suite": args.suite, "task": args.task, "seed": args.seed}, ) diff --git a/docs/source-en/rst_source/development/architecture.rst b/docs/source-en/rst_source/development/architecture.rst index 147469cd..e5b18a9e 100644 --- a/docs/source-en/rst_source/development/architecture.rst +++ b/docs/source-en/rst_source/development/architecture.rst @@ -101,31 +101,29 @@ components required for a run. On startup, it: 2. Resolves the env via ``get_env_spec(args.env_name)`` and calls ``env_spec.add_cli_args(parser, use_dashboard=args.dashboard)`` — the env registers its flags on the shared parser. ``use_dashboard=True`` makes - its otherwise-required flags optional so the dashboard can supply them. + task-specific flags optional because the Dashboard receives them later + through task commands. 3. Runs ``parser.parse_args()`` against the complete parser to perform argparse-level validation and produce the final ``args``, retaining argparse's standard usage and error output. 4. If ``--dashboard`` is set, starts the launcher with the current arguments as defaults and applies the submitted configuration back to ``args``. 5. Calls ``env_spec.parse_config(args)`` to validate the run configuration - and produce a - :class:`~rpent.envs.RunConfig` - (``recipe_tag`` / ``output_dir`` / ``prompt_vars`` / ``dashboard_state`` - / ``task_desc``). Under ``--dashboard``, this is where the env - enforces that its previously-optional flags were actually filled in. + and produce a :class:`~rpent.envs.RunConfig` (``recipe_tag`` / + ``output_dir`` / ``prompt_vars`` / ``task_desc``). 6. Calls ``init_output_dir`` to create the run's output directory and configure ``run.log``. 7. Builds the **planner** through ``rpent.planner.base.build_planner`` based on ``--planner``, then renders the system and user prompts from the env's prompt bundle. -8. Calls ``env_spec.init_runtime(args, output_dir)``. The env implementation - starts ``env_server`` and ``vla_server``, or connects to existing services - when ``--env-endpoint`` / ``--vla-endpoint`` is supplied, and returns - ``(daemons, primitives_kwargs)``. -9. Passes ``primitives_kwargs`` to the env's ``get_toolkit`` factory to - construct the **toolkit**. -10. Runs the tool-calling loop, streams to the dashboard if - ``--dashboard`` is set, and then writes +8. Calls ``env_spec.init_runtime(args, output_dir, dashboard_events)``. The env + implementation starts ``env_server``, ``vla_server``, and ``sam3_server``, or connects + to existing services when the corresponding endpoint is supplied, and + returns ``(daemons, primitives_kwargs)``. +9. Passes ``primitives_kwargs`` and a ``dashboard_events`` sink to the env's + ``get_toolkit`` factory to construct the **toolkit**. The one-shot path + uses a no-op event sink. +10. Runs the tool-calling loop, then writes ``/transcript_*.json`` and flushes toolkit recordings during cleanup. @@ -147,7 +145,7 @@ two factories exposed by that package: # robots/myenv/__init__.py def get_env_spec() -> EnvSpec: ... # identity, prompt bundle, and runner hooks def get_toolkit( - *, primitives_kwargs, video_path=None, dashboard=None + *, primitives_kwargs, dashboard_events, video_path=None ): ... ``EnvSpec`` gathers the environment's identity, its prompt templates, and the @@ -178,23 +176,25 @@ Dashboard (optional) -------------------- ``rpent/dashboard/`` contains a FastAPI application and a static -frontend. With ``--dashboard``, ``rpent/cli/main.py`` starts the -Dashboard using ``--dashboard-host`` and ``--dashboard-port``. It binds -to ``127.0.0.1`` by default and lets the operating system choose a free -port. Before the run starts, the launcher lets the user review or change -the configuration. +frontend. With ``--dashboard``, ``rpent/cli/main.py`` hands control to +``rpent/cli/dashboard.py``, which starts the Dashboard with +``--dashboard-host`` and ``--dashboard-port`` and confirms the configuration +before shared services start. VLA and SAM3 are reused while the Dashboard is +running; tasks submitted with ``/rpent-task`` use separate environment runtimes +and execute sequentially. -During the run, the Dashboard shows: +During a TaskRun, the Dashboard shows: - planner output and tool-call events; -- live camera and Pi0.5 views; +- live fixed-camera and wrist-camera views; - the action timeline and per-action clips; - the complete episode recording after the run, if one was generated. -The server sends state summaries over SSE, and the frontend fetches -detailed events, timeline data, and images as needed. The Dashboard -displays state produced by the planner and toolkit; it does not issue -robot actions directly. +The page accepts ordinary planner messages, new task commands, and interrupt +requests, but these controls do not issue robot actions directly. Planners, +toolkits, and environment runtimes publish display updates through a +``dashboard_events`` sink. The server sends state summaries over SSE, and the +frontend fetches detailed events, timeline data, and images as needed. Next steps ---------- diff --git a/docs/source-en/rst_source/development/interfaces.rst b/docs/source-en/rst_source/development/interfaces.rst index 0a96e2b0..a2c3f723 100644 --- a/docs/source-en/rst_source/development/interfaces.rst +++ b/docs/source-en/rst_source/development/interfaces.rst @@ -12,7 +12,7 @@ After you add ``robots//``, ``main.py`` calls two functions in ``__init__.p .. code-block:: python def get_env_spec() -> EnvSpec: ... - def get_toolkit(*, primitives_kwargs, video_path=None, dashboard=None): ... + def get_toolkit(*, primitives_kwargs, dashboard_events: DashboardEventSink, video_path=None): ... ``get_env_spec`` returns an ``EnvSpec``. You supply: @@ -34,10 +34,12 @@ After you add ``robots//``, ``main.py`` calls two functions in ``__init__.p ``output_dir``, and ``prompt_vars`` for prompt templating. * - ``init_runtime`` - Start or attach to env / VLA subprocesses; build ``primitives_kwargs`` - (env client, model client, etc.) for the toolkit's primitive driver. + (env client, model client, etc.) for the toolkit's primitive driver. A + ``DashboardEventSink`` reports runtime status. ``get_toolkit`` usually just passes ``primitives_kwargs`` into your env subclass; -``video_path`` and ``dashboard`` are passed by ``main.py`` — you rarely touch them. +``dashboard_events`` and ``video_path`` are supplied by ``main.py``, so you +normally do not need to change them. Reference: ``robots/libero/__init__.py``. @@ -58,6 +60,7 @@ Most users pick a built-in ``api``, ``claude_code``, or ``codex`` planner — se toolkit: Toolkit, max_turns: int, input_queue=None, + dashboard_interaction=None, ) -> PlannerResult: ... Contract: pass ``toolkit.get_tools_spec()`` to the model; dispatch each call via diff --git a/docs/source-en/rst_source/quickstart.rst b/docs/source-en/rst_source/quickstart.rst index b9ba30b5..a57f8446 100644 --- a/docs/source-en/rst_source/quickstart.rst +++ b/docs/source-en/rst_source/quickstart.rst @@ -43,18 +43,19 @@ To switch to another planner, such as ``codex`` or ``api``, see 3. Monitor the run in the Dashboard ----------------------------------- -Add ``--dashboard`` to start a local Dashboard service and print its URL -in the terminal. Open the URL to confirm the configuration on the -launcher screen. Once the run starts, the page streams the agent's -reasoning, live camera and Pi0 views, an action timeline, and clip -replays. Use ``--dashboard-language zh-cn`` for the Chinese UI. +Add ``--dashboard`` to start a local Dashboard and print its URL in the terminal: .. code-block:: bash rpent --env libero --dashboard --dashboard-language zh-cn \ - --suite libero_object_swap --task 2 --seed 0 \ --planner claude_code --model claude-opus-4-8 +Open the URL and confirm the configuration. Once the services are ready, enter +``/rpent-task libero_object_swap 2 0`` in the page to start a task. The Dashboard +streams agent reasoning, camera views, and the action timeline; submit another +task after the current one finishes. Use ``--dashboard-language zh-cn`` for the +Chinese UI. + Key CLI options --------------- diff --git a/docs/source-en/rst_source/usage/advanced_deployment.rst b/docs/source-en/rst_source/usage/advanced_deployment.rst index 78d699b9..5c265e1b 100644 --- a/docs/source-en/rst_source/usage/advanced_deployment.rst +++ b/docs/source-en/rst_source/usage/advanced_deployment.rst @@ -11,6 +11,10 @@ environment, ``--vla-endpoint`` for the Pi0.5 VLA, and ``--sam3-endpoint`` for SAM3. Each takes ``[protocol://]HOST:PORT`` — HTTP when the protocol is omitted, or ``socket://`` for socket RPC. +Dashboard Sessions do not support ``--env-endpoint`` because every TaskRun +uses a fresh environment service. ``--vla-endpoint`` and ``--sam3-endpoint`` +remain available in Dashboard mode. + LIBERO environment service -------------------------- diff --git a/docs/source-en/rst_source/usage/libero.rst b/docs/source-en/rst_source/usage/libero.rst index 4fd25442..0e25f106 100644 --- a/docs/source-en/rst_source/usage/libero.rst +++ b/docs/source-en/rst_source/usage/libero.rst @@ -156,18 +156,33 @@ These tools do not advance the environment. Live dashboard -------------- -Add ``--dashboard`` to start a local monitor. It selects an available -port and prints the URL in the terminal: +Add ``--dashboard`` to start a long-lived local Dashboard Session. It +selects an available port and prints the URL in the terminal: .. code-block:: bash rpent --env libero --dashboard \ - --suite libero_object_swap --task 2 --seed 0 \ --planner claude_code --model claude-opus-4-8 -The dashboard streams reasoning, agentview + wrist camera + Pi0.5 -overlays, and an action timeline. Use -``--dashboard-language zh-cn`` for the Chinese UI. +Open the URL, confirm the Session configuration, and click **Start Session**. +After the shared services are ready, start a TaskRun from the page with: + +.. code-block:: text + + /rpent-task libero_object_swap 2 0 + +Each TaskRun gets a fresh environment while the VLA and SAM3 services are +reused by the Session. Submit another ``/rpent-task`` after completion, or +submit one while a task is running to switch tasks. The Dashboard streams +agent reasoning and tool calls, fixed and wrist camera views, the action +timeline, and action and full-episode replays. During a TaskRun, normal +messages steer the agent and Esc requests an interruption. Press Ctrl+C in +the terminal to stop the Session. + +``--dashboard`` cannot be combined with ``--interactive`` or +``--env-endpoint``. External ``--vla-endpoint`` and ``--sam3-endpoint`` +services remain supported. Use ``--dashboard-language zh-cn`` for the +Chinese UI. Bringing your own VLA --------------------- diff --git a/docs/source-zh/rst_source/development/add_primitive.rst b/docs/source-zh/rst_source/development/add_primitive.rst index 9ebe6550..3adb28b3 100644 --- a/docs/source-zh/rst_source/development/add_primitive.rst +++ b/docs/source-zh/rst_source/development/add_primitive.rst @@ -116,10 +116,11 @@ primitive driver 方法，以及调用完成后的状态快照。区别仅在于 .. code-block:: python - def get_toolkit(*, primitives_kwargs, video_path=None): + def get_toolkit(*, primitives_kwargs, dashboard_events, video_path=None): from robots.myrobot.toolkit import MyRobotToolkit return MyRobotToolkit( primitives_kwargs=primitives_kwargs, + dashboard_events=dashboard_events, video_path=video_path, ) diff --git a/docs/source-zh/rst_source/development/add_robot.rst b/docs/source-zh/rst_source/development/add_robot.rst index 0e97a53e..d5e22ea5 100644 --- a/docs/source-zh/rst_source/development/add_robot.rst +++ b/docs/source-zh/rst_source/development/add_robot.rst @@ -43,6 +43,7 @@ RPent 的整体进程划分、服务职责和通信方式见 :doc:`系统设计 .. code-block:: python # robots/myenv/__init__.py + from rpent.dashboard.events import DashboardEventSink from rpent.envs.env_spec import EnvSpec, RunConfig from rpent.envs.prompt_bundle import PromptBundle from robots.myenv.prompt_bundle import system_prompt, user_prompt @@ -56,9 +57,13 @@ RPent 的整体进程划分、服务职责和通信方式见 :doc:`系统设计 init_runtime=_init_runtime, ) - def get_toolkit(*, primitives_kwargs, video_path=None): + def get_toolkit(*, primitives_kwargs, dashboard_events: DashboardEventSink, video_path=None): from robots.myenv.toolkit import MyEnvToolkit - return MyEnvToolkit(primitives_kwargs=primitives_kwargs, video_path=video_path) + return MyEnvToolkit( + primitives_kwargs=primitives_kwargs, + dashboard_events=dashboard_events, + video_path=video_path, + ) def _add_cli_args(parser, use_dashboard) -> None: """向共享 parser 注册环境参数。见第 4 节。""" @@ -68,7 +73,7 @@ RPent 的整体进程划分、服务职责和通信方式见 :doc:`系统设计 """校验最终的 args，返回 RunConfig。见第 4 节。""" ... - def _init_runtime(args, output_dir): + def _init_runtime(args, output_dir, dashboard_events: DashboardEventSink): """启动 env_server、vla_server 及所需的辅助服务，构造 primitives_kwargs。 返回 (daemons, primitives_kwargs)。见第 5 节。 @@ -253,13 +258,14 @@ primitive driver 的 ``__init__``。其中通常包含 **``_add_cli_args(parser, use_dashboard) -> None``。** 将环境参数注册到 main.py 已创建的共享 parser。``use_dashboard`` 决定原本必填的参数是否保持可选， -这些值随后由 Dashboard launcher 填入。main.py 会在 -``parser.parse_args()`` 之前调用该钩子，因此 argparse 的 usage 和错误信息也会 -包含环境参数。 +每个 Dashboard TaskRun 的 ``suite`` 与 ``task`` 会在 ``parse_config`` 调用前由 +``/rpent-task`` 命令提供。main.py 会在 ``parser.parse_args()`` 之前调用该钩子， +因此 argparse 的 usage 和错误信息也会包含环境参数。 -**``_parse_config(args) -> RunConfig``。** 在 ``parser.parse_args()`` 以及 -Dashboard launcher（如果启用）运行后调用。该钩子检查 Dashboard 模式下暂时设为 -可选的字段是否已经填入，并返回 :class:`~rpent.envs.RunConfig`： +**``_parse_config(args) -> RunConfig``。** 普通 CLI 模式下，该钩子在 +``parser.parse_args()`` 后调用；Dashboard 模式下，每个 TaskRun 会先把 +``/rpent-task`` 命令提供的 ``suite`` 与 ``task`` 写入任务参数，再调用该钩子。 +该钩子校验这些字段并返回 :class:`~rpent.envs.RunConfig`： - ``recipe_tag`` —— 单次运行的环境标签，用于 transcript 文件名和 recipe 路径 （LIBERO 使用 ``f"{suite.replace('libero_', '')}_t{task}_s{seed}"``）。 @@ -267,8 +273,6 @@ Dashboard launcher（如果启用）运行后调用。该钩子检查 Dashboard ``init_output_dir`` 创建目录并配置日志。 - ``prompt_vars`` —— 传给 ``PromptBundle.render`` 的字典，通常包含运行标识和 prompt 引用的其他变量。 -- ``dashboard_state`` —— ``args.dashboard`` 为真时是 - :class:`~rpent.dashboard.state.State`，否则为 ``None``。 - ``task_desc`` —— 环境特定的任务标识字典，会原样写入 transcript JSON 记录 （LIBERO 使用 ``{"suite": ..., "task": ..., "seed": ...}``）。 @@ -282,12 +286,11 @@ Dashboard launcher（如果启用）运行后调用。该钩子检查 Dashboard def _parse_config(args) -> RunConfig: if not args.suite: raise ValueError("--suite is required") - # ... 生成 recipe_tag、output_dir、prompt_vars、dashboard_state ... + # ... 生成 recipe_tag、output_dir 和 prompt_vars ... return RunConfig( recipe_tag=recipe_tag, output_dir=output_dir, prompt_vars=prompt_vars, - dashboard_state=dashboard_state, task_desc={"suite": args.suite, "task": args.task, "seed": args.seed}, ) diff --git a/docs/source-zh/rst_source/development/architecture.rst b/docs/source-zh/rst_source/development/architecture.rst index a026b460..70be30cd 100644 --- a/docs/source-zh/rst_source/development/architecture.rst +++ b/docs/source-zh/rst_source/development/architecture.rst @@ -89,28 +89,28 @@ Runner (``rpent/cli/main.py``) 2. 根据 ``args.env_name`` 调用 ``get_env_spec`` 加载环境定义，再通过 ``env_spec.add_cli_args(parser, use_dashboard=args.dashboard)`` 将该环境 的专用参数加入共享 parser。启用 Dashboard 时，原本必填的环境参数会暂时 - 设为可选，随后由配置页面填写。 + 设为可选，因为任务参数随后通过 Dashboard 命令提供。 3. 再调用 ``parser.parse_args()``，对完整参数集合执行 argparse 层的校验， 并生成最终的 ``args``；参数错误仍使用 argparse 的标准提示格式。 4. 如果启用了 ``--dashboard``，启动配置页面，以当前参数作为默认值，并将 用户提交的配置写回 ``args``。 5. 调用 ``env_spec.parse_config(args)`` 校验运行配置，并生成 :class:`~rpent.envs.RunConfig`，其中包含 ``recipe_tag``、``output_dir``、 - ``prompt_vars``、``dashboard_state`` 和 ``task_desc``。启用 Dashboard - 时，此处还会确认配置页面已经补齐所需的环境参数。 + ``prompt_vars`` 和 ``task_desc``。 6. 调用 ``init_output_dir`` 创建本次运行的输出目录，并配置 ``run.log``。 7. 根据 ``--planner`` 调用 ``rpent.planner.base.build_planner`` 构造 **planner**，并使用环境提供的 prompt bundle 生成 system prompt 和 user prompt。 -8. 调用 ``env_spec.init_runtime(args, output_dir)``。环境实现会启动 - ``env_server`` 和 ``vla_server``；如果指定了 ``--env-endpoint`` 或 - ``--vla-endpoint``，则连接已有服务。该方法返回 +8. 调用 ``env_spec.init_runtime(args, output_dir, dashboard_events)``。环境实现会 + 启动 ``env_server``、``vla_server`` 和 ``sam3_server``；如果指定了对应 endpoint， + 则连接已有服务。该方法返回 ``(daemons, primitives_kwargs)``。 -9. 将 ``primitives_kwargs`` 传给环境的 ``get_toolkit`` 工厂，构造 - **toolkit**。 -10. 执行工具调用循环；启用 Dashboard 时，同时将运行事件发送到监控页面。 - 循环结束后保存 ``/transcript_*.json``，并在清理 toolkit - 时完成回合录像等收尾工作。 +9. 将 ``primitives_kwargs`` 和 ``dashboard_events`` 事件接收器传给环境的 + ``get_toolkit`` 工厂，构造 **toolkit**。一次性运行链路使用不执行任何 + 操作的事件接收器。 +10. 执行工具调用循环。循环结束后保存 + ``/transcript_*.json``，并在清理 toolkit 时完成回合录像等 + 收尾工作。 ``main.py`` 只负责连接上述步骤。环境相关实现集中在 ``robots//``， planner 后端集中在 ``rpent/planner/``， @@ -128,7 +128,7 @@ planner 后端集中在 ``rpent/planner/``， # robots/myenv/__init__.py def get_env_spec() -> EnvSpec: ... # 环境标识、提示词模板与 Runner 钩子 def get_toolkit( - *, primitives_kwargs, video_path=None, dashboard=None + *, primitives_kwargs, dashboard_events, video_path=None ): ... ``EnvSpec`` 汇集了环境的标识、prompt 模板与三个 Runner 钩子 @@ -155,19 +155,22 @@ Dashboard（可选） ----------------- ``rpent/dashboard/`` 由 FastAPI 应用和静态前端组成。启用 ``--dashboard`` 后， -``rpent/cli/main.py`` 会根据 ``--dashboard-host`` 和 ``--dashboard-port`` -启动 Dashboard；默认绑定 ``127.0.0.1``，并由操作系统分配可用端口。运行开始前， -用户可以先在配置页面确认或修改参数。 +``rpent/cli/main.py`` 会将控制权交给 ``rpent/cli/dashboard.py``，由后者根据 +``--dashboard-host`` 和 ``--dashboard-port`` 启动 Dashboard，并在启动共享服务前 +确认配置。VLA 和 SAM3 会在 Dashboard 运行期间复用；通过 ``/rpent-task`` 提交的 +任务使用独立的运行环境，并按顺序执行。 -运行期间，Dashboard 页面提供： +TaskRun 运行期间，Dashboard 页面提供： - planner 输出以及工具调用事件； -- 实时相机画面和 Pi0.5 视图； +- 实时固定相机和腕部相机画面； - 动作时间线和单步动作片段； - 运行结束后的完整回合录像（如果已生成）。 +页面可以提交普通 planner 消息、新任务命令和中断请求，但不会直接发出机器人 +动作。planner、toolkit 和环境运行时通过 ``dashboard_events`` 事件接收器发布 +展示更新。 服务端通过 SSE 推送运行状态摘要，前端再按需读取详细事件、时间线和图像。 -Dashboard 使用 planner 与 toolkit 产生的状态进行展示，不直接发出机器人动作。 下一步 ------ diff --git a/docs/source-zh/rst_source/development/interfaces.rst b/docs/source-zh/rst_source/development/interfaces.rst index e46c980a..40de199a 100644 --- a/docs/source-zh/rst_source/development/interfaces.rst +++ b/docs/source-zh/rst_source/development/interfaces.rst @@ -12,7 +12,7 @@ .. code-block:: python def get_env_spec() -> EnvSpec: ... - def get_toolkit(*, primitives_kwargs, video_path=None, dashboard=None): ... + def get_toolkit(*, primitives_kwargs, dashboard_events: DashboardEventSink, video_path=None): ... ``get_env_spec`` 返回 ``EnvSpec``，其中你需要提供： @@ -34,10 +34,11 @@ 三项需由你正确填写（供 prompt 模板插值）。 * - ``init_runtime`` - 启动或连接 env 与 VLA 等子进程，构造 ``primitives_kwargs`` 字典 - （env 客户端、模型客户端等），供 toolkit 组装 primitive driver。 + （env 客户端、模型客户端等），供 toolkit 组装 primitive driver； + ``DashboardEventSink`` 用于上报运行时状态。 -``get_toolkit`` 一般只需把 ``primitives_kwargs`` 传给环境子类；``video_path``、 - ``dashboard`` 由 ``main.py`` 传入，通常不用改。 +``get_toolkit`` 一般只需把 ``primitives_kwargs`` 传给环境子类； +``dashboard_events``、``video_path`` 由 ``main.py`` 传入，通常不用改。 参考实现：``robots/libero/__init__.py``。 @@ -58,6 +59,7 @@ Planner toolkit: Toolkit, max_turns: int, input_queue=None, + dashboard_interaction=None, ) -> PlannerResult: ... 约定：用 ``toolkit.get_tools_spec()`` 把工具交给模型；每次调用 ``toolkit.execute_tool(name, input_dict)``； diff --git a/docs/source-zh/rst_source/quickstart.rst b/docs/source-zh/rst_source/quickstart.rst index b9cf71c7..faf6ee0f 100644 --- a/docs/source-zh/rst_source/quickstart.rst +++ b/docs/source-zh/rst_source/quickstart.rst @@ -42,14 +42,18 @@ LIBERO-PRO 仿真资源。下面以 LIBERO-PRO 和 ``claude_code`` planner 3. 通过 Dashboard 查看运行过程 ------------------------------ -添加 ``--dashboard`` 后，RPent 会启动本地 Dashboard 服务，并在终端输出访问地址。打开该地址后，可以先在启动页面确认配置。运行开始后，Dashboard 会实时显示智能体的推理过程、相机与 Pi0 视图、动作时间线和片段回放。使用 ``--dashboard-language zh-cn`` 可切换到中文界面。 +添加 ``--dashboard`` 后，RPent 会启动本地 Dashboard，并在终端输出访问地址： .. code-block:: bash rpent --env libero --dashboard --dashboard-language zh-cn \ - --suite libero_object_swap --task 2 --seed 0 \ --planner claude_code --model claude-opus-4-8 +打开该地址并确认配置；服务就绪后，在页面输入 +``/rpent-task libero_object_swap 2 0`` 启动任务。Dashboard 会实时显示智能体的 +推理过程、相机画面和动作时间线；任务结束后可以继续提交下一任务。使用 +``--dashboard-language zh-cn`` 可切换到中文界面。 + 关键 CLI 选项 ------------- diff --git a/docs/source-zh/rst_source/usage/advanced_deployment.rst b/docs/source-zh/rst_source/usage/advanced_deployment.rst index 63e7fab4..8060fbd7 100644 --- a/docs/source-zh/rst_source/usage/advanced_deployment.rst +++ b/docs/source-zh/rst_source/usage/advanced_deployment.rst @@ -10,6 +10,9 @@ Pi0.5 VLA 用 ``--vla-endpoint``，SAM3 用 ``--sam3-endpoint``。每个都取 ``[protocol://]HOST:PORT``，省略 protocol 时默认 HTTP，也可用 ``socket://`` 改走 socket RPC。 +Dashboard Session 不支持 ``--env-endpoint``，因为每个 TaskRun 都需要使用新的 +环境服务；Dashboard 模式仍可使用 ``--vla-endpoint`` 和 ``--sam3-endpoint``。 + LIBERO 环境服务 --------------- diff --git a/docs/source-zh/rst_source/usage/libero.rst b/docs/source-zh/rst_source/usage/libero.rst index 2675458b..28cb9fa5 100644 --- a/docs/source-zh/rst_source/usage/libero.rst +++ b/docs/source-zh/rst_source/usage/libero.rst @@ -150,17 +150,30 @@ LIBERO 工具分为物理动作工具和只读工具。 Dashboard --------- -加上 ``--dashboard`` 可启动本地监控服务。系统会自动选择一个空闲端口， -并在终端输出访问 URL： +加上 ``--dashboard`` 可启动长生命周期的本地 Dashboard Session。系统会自动 +选择一个空闲端口，并在终端输出访问 URL： .. code-block:: bash rpent --env libero --dashboard \ - --suite libero_object_swap --task 2 --seed 0 \ --planner claude_code --model claude-opus-4-8 -Dashboard 会实时展示推理过程、agentview 视图、腕部相机视图、Pi0.5 -叠加信息和动作时间线。使用 ``--dashboard-language zh-cn`` 切换中文 UI。 +打开该地址，确认 Session 配置并点击 **Start Session**。共享服务就绪后，在页面 +输入以下命令启动 TaskRun： + +.. code-block:: text + + /rpent-task libero_object_swap 2 0 + +每个 TaskRun 都使用新的环境，VLA 和 SAM3 服务则由 Session 复用。任务完成后可以 +继续提交另一条 ``/rpent-task``；任务运行期间提交新命令则会切换任务。Dashboard +会实时展示智能体的推理和工具调用、固定相机与腕部相机视图、动作时间线，以及动作 +片段和完整 episode 回放。TaskRun 运行时，输入普通消息可以引导智能体，按 Esc +可以请求中断。在终端按 Ctrl+C 可结束整个 Session。 + +``--dashboard`` 不能与 ``--interactive`` 或 ``--env-endpoint`` 同时使用；外部 +``--vla-endpoint`` 和 ``--sam3-endpoint`` 服务仍然可用。使用 +``--dashboard-language zh-cn`` 可切换中文 UI。 接入自定义 VLA ----------------