MolmoAct2: Action Reasoning Models for Real-world Deployment

MolmoAct2 is Ai2's open family of action reasoning models for robot control and real-world deployment. It builds on the Molmo2-ER embodied-reasoning vision-language backbone, adds robot state and action modeling, and connects the VLM to a flow-matching continuous action expert for closed-loop manipulation. The release includes base checkpoints for continued training, fine-tuned robot policies for evaluation and deployment, and the datasets used to build MolmoAct2 and Molmo2-ER.

---

Updates

• [2026/05/28] 🔥 MolmoAct2 has been fully integrated into Huggingface, LeRobot official repo at MolmoAct2.
• [2026/05/19] 🔥 We've also released MolmoAct2-Cortex evaluation rollouts on YAM bimanual setups (useful for failure annotation and reward model training) at Policy Rollouts.
• [2026/05/17] 🔥 We have released FastAPI inference servers for MolmoAct2 using DROID and YAM setups at Inference Servers (implemented by Jie Wang).
• [2026/05/14] 🔥 We have released MolmoAct2 lerobot workflow for fine-tuning and inference. Check it out.
• [2025/05/06] 🔥 Detail implementation and setup for Franka, SO-100/101, and bimanual YAM have been released at Real-world Deployment.
• [2026/05/05] 🔥 MolmoAct2 has been released!

1. Models

Base Models

We provide base checkpoints at every training stage for continued MolmoAct2 training and robot fine-tuning. These are foundation checkpoints rather than one-size-fits-all deployment policies.

Model	Use Case	Description	Checkpoint Path
MolmoAct2	Fine-tuning	Post-trained MolmoAct2 model with a continuous flow-matching action expert. Use as the default foundation checkpoint for adapting to a target robot embodiment or benchmark.	https://huggingface.co/allenai/MolmoAct2
MolmoAct2-Think	Fine-tuning	MolmoAct2 foundation checkpoint with depth-token reasoning. Use when downstream policies should reason over compact depth predictions before acting.	https://huggingface.co/allenai/MolmoAct2-Think
MolmoAct2-Pretrain	Post-training	Pre-trained discrete autoregressive VLA backbone before the continuous action expert is attached. Intended for continuing MolmoAct2 training stages, not direct continuous-control inference.	https://huggingface.co/allenai/MolmoAct2-Pretrain
Molmo2-ER	Pre-training	Embodied-reasoning VLM backbone used as the starting point for MolmoAct2 action models.	https://huggingface.co/allenai/Molmo2-ER

Finetuned Models

We also provide fine-tuned checkpoints for common robot platforms and benchmarks. These models are intended to run directly in their target setting, or to serve as a stronger starting point for closely related robots. As with any robot policy, performance depends on hardware, cameras, calibration, action conventions, and language/task distribution.

Model	Use Case	Description	Checkpoint Path
MolmoAct2-DROID	Inference / Fine-tuning	MolmoAct2 fine-tuned on the filtered DROID Franka mixture with absolute joint-pose control. Intended for DROID-style policy inference or further fine-tuning.	https://huggingface.co/allenai/MolmoAct2-DROID
MolmoAct2-BimanualYAM	Inference / Fine-tuning	MolmoAct2 fine-tuned on the bimanual YAM mixture with absolute joint-pose control and annotated language instructions.	https://huggingface.co/allenai/MolmoAct2-BimanualYAM
MolmoAct2-SO100_101	Inference / Fine-tuning	MolmoAct2 fine-tuned on SO-100/SO-101 datasets with absolute joint-pose control and annotated language instructions.	https://huggingface.co/allenai/MolmoAct2-SO100_101
MolmoAct2-LIBERO	Inference / Fine-tuning	MolmoAct2 fine-tuned on the full LIBERO training mixture, combining Spatial, Object, Goal, and Long suites.	https://huggingface.co/allenai/MolmoAct2-LIBERO
MolmoAct2-Think-LIBERO	Inference / Fine-tuning	MolmoAct2-Think fine-tuned on LIBERO with depth-and-action examples and adaptive depth reasoning.	https://huggingface.co/allenai/MolmoAct2-Think-LIBERO

2. Datasets

Data	Description	Dataset Path
MolmoAct2-BimanualYAM Dataset	Collection of bimanual YAM datasets and related resources used for MolmoAct2 bimanual training and evaluation.	https://huggingface.co/collections/allenai/molmoact2-bimanualyam-dataset-69f81e17b140ec34f430a35e
MolmoAct2 Robotics Datasets	Robotics datasets for MolmoAct2 training and fine-tuning, including SO-100/SO-101, DROID, MolmoAct Dataset, BC-Z, Bridge, and RT-1.	https://huggingface.co/collections/allenai/molmoact2-datasets-69f81e316ec3daafe3f9555c
Molmo2-ER Datasets	Embodied reasoning datasets used for Molmo2-ER and MolmoAct2 backbone training, including spatial, 3D, robotics, and visual reasoning data.	https://huggingface.co/collections/allenai/molmo2-er-datasets-69f8d605d92d46a5fc24ced2

Note that all of the robotics datasets for pre-training and post-training are in LeRobot v3.0 format, paired with extra language annotations.

3. LeRobot Integration

MolmoAct2 is integrated into LeRobot as a policy implementation, so users can train, evaluate, and deploy MolmoAct2 with standard LeRobot datasets and workflows. This repository includes the LeRobot integration as a Git submodule at lerobot/, pinned to the branch allenai/lerobot:molmoact2-policy.

For training, although all of our experiments start from the base checkpoint allenai/MolmoAct2, we recommend starting from the fine-tuned checkpoints listed in the Finetuned Models section above if your embodiment is similar to Bimanual YAM, DROID Franka, or SO-100/SO-101, as they can provide better initialization and downstream performance. For generic use, use the base checkpoint.

After cloning this repository, initialize the submodule from the repo root:

git submodule update --init --recursive
cd lerobot

For training, evaluation, and deployment instructions, see the MolmoAct2 LeRobot documentation at docs/source/molmoact2.mdx. To reproduce the original LIBERO benchmark results exactly with the v0.5.1 evaluation stack, use the pinned inference branch allenai/lerobot:molmoact2-hf-inference with instructions in MolmoAct2 README.

4. Real-world Deployment

MolmoAct2 supports out-of-the-box deployment on three robot embodiments:

• SO-100
• Bimanual YAMs
• Franka DROID setup

SO-100/101 Setup

For the best performance, we recommend using an SO-100 with the standard wrist configuration and a third-person camera. Here is an open implementation by Irene Grace. Code

Bimanual YAM Setup

For the best performance, please build your Bimanual YAM setup following the reference design below:

All required components can be purchased using this Bimanual YAM parts list.

Implementation code for setting up, data collection, and inference for Bimanual YAM is here

Franka Setup

For the Franka setup, we recommend following the official DROID implementation for best results.

5. Inference Servers

This repository ships two FastAPI inference servers under examples/, one per fine-tuned checkpoint. Each server exposes the same /act wire protocol — json_numpy-encoded request/response — but with an embodiment-specific schema (camera count, state dimension, normalisation tag).

Server	Checkpoint	Default port	State dim	Cameras
examples/droid/host_server_droid.py	allenai/MolmoAct2-DROID	8000	(8,) = [q1..q7, gripper]	external, wrist
examples/yam/host_server_yam.py	allenai/MolmoAct2-BimanualYAM	8202	(14,) (per-arm 7-D × 2 arms)	top, left, right (order matters)

1. Install uv

curl -LsSf https://astral.sh/uv/install.sh | sh
exec $SHELL          # reload PATH so the `uv` binary is picked up
uv --version

2. Create the project environment

The pinned dependencies (CUDA-12.1 PyTorch wheels, transformers, fastapi, json-numpy, …) live in pyproject.toml. From the repo root:

uv sync                  # creates .venv/ and installs all deps
uv run python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
# expected: True NVIDIA RTX A6000

uv reads .python-version (3.11) and downloads a matching interpreter if needed. Re-run uv sync after pulling new commits.

3. Download the checkpoint (~22 GB each)

export HF_HUB_ENABLE_HF_TRANSFER=1                       # fast parallel download
uv run hf download allenai/MolmoAct2-DROID               # for the DROID server
uv run hf download allenai/MolmoAct2-BimanualYAM         # for the YAM server

To put the cache on a different disk, set HF_HOME=/path/to/cache before the download (and when starting the server).

4. Start a server

# DROID (Franka)
uv run python examples/droid/host_server_droid.py --host 0.0.0.0 --port 8000 --dtype bfloat16

# Bimanual YAM
uv run python examples/yam/host_server_yam.py --host 0.0.0.0 --port 8202 --dtype bfloat16

Useful flags (both servers):

• --dtype bfloat16|float16|float32 — default bfloat16. The DROID model card uses float32 (~88 GB), which only fits on ~96 GB of free VRAM. The YAM model card reports float32 at ~26 GB (fits on a single A6000), bfloat16 under 16 GB. bfloat16 is the safe default for both.
• --device cuda:0
• --cuda-graph — enables CUDA-graph capture for the action expert (~2× faster per call, ~2 GB extra VRAM). Disabled by default so the server coexists with other GPU workloads.
• --no-warmup — skip the dummy forward pass at startup.

bf16 patches

Loading in bfloat16 is not officially supported by the upstream MolmoAct2 code; each server applies two idempotent patches to the cached modeling_molmoact2.py at startup:

1. flow-matching trajectory uses the model dtype instead of hardcoded float32 (otherwise the action expert errors with mat1 and mat2 must have the same dtype),
2. _to_array casts to float32 before .numpy() (numpy has no bf16 dtype).

Both are marked with # patched_bf16_* comments and re-applied on every server start, so re-downloading the checkpoint won't break things. Newer snapshot revisions (e.g. YAM) have already fixed both upstream; the server will log "needle not found" warnings, which are harmless.

5. Reach it from the LAN

Bound to 0.0.0.0, the server is reachable on every interface of this host. Health check:

curl http://<lan-ip>:8000/act
# DROID: {"status":"ok","repo_id":"allenai/MolmoAct2-DROID","norm_tag":"franka_droid",...}

curl http://<lan-ip>:8202/act
# YAM:   {"status":"ok","repo_id":"allenai/MolmoAct2-BimanualYAM","norm_tag":"yam_dual_molmoact2","num_cameras":3,"state_dim":14,...}

The wire format (json_numpy-encoded request) is documented in the docstring at the top of each server file. The DROID server expects external_cam, wrist_cam, instruction, state; the YAM server expects top_cam, left_cam, right_cam, instruction, state. Both return actions ((N, D) float32) and dt_ms.

Firewall / port

If clients on the LAN can't connect, open the port locally:

sudo ufw allow from <subnet> to any port 8000 proto tcp   # DROID
sudo ufw allow from <subnet> to any port 8202 proto tcp   # YAM

6. Coming Soon

Full code for training, fine-tuning, deployment, evaluation, and more details are coming soon.

7. License

This model is licensed under Apache 2.0. It is intended for research and educational use in accordance with Ai2's Responsible Use Guidelines.

8. Model and Hardware Safety

MolmoAct2 generate robot actions from visual observations and language instructions, but their behavior may vary across embodiments, environments, and hardware configurations. Users should carefully validate model outputs before deployment, especially when operating physical robots or other actuated systems. Where possible, actions should be monitored through interpretable intermediate outputs (adaptive depth map), simulation rollouts, action limits, or other safety checks before execution on hardware. The model’s action space should be bounded by the training data, robot controller limits, and task-specific safety constraints, including limits on speed, workspace, torque, and contact force. Users should follow the hardware manufacturer’s safety guidelines, use appropriate emergency-stop mechanisms, and operate the system only in a safely configured environment with human supervision.

9. Contacts

For questions, collaborations, or support, please contact with:

{hqfang,duanj1}@cs.washington.edu 

Found a bug or have a feature request? Please open a GitHub issue.

10. Citation

@misc{fang2026molmoact2actionreasoningmodels,
      title={MolmoAct2: Action Reasoning Models for Real-world Deployment}, 
      author={Haoquan Fang and Jiafei Duan and Donovan Clay and Sam Wang and Shuo Liu and Weikai Huang and Xiang Fan and Wei-Chuan Tsai and Shirui Chen and Yi Ru Wang and Shanli Xing and Jaemin Cho and Jae Sung Park and Ainaz Eftekhar and Peter Sushko and Karen Farley and Angad Wadhwa and Cole Harrison and Winson Han and Ying-Chun Lee and Eli VanderBilt and Rose Hendrix and Suveen Ellawela and Lucas Ngoo and Joyce Chai and Zhongzheng Ren and Ali Farhadi and Dieter Fox and Ranjay Krishna},
      year={2026},
      eprint={2605.02881},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2605.02881}, 
}