开源可部署的机器人动作推理模型，兼顾性能与效率。
MolmoAct2: Action Reasoning Models for Real-world Deployment
- 采用空间具身推理专用视觉语言模型，基于330万样本训练。
- 在7个仿真与真实场景中超越基线，成功率显著提升。
- 适合需要低延迟、低成本部署的机器人研究与应用。
视觉-语言-动作（VLA）模型旨在为机器人提供统一的通用控制器，但现有系统在真实部署中仍存短板：前沿模型闭源，开源替代品依赖昂贵硬件，增强推理策略带来高延迟，微调成功率未达可靠使用阈值。本文提出MolmoAct2，一个面向实际部署的全开源动作推理模型，在前代基础上沿五个方向推进：引入MolmoER，一种专用于空间与具身推理的视觉语言模型，基于330万样本数据集，采用先专精后重演的训练策略；发布三个新数据集，涵盖低至中等成本平台，包括目前最大的开源双臂数据集MolmoAct2-BimanualYAM（720小时遥操作双臂轨迹），以及经质量过滤的Franka（DROID）和SO100/101子集；推出OpenFAST，一个基于五种具身形态上百万轨迹训练的开源权重、开数据动作分词器；重新设计架构，通过逐层键值缓存条件化，将连续动作专家融合至离散令牌视觉语言模型；提出MolmoThink，一种自适应深度推理变体，仅对时间步间变化区域重预测深度令牌，以极低延迟保持几何定位。在迄今为止最全面的开源VLA实证研究中，覆盖7个仿真与真实世界基准，MolmoAct2优于强基线如Pi-05，而MolmoER在13个具身推理基准上超越GPT-5与Gemini Robotics ER-1.5。模型权重、训练代码与完整训练数据均已公开。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models aim to provide a single generalist controller for robots, but today's systems fall short on the criteria that matter for real-world deployment. Frontier models are closed, open-weight alternatives are tied to expensive hardware, reasoning-augmented policies pay prohibitive latency for their grounding, and fine-tuned success rates remain below the threshold for dependable use. We present MolmoAct2, a fully open action reasoning model built for practical deployment, advancing its predecessor along five axes. We introduce MolmoER, a VLM backbone specialized for spatial and embodied reasoning, trained on a 3.3M-sample corpus with a specialize-then-rehearse recipe. We release three new datasets spanning low-to-medium cost platforms, including MolmoAct2-BimanualYAM, 720 hours of teleoperated bimanual trajectories that constitute the largest open bimanual dataset to date, together with quality-filtered Franka (DROID) and SO100/101 subsets. We provide OpenFAST, an open-weight, open-data action tokenizer trained on millions of trajectories across five embodiments. We redesign the architecture to graft a flow-matching continuous-action expert onto a discrete-token VLM via per-layer KV-cache conditioning. Finally, we propose MolmoThink, an adaptive-depth reasoning variant that re-predicts depth tokens only for scene regions that change between timesteps, retaining geometric grounding at a fraction of prior latency. In the most extensive empirical study of any open VLA to date, spanning 7 simulation and real-world benchmarks, MolmoAct2 outperforms strong baselines including Pi-05, while MolmoER surpasses GPT-5 and Gemini Robotics ER-1.5 across 13 embodied-reasoning benchmarks. We release model weights, training code, and complete training data. Project page: https://allenai.org/blog/molmoact2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。