统一建模世界动态与动作,让机器人更聪明地执行复杂任务。
Motubrain: An Advanced World Action Model for Robot Control
- 用三流混合变换器联合建模视频与动作,支持多任务学习。
- 在真实环境中实现11赫兹推理速度,成功率达95.8%以上。
- 仅需50~100条新数据即可适配新机器人,适合实际部署。
视觉-语言-动作(VLA)模型虽语义泛化能力强,但对世界动态的细粒度建模不足。我们提出Motubrain,一种基于UniDiffuser框架的统一世界动作模型,采用三流混合变换器架构,联合建模视频与动作。单一模型支持策略学习、世界建模、视频生成、逆动力学及视频-动作联合预测,并可扩展至异构多模态数据,如仅视频、任务无关及跨机器人形态数据。在Motus基础上,Motubrain引入统一多视角建模、独立文本流强化语言-动作耦合、共享跨形态动作表示,以及高效后训练与部署方案,适用于长时序真实世界控制。推理栈融合步数缩减、编译、FP8量化、DiT缓存、V2A式仅动作推理与实时分块闭环执行,相比基线提速超50倍,最高达11赫兹。实验表明,Motubrain在RoboTwin 2.0上清洁与随机设置下平均成功率分别为95.8%和96.1%,在WorldArena中取得最强报告的EWMScore,且仅需50–100条轨迹即可适配新类人机器人形态。结果表明,统一世界动作模型可在泛化性、预测精度与实际部署能力上实现协同提升。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models generalize semantically well but often lack fine-grained modeling of world dynamics. We present Motubrain, a unified World Action Model that jointly models video and action under a UniDiffuser formulation with a three-stream Mixture-of-Transformers architecture. A single model supports policy learning, world modeling, video generation, inverse dynamics, and joint video-action prediction, while scaling to heterogeneous multimodal data such as video-only, task-agnostic, and cross-embodiment robot data. Building on Motus, Motubrain further introduces unified multiview modeling, an independent text stream for stronger language-action coupling, a shared cross-embodiment action representation, and an efficient post-training and deployment recipe for long-horizon real-world control. Our inference stack combines step reduction, compilation, FP8 quantization, DiT caching, V2A-style action-only inference, and real-time chunked closed-loop execution, achieving over 50x speedup over a naive baseline and up to 11 Hz inference. Experimentally, Motubrain achieves 95.8% and 96.1% average success on RoboTwin 2.0 under clean and randomized settings, respectively, attains the strongest reported EWMScore in our WorldArena comparison, and adapts to new humanoid embodiments with only 50--100 trajectories. These results show that unified world action models can scale in generality, predictive accuracy, and real-world deployability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。