arXiv:2602.12684cs.ROcs.LG2026-02被引 33

小米自研视觉语言动作模型,实现实时流畅机器人控制。

Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution

  • 通过跨机器人轨迹与多模态数据预训练,提升动作生成泛化能力。
  • 在仿真和真实双臂操作任务中均达顶尖性能,消费级显卡即可实时运行。
  • 采用异步推理优化与动作块对齐策略,保障真实机器人连续稳定执行。

本文介绍 Xiaomi-Robotics-0,一个面向高性能与快速平滑实时执行的视觉语言动作(VLA)模型。其核心在于精心设计的训练配方与部署策略。该模型首先在大规模跨体感机器人轨迹及视觉语言数据上进行预训练,赋予其广泛且可迁移的动作生成能力,同时避免底层预训练视觉语言模型(VLM)的语义知识灾难性遗忘。后训练阶段,提出多项技术以实现异步执行训练,解决真实机器人推理延迟问题。部署时,通过精确对齐连续动作块的时间步,确保实时推演的连续性与无缝性。在多个仿真基准和两项需要精细灵巧双臂操作的真实机器人任务中进行了全面评估。结果表明,该方法在所有仿真基准中均达到最先进水平。此外,Xiaomi-Robotics-0 可在消费级 GPU 上实现快速平稳的真实机器人推演,在两项真实任务中均取得高成功率与高吞吐量。为促进后续研究,代码与模型检查点已开源:https://xiaomi-robotics-0.github.io

原文摘要 · Abstract (English)

In this report, we introduce Xiaomi-Robotics-0, an advanced vision-language-action (VLA) model optimized for high performance and fast and smooth real-time execution. The key to our method lies in a carefully designed training recipe and deployment strategy. Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities while avoiding catastrophic forgetting of the visual-semantic knowledge of the underlying pre-trained VLM. During post-training, we propose several techniques for training the VLA model for asynchronous execution to address the inference latency during real-robot rollouts. During deployment, we carefully align the timesteps of consecutive predicted action chunks to ensure continuous and seamless real-time rollouts. We evaluate Xiaomi-Robotics-0 extensively in simulation benchmarks and on two challenging real-robot tasks that require precise and dexterous bimanual manipulation. Results show that our method achieves state-of-the-art performance across all simulation benchmarks. Moreover, Xiaomi-Robotics-0 can roll out fast and smoothly on real robots using a consumer-grade GPU, achieving high success rates and throughput on both real-robot tasks. To facilitate future research, code and model checkpoints are open-sourced at https://xiaomi-robotics-0.github.io

机器人视觉语言实时控制VLA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。