arXiv:2503.10631cs.CVcs.RO2025-03被引 197

融合扩散与自回归,让机器人更精准地理解指令并执行动作。

HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model

论文配图:HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
图 1 · 摘自论文原文
  • 用统一框架结合扩散模型的连续动作预测和自回归的语言推理能力。
  • 在仿真和真实任务中分别提升14%和19%的成功率,对未知配置也稳定有效。
  • 适合需要高精度动作控制的机器人系统研发人员参考。

操纵策略设计的核心目标是使机器人能够理解人类指令、推理场景线索,并在动态环境中执行泛化动作。现有自回归视觉-语言-动作(VLA)方法继承视觉-语言模型(VLM)的常识推理能力,用于下一步动作标记预测,但将动作离散化为离散区间,破坏了精确控制所需的连续性。而现有的基于扩散的VLA方法虽能预测连续动作,仅依赖VLM提取的特征表示,未充分利用其通过标记级生成实现的预训练推理能力。为此,我们提出HybridVLA,一个统一框架,将扩散模型的连续动作特性与自回归的上下文推理能力整合进单一大型语言模型中。为缓解两种生成范式间的干扰,我们设计了一种协作训练方案,将扩散去噪过程无缝融入下一步标记预测中。实验发现,两种方法不仅相互增强,且在不同任务中表现强度各异。因此,我们进一步设计了协作动作集成机制,自适应融合两者预测结果,实现更鲁棒的控制。HybridVLA在仿真和真实世界任务中的平均成功率分别较之前最优方法提升14%和19%,并在未见过的配置下展现出稳定的操作性能。

原文摘要 · Abstract (English)

A fundamental objective of manipulation policy design is to endow robots to comprehend human instructions, reason about scene cues, and execute generalized actions in dynamic environments. Recent autoregressive vision-language-action (VLA) methods inherit common-sense reasoning capabilities from vision-language models (VLMs) for next action-token prediction. However, these methods quantize actions into discrete bins, which disrupts the continuity required for precise control. In contrast, existing diffusion-based VLA methods incorporate an additional diffusion head to predict continuous actions solely conditioned on feature representations extracted by the VLM, without fully leveraging the VLM's pretrained reasoning capabilities through token-level generation. To address these limitations, we introduce HybridVLA, a unified framework that absorbs the continuous nature of diffusion-based actions and the contextual reasoning of autoregression within a single large language model. To mitigate interference between the two generation paradigms, we propose a collaborative training recipe that seamlessly incorporates diffusion denoising into the next-token prediction process. With this recipe, we find these two action prediction methods not only reinforce each other but also exhibit varying strength across different tasks. Therefore, we design a collaborative action ensemble mechanism that adaptively fuses both predictions, leading to more robust control. HybridVLA outperforms previous state-of-the-art VLA methods by 14\% and 19\% in mean success rate on simulation and real-world tasks, respectively, while demonstrating stable manipulation in unseen configurations.

机器人视觉语言动作生成扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。