让机器人通过逐步细化动作意图来思考,提升复杂任务执行能力。
ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models
- 用粗粒度动作序列作为思考过程,直接指导最终动作生成。
- 在真实与仿真环境中均显著优于现有方法,动作成功率更高。
- 适合需要精准动作规划的机器人操控研究者参考。
视觉-语言-动作模型已成为执行多样化操作任务的通用机器人策略,传统方法依赖视觉-语言模型嵌入直接将多模态输入转化为动作。近期进展引入了显式的中间推理机制,如子任务预测(语言)或目标图像生成(视觉),但这些中间步骤往往间接且难以传递精确动作执行所需的完整细节信息。本文提出,最有效的推理方式应直接在动作空间中进行。为此,我们提出动作链式思维(ACoT)范式,将推理过程建模为一系列结构化的粗粒度动作意图,以引导最终策略。本文进一步提出ACoT-VLA架构,包含两个互补组件:显式动作推理器(EAR)生成显式的参考轨迹作为动作级推理步骤;隐式动作推理器(IAR)从多模态输入的内部表征中提取潜在动作先验,共同构成一个条件化下游动作头的ACoT,实现具身化策略学习。大量真实世界与仿真环境实验表明该方法具有显著优势。代码已公开:https://github.com/AgibotTech/ACoT-VLA。
原文摘要 · Abstract (English)
Vision-Language-Action models have emerged as essential generalist robot policies for diverse manipulation tasks, conventionally relying on directly translating multimodal inputs into actions via Vision-Language Model embeddings. Recent advancements have introduced explicit intermediary reasoning-such as sub-task prediction (language) or goal image synthesis (vision)-to guide action generation. However, these intermediate reasoning are often indirect and inherently limited in their capacity to convey the full, granular information required for precise action execution. Instead, we posit that the most effective form of reasoning is one that deliberates directly in the action space. We introduce Action Chain-of-Thought (ACoT), a paradigm where the reasoning process itself is formulated as a structured sequence of coarse action intents that guide the final policy. In this paper, we propose ACoT-VLA, a novel architecture that materializes the ACoT paradigm. Specifically, we introduce two complementary components: an Explicit Action Reasoner (EAR) and Implicit Action Reasoner (IAR). The former proposes coarse reference trajectories as explicit action-level reasoning steps, while the latter extracts latent action priors from internal representations of multimodal input, co-forming an ACoT that conditions the downstream action head to enable grounded policy learning. Extensive experiments in real-world and simulation environments demonstrate the superiority of our proposed method. Code is available at: https://github.com/AgibotTech/ACoT-VLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。