提出并行双链思维框架,让机器人更聪明地规划复杂动作。
DualCoT-VLA: Visual-Linguistic Chain of Thought via Parallel Reasoning for Vision-Language-Action Models
- 用视觉与语言双链并行推理,兼顾细节感知与逻辑规划。
- 在LIBERO和RoboCasa上超越现有方法,真实场景表现优异。
- 单步推理降低延迟,适合对实时性要求高的机器人应用。
视觉-语言-动作(VLA)模型将视觉输入和语言指令直接映射为机器人动作。尽管在简单任务中有效,标准VLA模型在需要逻辑规划的多步骤复杂任务以及需精细空间感知的精确操作上表现不佳。近期工作引入链式思维(CoT)以赋予VLA模型“先思考后行动”能力,但现有方法存在两大局限:1)依赖单一模态的孤立链式思维,难以同时捕捉低层视觉细节与高层逻辑规划;2)因逐步自回归解码导致推理延迟高且错误累积严重。为此,我们提出DualCoT-VLA,一种支持并行推理的视觉-语言链式思维方法。通过整合视觉链式思维(用于低层空间理解)与语言链式思维(用于高层任务规划),实现全面多模态推理。此外,引入两组可学习查询标记,将自回归推理转为单步前向推理,突破延迟瓶颈。大量实验表明,DualCoT-VLA在LIBERO和RoboCasa GR1基准上达到当前最优性能,并在真实机器人平台表现卓越。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models map visual observations and language instructions directly to robotic actions. While effective for simple tasks, standard VLA models often struggle with complex, multi-step tasks requiring logical planning, as well as precise manipulations demanding fine-grained spatial perception. Recent efforts have incorporated Chain-of-Thought (CoT) reasoning to endow VLA models with a ``thinking before acting'' capability. However, current CoT-based VLA models face two critical limitations: 1) an inability to simultaneously capture low-level visual details and high-level logical planning due to their reliance on isolated, single-modal CoT; 2) high inference latency with compounding errors caused by step-by-step autoregressive decoding. To address these limitations, we propose DualCoT-VLA, a visual-linguistic CoT method for VLA models with a parallel reasoning mechanism. To achieve comprehensive multi-modal reasoning, our method integrates a visual CoT for low-level spatial understanding and a linguistic CoT for high-level task planning. Furthermore, to overcome the latency bottleneck, we introduce a parallel CoT mechanism that incorporates two sets of learnable query tokens, shifting autoregressive reasoning to single-step forward reasoning. Extensive experiments demonstrate that our DualCoT-VLA achieves state-of-the-art performance on the LIBERO and RoboCasa GR1 benchmarks, as well as in real-world platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。