让机器人像人一样思考动作:先预测下一步,再反推怎么做。
ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation

- 用统一生成框架交替处理视觉与语言推理,分步规划动作。
- 在长任务中成功率显著提升,仿真和真实场景均超越现有方法。
- 适合需要复杂推理的机器人操控,如多步骤组装任务。
大多数视觉-语言-动作(VLA)模型直接将观测映射为动作,缺乏显式推理能力,限制了其在长程任务中的表现。现有方法虽采用思维链(CoT)进行子目标分解和空间预判,但缺乏统一的跨模态推理架构,且未显式包含基于目标状态的逆向推理。我们认为,操作规划自然可分为前向预测(预测下一视觉状态)与逆向动力学(推断达成目标所需动作)。为此,我们提出 extbf{ThinkingVLA},一种在统一混合变压器架构中实现该分解的生成模型。其包含前向思维链以确定即时子目标并引导视觉预测;预测图像作为目标状态,触发逆向思维链,基于该图像推理空间关系与动作意图;最终动作则基于完整推理上下文生成。在仿真与真实世界基准上的大量实验表明,ThinkingVLA持续优于当前最优基线,尤其在长程操作任务中表现突出。
原文摘要 · Abstract (English)
Most Vision-Language-Action (VLA) models map observations directly to actions without explicit reasoning, limiting their capacity for reasoning-intensive long-horizon tasks. To address this, existing approaches adopt Chain-of-Thought (CoT) reasoning to enable subgoal decomposition and spatial anticipation. However, those methods lack a unified architecture for effective cross-modal reasoning and fail to explicitly include inverse reasoning ability based on the target state. We argue that manipulation planning naturally decomposes into prediction, anticipating the next visual state, and inverse dynamics, inferring the actions to reach it. Bridging both requires a unified autoregressive architecture that interleaves textual and visual reasoning in a single generation process. We propose \textbf{ThinkingVLA}, a generative model that realizes this decomposition within a unified Mixture-of-Transformers architecture. ThinkingVLA consists of a forward CoT that identifies the immediate subgoal and guides the visual forecasting; the predicted image then serves as the target state, grounding an inverse CoT that reasons about spatial relationships and action intent based on the predicted image; and the final action is generated conditioned on this full reasoning context. Extensive experiments on simulation and real-world benchmarks demonstrate that ThinkingVLA consistently outperforms state-of-the-art baselines, with particularly large gains on long-horizon manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。