提出新架构让视觉语言动作模型真正实现有效推理。
DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models
- 用混合注意力解码器分离语言与动作生成,避免干扰
- 通过两阶段训练使推理链与任务成功直接关联,提升性能21.7点
- 在真实机器人上验证可行,适合需要可靠决策的复杂任务
链式思维(CoT)推理是否真的提升视觉-语言-动作(VLA)模型能力,还是仅增加开销?现有系统效果有限且不一致,但缺乏对何时何故有效的系统分析。通过实验,我们发现两个必要条件:(1) 解码对齐:语言与动作需用适配模态的机制生成;若共用自回归解码器,性能下降4.2个百分点;(2) 因果对齐:推理必须通过结果反馈优化以建立因果关系;否则监督式推理在动态变化下表现与无推理几乎无异,性能下降32.0个百分点,接近无推理基线的31.6个百分点下降。基于此,我们构建DeepThinkVLA:采用混合注意力解码器实现第一条件,用两阶段监督微调+强化学习实现第二条件。该模型在LIBERO上达到97.0%成功率,LIBERO-Plus上鲁棒性达79.0%(优于π₀-FAST的61.6%),RoboTwin 2.0上成功率达59.3%,超越最强基线21.7个百分点。真实机器人实验初步验证了其物理适用性。代码已开源。
原文摘要 · Abstract (English)
Does Chain-of-Thought (CoT) reasoning genuinely improve Vision Language Action (VLA) models, or does it merely add overhead? Existing CoT-VLA systems report limited and inconsistent gains, yet no prior work has rigorously diagnosed when and why CoT helps robots act. Through systematic experiments, we identify two necessary conditions that must be jointly satisfied for CoT to be effective in VLA: (1) Decoding Alignment: CoT and actions must be generated with modality-appropriate mechanisms; forcing both through a single autoregressive decoder is not merely suboptimal but actively harmful, degrading performance by 4.2 percentage points; (2) Causal Alignment: CoT must be causally linked to task success via outcome-based optimization; without it, supervised CoT is indistinguishable from no reasoning at all under action-execution-sensitive dynamics shift, exhibiting a 32.0 pp performance drop nearly identical to the 31.6 pp drop of a reasoning-free baseline. Guided by these findings, we build DeepThinkVLA: a hybrid-attention decoder satisfies Condition 1 by pairing causal attention for language with bidirectional attention for parallel action decoding, while a two-stage SFT-then-RL pipeline satisfies Condition 2 by aligning the full reasoning: action chain with sparse task-success rewards. DeepThinkVLA achieves 97.0\% success on LIBERO, 79.0\% robustness on LIBERO-Plus (vs. 61.6\% for $π_0$-FAST), and 59.3\% success on RoboTwin 2.0, exceeding the strongest baseline by 21.7 points. Furthermore, real-robot experiments provide preliminary evidence for the physical applicability of our CoT data construction and hybrid architecture. Our codes are available at https://github.com/OpenBMB/DeepThinkVLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。