不需额外训练,用注意力与动作引导提升视觉语言模型推理效率
ATA: Bridging Implicit Reasoning with Attention-Guided and Action-Guided Inference for Vision-Language Action Models
- 通过注意力与动作引导的隐式推理,无需标注即可优化视觉输入
- 在多个任务上提升成功率与鲁棒性,推理速度反而更快
- 可直接插入现有模型,适合追求高效部署的机器人应用
视觉-语言-动作(VLA)模型依赖当前观测(图像、语言指令、机器人状态)预测动作并完成任务。尽管精准视觉感知对动作预测至关重要,近期研究尝试在推理中引入显式推理以提升性能,但此类方法常依赖数据密集型标注(如思维链标注)和额外视觉定位标注(如边界框或掩码),且需耗时的数据构建、标注与重训练,导致推理序列变长、效率下降。为此,我们提出ATA——一种无需训练的新型框架,通过互补的注意力引导与动作引导策略,在推理中实现隐式推理。ATA不依赖额外标注,而是将注意力图与基于动作的感兴趣区域(RoI)融合,自适应地优化视觉输入。该方法为VLA模型提供即插即用的隐式推理能力,轻量高效。大量实验证明,ATA持续提升任务成功率与鲁棒性,同时保持甚至增强推理效率。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models rely on current observations, including images, language instructions, and robot states, to predict actions and complete tasks. While accurate visual perception is crucial for precise action prediction and execution, recent work has attempted to further improve performance by introducing explicit reasoning during inference. However, such approaches face significant limitations. They often depend on data-intensive resources such as Chain-of-Thought (CoT) style annotations to decompose tasks into step-by-step reasoning, and in many cases require additional visual grounding annotations (e.g., bounding boxes or masks) to highlight relevant image regions. Moreover, they involve time-consuming dataset construction, labeling, and retraining, which ultimately results in longer inference sequences and reduced efficiency. To address these challenges, we propose ATA, a novel training-free framework that introduces implicit reasoning into VLA inference through complementary attention-guided and action-guided strategies. Unlike CoT or explicit visual-grounding methods, ATA formulates reasoning implicitly by integrating attention maps with an action-based region of interest (RoI), thereby adaptively refining visual inputs without requiring extra training or annotations. ATA is a plug-and-play implicit reasoning approach for VLA models, lightweight yet effective. Extensive experiments show that it consistently improves task success and robustness while preserving, and even enhancing, inference efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。