让视觉语言动作模型学会关注过去，提升机器人任务执行能力。
AVA-VLA: Improving Vision-Language-Action models with Active Visual Attention
- 用循环状态记录任务历史，让模型具备记忆能力。
- 动态重加权视觉特征，聚焦与指令和历史最相关的区域。
- 在多个机器人基准上表现领先，适合真实双臂操作场景。
视觉-语言-动作（VLA）模型在具身任务中取得了显著进展，但多数方法在每个时间步独立处理视觉观测，忽略了历史信息。这种无历史依赖的设计将机器人操作视为马尔可夫决策过程，而现实中机器人控制本质上是部分可观测的，需基于过往交互进行推理。为此，本文从部分可观测马尔可夫决策过程视角重新设计VLA策略学习，提出AVA-VLA框架：通过循环状态作为任务历史的神经近似信念，指导动作生成。在此基础上，引入主动视觉注意力（AVA），根据指令与执行历史动态调整当前观测中视觉标记的权重，聚焦最相关区域。大量实验表明，AVA-VLA在标准机器人基准（如LIBERO和CALVIN）上达到当前最优性能，并有效迁移至真实世界双臂操作任务。结果验证了时序锚定的主动视觉处理对提升VLA在机器人序列决策中的有效性。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have shown remarkable progress in embodied tasks recently, but most methods process visual observations independently at each timestep. This history-agnostic design treats robot manipulation as a Markov Decision Process, even though real-world robotic control is inherently partially observable and requires reasoning over past interactions. To address this mismatch, we reformulate VLA policy learning from a Partially Observable Markov Decision Process perspective and propose AVA-VLA, a framework that conditions action generation on a recurrent state that serves as a neural approximation to the agent's belief over task history. Built on this recurrent state, we introduce Active Visual Attention (AVA), which dynamically reweights visual tokens in the current observation to focus on regions most relevant given both the instruction and execution history. Extensive experiments show that AVA-VLA achieves state-of-the-art performance on standard robotic benchmarks, including LIBERO and CALVIN, and transfers effectively to real-world dual-arm manipulation tasks. These results demonstrate the effectiveness of temporally grounded active visual processing for improving VLA performance in robotic sequential decision-making. The project page is available at https://liauto-dsr.github.io/AVA-VLA-Page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。