提出高效视觉问答框架,让智能体快速定位目标并准确回答问题。
FAST-EQA: Efficient Embodied Question Answering with Global and Local Region Relevancy
- 通过全局区域评分与局部目标搜索结合,引导智能体高效导航。
- 在HMEQA和EXPRESS-Bench上达到当前最优性能,推理速度显著更快。
- 适合需要快速响应的机器人视觉问答场景,如家庭服务或巡检。
具身问答(EQA)融合视觉理解、目标导向探索、空间时间推理,在部分可观测环境下具有挑战性。核心难题在于将物理搜索限制在与问题相关的子空间,同时保持紧凑可操作的观测记忆。为实现真实部署,探索过程中的快速推理至关重要。本文提出FAST-EQA,一种问题条件化框架:(i) 识别可能的视觉目标;(ii) 对全局兴趣区域进行评分以指导导航;(iii) 基于视觉记忆进行链式思维(CoT)推理以自信作答。该框架维护一个容量固定的场景记忆,存储一组固定数量的区域-目标假设,并在线更新,避免无界增长,有效应对单目标与多目标问题。为高效扩展覆盖范围,全局探索策略将狭窄通道和门视为高价值前沿,与局部目标搜索互补,计算开销极小。上述组件共同聚焦注意力,提升场景覆盖率与答案可靠性,且运行速度显著优于以往方法。在HMEQA与EXPRESS-Bench上达到顶尖表现,同时在OpenEQA与MT-HM3D上表现竞争力。
原文摘要 · Abstract (English)
Embodied Question Answering (EQA) combines visual scene understanding, goal-directed exploration, spatial and temporal reasoning under partial observability. A central challenge is to confine physical search to question-relevant subspaces while maintaining a compact, actionable memory of observations. Furthermore, for real-world deployment, fast inference time during exploration is crucial. We introduce FAST-EQA, a question-conditioned framework that (i) identifies likely visual targets, (ii) scores global regions of interest to guide navigation, and (iii) employs Chain-of-Thought (CoT) reasoning over visual memory to answer confidently. FAST-EQA maintains a bounded scene memory that stores a fixed-capacity set of region-target hypotheses and updates them online, enabling robust handling of both single and multi-target questions without unbounded growth. To expand coverage efficiently, a global exploration policy treats narrow openings and doors as high-value frontiers, complementing local target seeking with minimal computation. Together, these components focus the agent's attention, improve scene coverage, and improve answer reliability while running substantially faster than prior approaches. On HMEQA and EXPRESS-Bench, FAST-EQA achieves state-of-the-art performance, while performing competitively on OpenEQA and MT-HM3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。