提出空间对齐记忆架构,让机器人连续问答更准确高效。
Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

- 用3D几何结构化存储视觉语义信息,支持跨问题记忆
- 相比传统方法,准确率提升18.7%,导航成本降低32%
- 适合需要持续交互的机器人系统开发者
传统具身问答(EQA)评估采用独立回合制,每轮任务后重置内部状态。但真实机器人需持续运行并累积、保留与复用先前交互信息。本文研究不同记忆架构在连续多问场景下的表现:仅保留可通行性信息(如2D占用图)的代理无法保存视觉-语义证据;基于短时序回合数据训练的代理在连续多问中遭遇严重时间错配,难以形成可复用的场景表征。实验表明,将持久视觉观测映射到度量3D几何结构的记忆架构能构建连贯的场景表示,在模拟环境中同时实现更高准确率(+18.7%)和更低导航开销(-32%)。实机验证进一步证明,空间对齐的视觉记忆对物理环境中的连续智能操作至关重要。
原文摘要 · Abstract (English)
Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. However, real-world robots operate continuously and must accumulate, retain, and selectively reuse information acquired from prior interactions. Despite this practical requirement, the architectural mechanisms needed to support sequential memory in EQA remain underexplored. In this work, we investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries. We find that simply preserving existing memory is often insufficient. Agents that retain only traversability information, such as 2D occupancy maps, remember where the robot has explored but not the visual-semantic evidence needed for later questions. Agents trained on short-horizon episodic data face a different challenge: when exposed to continuous, multi-query histories, their inherited context suffers from severe temporal mismatch, rather than forming a reusable scene representation. To overcome this architectural bottleneck, we highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation. Extensive experiments in simulated environments reveal that this form of memory breaks the accuracy-efficiency tradeoff in sequential settings, simultaneously achieving higher answer accuracy and lower navigation costs. We further validate these findings on a real-world mobile robot, demonstrating that spatially grounded visual memory is critical for enabling continuous, intelligent operation in physical environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。