让机器人记住物体跨时间的去向,解决长期环境中的记忆丢失问题。
LT-Mem: Volatility-Aware Spatio-Temporal Memory for Lifelong Scene Understanding

- 基于动态波动性判断物体记忆该更新、保留还是存多假设。
- 在多会话场景下实现90%以上物体身份识别准确率,推理耗能低一个数量级。
- 适合需要长期感知与历史追溯的机器人导航、智能助手等应用。
在持续演化的环境中,机器人需具备跨会话的物体级理解能力。现有系统或覆盖旧信息以保持地图最新,或仅存储语义快照而缺乏一致的物体身份,导致时间性遗忘:无法回答“绿色椅子在所有会话中出现在哪里”这类问题。本文提出LT-Mem,一种基于波动性的时空记忆演化框架,融合空间对齐的实例级3D感知与波动性条件下的时序推理。首先,多会话SLAM骨干网络提供跨会话的物体空间对齐观测;其次,推理层控制记忆演化:确定性证据评分维持跨会话身份一致性,波动性感知策略根据物体动态选择覆盖、保留或多重假设动作;第三,形成的三重记忆结构(实时、增量、元)同时保存当前状态与事件历史,支持纵向物体中心推理。进一步构建了包含多会话录制、持久身份标注与时间问答对的LT-VQA数据集与评估套件。实验表明,LT-Mem在各项指标上均显著优于基线,且令牌消耗降低一个数量级;消融实验证实性能提升源于结构化记忆设计而非大模型容量。
原文摘要 · Abstract (English)
Long-term robot operation in evolving environments requires object-level understanding that persists across repeated revisits. Existing systems either overwrite history to maintain an up-to-date map or store semantic snapshots without consistent cross-session object identity, resulting in temporal amnesia: the systematic loss of object history that prevents answering queries such as "Where has the green chair been across all sessions?" We propose LT-Mem, a volatility-aware memory evolution framework that unifies spatially aligned instance-level 3D perception with volatility-conditioned temporal reasoning. First, a multi-session SLAM backbone provides spatially aligned per-object observations across sessions. Second, a reasoning layer governs how object memory evolves: deterministic evidence scoring preserves cross-session identity, and a volatility-aware policy selects among overwrite, hold, and multi-hypothesis actions based on each object's dynamics. Third, the resulting Tri-Memory structure (Live, Delta, Meta) preserves both current states and event histories, enabling longitudinal object-centric reasoning. We further introduce LT-VQA, a dataset and evaluation suite comprising multi-session recordings, persistent identity annotations, and temporal QA pairs. Experiments show that LT-Mem consistently outperforms baselines across all metrics while consuming an order of magnitude fewer tokens, and ablations confirm that gains are driven by the structured memory architecture rather than LLM capacity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。