让多模态智能体不仅能回忆，还能分析长期记忆中的信息。
Beyond Retrieval: Analytic Memory for Multimodal Agents
- 将重复的多模态数据结构化为可查询的分析型记忆
- 在两个基准上分别提升11.3%和6.9%性能
- 适合需要长期理解与推理的对话系统开发者
长期多模态记忆不仅要支持信息检索，还需能对跨交互积累的观察进行计算。现有系统主要依赖检索记忆，通过摘要与索引组织交互历史，实现从高层抽象到底层记录的多粒度返回。本文提出分析型记忆作为补充抽象，将重复的多模态观察组织成支持过滤、聚合、排序和时间比较的可查询结构。我们提出AdaMM框架，联合支持检索与分析记忆。它不依赖应用定义的模式，而是从对话、图像和上下文元数据中提取带有来源关联的属性-值观测，发现重复字段结构并实例化以供分析访问。推理时，记忆感知规划器将查询分解为检索与分析操作，并路由至相应工具。在两个长期多模态记忆基准MemEye和MemGallery上的实验表明，AdaMM分别提升了11.3%和6.9%的性能。
原文摘要 · Abstract (English)
Long-term multimodal memory must support not only retrieving relevant information but also computing over observations accumulated across interactions. Existing systems largely emphasize \emph{retrieval memory}, organizing interaction histories through summaries and indexes to return query-relevant information at multiple granularities, from high-level abstractions to underlying records. In this paper, we formulate \emph{analytic memory} as a complementary abstraction that organizes recurring multimodal observations into queryable structures supporting filtering, aggregation, ranking, and temporal comparison. We present AdaMM, a framework that jointly supports retrieval and analytic memory. Rather than relying on application-defined schemas, AdaMM extracts provenance-linked attribute-value observations from dialogue, images, and contextual metadata, discovers recurring field structures, and materializes them for analytical access. At inference time, a memory-aware planner decomposes queries into retrieval and analytic operations and routes each operation to the appropriate tools. Experiments on two long-term multimodal memory benchmarks, MemEye and MemGallery, show that AdaMM improves performance by up to 11.3\% and 6.9\%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。