首个跨模态个人记忆问答基准，测试长期个性化记忆推理能力
According to Me: Long-Term Personalized Referential Memory QA
- 用结构化框架统一多源记忆数据，支持跨模态引用理解
- 在硬样本集上模型准确率低于20%，暴露现有系统瓶颈
- 适合研究个性化记忆、多源推理与长时记忆的学者使用
个性化AI助手需回忆并推理长期用户记忆，这些记忆自然涵盖图像、视频、邮件等多种模态和来源。然而，现有长期记忆评测主要聚焦对话历史，无法反映基于真实生活经验的个性化引用。我们提出ATM-Bench，首个面向多模态、多源个性化引用记忆问答的基准。ATM-Bench包含约四年的隐私保护个人记忆数据，以及人工标注的问答对和真实记忆证据，涵盖需解析个人引用、多源证据推理及冲突证据处理的查询。我们提出基于模式的内存（SGM）来结构化表示来自不同来源的记忆项。实验中，我们实现了5个前沿记忆系统及标准RAG基线，评估了不同记忆摄入、检索与生成技术的变体。结果发现，在ATM-Bench-Hard集上性能极差（准确率低于20%），且SGM相比传统描述性记忆显著提升表现。代码已开源：https://github.com/JingbiaoMei/ATM-Bench
原文摘要 · Abstract (English)
Personalized AI assistants must recall and reason over long-term user memory, which naturally spans multiple modalities and sources such as images, videos, and emails. However, existing Long-term Memory benchmarks focus primarily on dialogue history, failing to capture realistic personalized references grounded in lived experience. We introduce ATM-Bench, the first benchmark for multimodal, multi-source personalized referential Memory QA. ATM-Bench contains approximately four years of privacy-preserving personal memory data and human-annotated question-answer pairs with ground-truth memory evidence, including queries that require resolving personal references, multi-evidence reasoning from multi-source and handling conflicting evidence. We propose Schema-Guided Memory (SGM) to structurally represent memory items originated from different sources. In experiments, we implement 5 state-of-the-art memory systems along with a standard RAG baseline and evaluate variants with different memory ingestion, retrieval, and answer generation techniques. We find poor performance (under 20\% accuracy) on the ATM-Bench-Hard set, and that SGM improves performance over Descriptive Memory commonly adopted in prior works. Code available at: https://github.com/JingbiaoMei/ATM-Bench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。