用视觉遗忘机制让设备端智能体高效存长期记忆
ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting
- 模仿生物遗忘，用降分辨率压缩旧记忆降低存储开销
- 在ATM-Bench上实现51.0%联合准确率与93%存储节省
- 适合资源受限设备上的多模态智能体长期记忆需求
由于存储成本高和多模态复杂性，大模型智能体在资源受限的边缘设备上实现长期个性化记忆极具挑战。为此，我们提出ScrapMem框架，将多模态数据整合为“剪贴簿页”。ScrapMem引入光学遗忘机制，逐步降低旧记忆的分辨率，降低存储开销并抑制低价值细节。为保持语义一致性，构建了事件记忆图（EM-Graph），将关键事件组织成因果时序结构。在多模态ATM-Bench上的大量实验表明，ScrapMem具备三大优势：(1) 强性能，达到新基准，联合准确率（Joint@10）达51.0%；(2) 高存储效率，通过光学遗忘将内存使用减少高达93%；(3) 更优召回，通过结构化聚合使召回率（Recall@10）提升至70.3%。ScrapMem为多模态大模型智能体在设备端的长期记忆提供了高效且节能的解决方案。
原文摘要 · Abstract (English)
Long-term personalized memory for LLM agents is challenging on resource-limited edge devices due to high storage costs and multimodal complexity. To address this, we propose ScrapMem, a framework that integrates multimodal data into "Scrapbook Page." ScrapMem introduces Optical Forgetting, an optical compression mechanism that progressively reduces the resolution of older memories, lowering storage cost while suppressing low-value details. To maintain semantic consistency, we construct an Episodic Memory Graph (EM-Graph) that organizes key events into a causal-temporal structure. Extensive experiments on the multimodal ATM-Bench showcase that ScrapMem provides three main benefits: (1) strong performance, achieving a new state-of-the-art with a 51.0% Joint@10 score; (2) high storage efficiency, reducing memory usage by up to 93% via optical forgetting; and (3) improved recall, increasing Recall@10 to 70.3% through structured aggregation. ScrapMem offers an effective and storage-efficient solution for on-device long-term memory in multimodal LLM agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。