定位大模型记忆系统错误根源，实现自动修复与性能提升。
MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems
- 将记忆流程转为可执行的演化图，追踪信息流动路径。
- 发现记忆失败源于操作级问题，如信息丢失与检索错位。
- 基于溯源结果优化提示词，闭环提升任务表现最高7.62%。
记忆对支持大语言模型长时推理至关重要，但现有记忆系统不可靠且难调试。追踪记忆动态演化是理解信息合成、传播或损坏过程的关键。本文研究大模型记忆系统中的错误追踪与归因问题，提出新框架将记忆流水线转化为可执行的记忆演化图，实现操作信息流的细粒度追踪。构建了MemTraceBench基准，涵盖Long-Context、RAG、Mem0和EverMemOS等代表性系统，系统分析记忆失效模式。提出自动归因方法，通过迭代追踪操作子图，精确定位失败案例的根本原因。分析表明，记忆失败具有系统性，源自操作级问题如信息丢失与检索错位。关键的是，利用这些细粒度归因信号指导下游提示词优化，建立闭环系统，自动纠正故障，使端任务性能提升最高达7.62%。代码将开源于https://github.com/zjunlp/MemTrace。
原文摘要 · Abstract (English)
Memory is essential for enabling large language models to support long-horizon reasoning, yet existing memory systems remain unreliable and difficult to debug. Tracing memory's dynamic evolution is crucial to understand how information is synthesized, propagated, or corrupted over time. In this work, we study the new problem of error tracing and attribution in LLM memory systems. We propose a novel framework that transforms memory pipelines into executable memory evolution graphs, enabling fine-grained tracing of operational information flow. We then construct MemTraceBench, a benchmark collected from representative memory systems such as Long-Context, RAG, Mem0, and EverMemOS, to systematically study memory failure modes. We further introduce an automatic attribution method that iteratively traces operation subgraphs to pinpoint the root cause of any failed case. Our analysis reveals that memory failures are systematic, stemming from operation-level issues like information loss and retrieval misalignment. Crucially, we leverage these fine-grained attribution signals to guide downstream prompt optimization, establishing a closed-loop system that automatically corrects faults and boosts end-task performance by up to 7.62%. Code will be released at https://github.com/zjunlp/MemTrace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。