评测智能体长时记忆，发现现有系统依赖错误检索方法。
AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
- 构建真实与合成轨迹结合的评估基准，模拟长期交互过程。
- 新方法在基准上达57.22%准确率，比最强基线高11.16%。
- 适合研究长时记忆、自主智能体的学者和开发者。
大型语言模型正被广泛应用于复杂、长周期的自主智能体任务中，有效记忆对持续性能至关重要。然而现有记忆评测多集中于对话场景，而真实智能体记忆包含状态、动作、观察与工具输出构成的连续交互轨迹。为填补这一空白，我们提出AMA-Bench（Agent Memory with Any length），一个面向真实智能体场景下长时记忆的评测基准。AMA-Bench融合来自代表性应用的真实智能体轨迹与专家标注的问题问答，以及可扩展至任意长度的规则生成合成轨迹与对应问题。研究发现，现有记忆系统表现不佳，因其未能捕捉因果与客观信息，且过度依赖有损的相似性检索。为此，我们提出基于因果图构建与工具增强检索的AMA-Agent。该系统在AMA-Bench上取得57.22%的准确率，较最强基线提升11.16%。相关资源已开源：https://ama-bench.github.io/
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used as autonomous agents in complex, long-horizon applications, where effective memory is critical for sustained performance. Yet existing memory benchmarks are largely dialogue-centric, while real agent memory consists of continuous agent-environment interaction trajectories composed of states, actions, observations, and tool outputs. To address this gap, we introduce **AMA-Bench** (**A**gent **M**emory with **A**ny length), a benchmark for evaluating long-horizon memory in realistic agentic settings. AMA-Bench combines real-world agent trajectories from representative applications with expert-curated QA, as well as synthetic trajectories that scale to arbitrary horizons with rule-based QA. Our study shows that existing memory systems underperform because they fail to capture causal and objective information and rely heavily on lossy similarity-based retrieval. We further propose **AMA-Agent**, a memory system based on causality-graph construction and tool-augmented retrieval. AMA-Agent achieves **57.22%** accuracy on AMA-Bench, outperforming the strongest baseline by **11.16%**. Resources are available at: [https://ama-bench.github.io/](https://ama-bench.github.io/).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。