arXiv:2509.03312cs.CLcs.MA2025-09被引 87

首个自动定位大模型代理系统故障根源的框架,提升诊断准确率超18%。

AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?

  • 通过反事实重播与故障注入生成故障轨迹数据集
  • 在Who&When基准上比谷歌、claude等大模型高18.18%
  • 可让现有代理系统性能提升4.8%-14.2%,支持自我修复

基于大语言模型(LLM)的代理系统通常由多个模型、复杂的工具调用和编排协议组成,显著优于单体代理。然而,其复杂性也加剧了系统的脆弱性,更易发生故障。在长执行轨迹中定位具体导致错误的代理或步骤,是代理系统故障归因的核心挑战。当前最先进的推理型LLM在该任务上表现极差,准确率普遍低于10%。为此,我们提出AgenTracer,首个通过反事实重播与程序化故障注入自动标注失败多代理轨迹的框架,构建了精选数据集TracerTraj。基于此资源,我们开发了AgenTracer-8B——一个采用多粒度强化学习训练的轻量级故障追踪器,能高效诊断复杂多代理交互中的错误。在Who&When基准上,AgenTracer-8B相比巨量闭源模型Gemini-2.5-Pro和Claude-4-Sonnet最高提升18.18%,树立了新标准。更重要的是,AgenTracer-8B可为MetaGPT、MaAS等现成多代理系统提供可操作反馈,带来4.8%-14.2%的性能提升,推动代理型AI实现自我修正与演化。

原文摘要 · Abstract (English)

Large Language Model (LLM)-based agentic systems, often comprising multiple models, complex tool invocations, and orchestration protocols, substantially outperform monolithic agents. Yet this very sophistication amplifies their fragility, making them more prone to system failure. Pinpointing the specific agent or step responsible for an error within long execution traces defines the task of agentic system failure attribution. Current state-of-the-art reasoning LLMs, however, remain strikingly inadequate for this challenge, with accuracy generally below 10%. To address this gap, we propose AgenTracer, the first automated framework for annotating failed multi-agent trajectories via counterfactual replay and programmed fault injection, producing the curated dataset TracerTraj. Leveraging this resource, we develop AgenTracer-8B, a lightweight failure tracer trained with multi-granular reinforcement learning, capable of efficiently diagnosing errors in verbose multi-agent interactions. On the Who&When benchmark, AgenTracer-8B outperforms giant proprietary LLMs like Gemini-2.5-Pro and Claude-4-Sonnet by up to 18.18%, setting a new standard in LLM agentic failure attribution. More importantly, AgenTracer-8B delivers actionable feedback to off-the-shelf multi-agent systems like MetaGPT and MaAS with 4.8-14.2% performance gains, empowering self-correcting and self-evolving agentic AI.

故障诊断多智能体LLM自修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。