定位大模型多智能体系统故障根源,精准找出出错的智能体和最早失误步骤。
Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems

- 用大模型判断+多视角独立验证,结合置信度聚合定位故障。
- 在两个基准上均优于现有方法,且耗时和令牌消耗低。
- 适合研究多智能体系统可解释性与可靠性的人看。
基于大语言模型(LLM)的多智能体系统通过协同推理与行动实现复杂任务求解,但其分布式结构也带来了系统级故障诊断的新挑战。当执行失败时,由于长周期交互和智能体行为高度耦合,难以确定是哪个智能体导致失败,以及轨迹何时开始不可逆地偏离。本文研究了这一问题,提出AgentLocate框架,能够将失败归因于特定智能体及最早的决定性步骤。该框架结合大模型判断机制与独立评估者的多视角验证,通过置信度感知策略聚合评估结果,并利用反馈对裁判模型进行轻量微调以提升归因质量。我们在两个互补基准上评估了AgentLocate,涵盖多样任务、智能体配置与轨迹长度。实验表明,它在识别责任智能体和故障步骤方面持续优于现有方法,同时在令牌使用量和运行时间上保持高效。
原文摘要 · Abstract (English)
Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures. When an execution fails, identifying which agent is responsible and at what point the trajectory first becomes irreversibly misdirected is difficult due to long-horizon interactions and tightly coupled agent behaviors. In this paper, we study the problem of failure localization in LLM-based multi-agent systems and present AgentLocate, a framework that attributes failures to both a specific agent and the earliest decisive step. AgentLocate combines an LLM-based judging mechanism with multi-perspective verification by independent evaluators, whose assessments are aggregated using a confidence-aware strategy. The resulting feedback is further used to adapt the judge through lightweight fine-tuning, improving attribution quality. We evaluate AgentLocate on two complementary benchmarks covering diverse tasks, agent configurations, and trajectory lengths. Experimental results show that AgentLocate consistently outperforms existing failure localization methods in identifying both responsible agents and failure steps, while remaining efficient in terms of token usage and running time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。