让多智能体系统自己找错并不断改进诊断能力
Towards Self-Improving Error Diagnosis in Multi-Agent Systems

- 用三阶段流程自动定位错误源头和责任智能体
- 在两个基准测试中显著提升错误步骤定位精度
- 无需人工标注,可跨领域迁移且持续优化
基于大语言模型的多智能体系统虽能解决复杂问题,但调试困难,表现为交互痕迹长、智能体间依赖强、错误显现延迟。现有诊断方法依赖昂贵的人工标注或LLM作为裁判,难以在长上下文中精确定位关键错误步骤。本文提出ErrorProbe,一种自提升的语义失败归因框架,可识别责任智能体及错误起始步骤。该框架采用三阶段流程:(1)将系统失败分类体系转化为可操作的局部异常检测;(2)通过症状驱动的逆向追溯剔除无关上下文;(3)由策略师、调查员、仲裁员组成的专用多智能体团队,通过工具化执行验证错误假设。关键在于,ErrorProbe维护一个仅在可执行证据确认后才更新的已验证情节记忆,无需标注。在TracerTraj与Who&When基准测试中,ErrorProbe显著优于基线,尤其在步骤级定位上表现突出,且已验证记忆支持无需重训练的跨领域迁移。
原文摘要 · Abstract (English)
Large Language Model (LLM)-based Multi-Agent Systems (MAS) enable complex problem-solving but introduce significant debugging challenges, characterized by long interaction traces, inter-agent dependencies, and delayed error manifestation. Existing diagnostic approaches often rely on expensive expert annotation or ''LLM-as-a-judge'' paradigms, which struggle to pinpoint decisive error steps within extended contexts. In this paper, we introduce ErrorProbe, a self-improving framework for semantic failure attribution that identifies responsible agents and the originating error step. The framework operates via a three-stage pipeline: (1) operationalizing the MAS failure taxonomy to detect local anomalies, (2) performing symptom-driven backward tracing to prune irrelevant context, and (3) employing a specialized multi-agent team (Strategist, Investigator, Arbiter) to validate error hypotheses through tool-grounded execution. Crucially, ErrorProbe maintains a verified episodic memory that updates only when error patterns are confirmed by executable evidence, without the need for annotation. Experiments across the TracerTraj and Who&When benchmarks demonstrate that ErrorProbe significantly outperforms baselines, particularly in step-level localization, while the verified memory enables robust cross-domain transfer without retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。