提出新方法ECHO，精准定位多智能体系统中的错误来源。
Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution
- 构建分层上下文表示，结合目标分析与共识投票
- 在复杂推理和依赖关系场景下准确率显著提升
- 适合调试协作式AI系统，尤其对微妙错误敏感
大型语言模型多智能体系统的错误归因是调试和改进协作式AI系统的重要挑战。现有方法在分析交互轨迹中的智能体与步骤级故障时，无论采用一次性评估、逐步分析还是二分搜索，面对复杂模式时均存在准确性和一致性不足的问题。本文提出ECHO（通过上下文层次与客观共识分析进行错误归因）算法，融合分层上下文表征、基于目标的评估及共识投票机制，提升错误归因精度。该方法通过位置感知的上下文层级化理解，保持客观评价标准，最终通过共识机制得出结论。实验表明，ECHO在多种多智能体交互场景中优于现有方法，尤其在处理细微推理错误和复杂依赖关系时表现突出。研究结果表明，结构化的分层上下文表征与基于共识的客观决策相结合，为多智能体系统错误归因提供了更稳健的框架。
原文摘要 · Abstract (English)
Error attribution in Large Language Model (LLM) multi-agent systems presents a significant challenge in debugging and improving collaborative AI systems. Current approaches to pinpointing agent and step level failures in interaction traces - whether using all-at-once evaluation, step-by-step analysis, or binary search - fall short when analyzing complex patterns, struggling with both accuracy and consistency. We present ECHO (Error attribution through Contextual Hierarchy and Objective consensus analysis), a novel algorithm that combines hierarchical context representation, objective analysis-based evaluation, and consensus voting to improve error attribution accuracy. Our approach leverages a positional-based leveling of contextual understanding while maintaining objective evaluation criteria, ultimately reaching conclusions through a consensus mechanism. Experimental results demonstrate that ECHO outperforms existing methods across various multi-agent interaction scenarios, showing particular strength in cases involving subtle reasoning errors and complex interdependencies. Our findings suggest that leveraging these concepts of structured, hierarchical context representation combined with consensus-based objective decision-making, provides a more robust framework for error attribution in multi-agent systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。