用轻量图网络识别多智能体系统中的故障源和错误类型。
Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution

- 构建基于图神经网络的AFANet,通过语义信号与关系建模轨迹。
- 参数量少、推理开销极低,性能媲美甚至超越大模型基线。
- 适合需要低成本部署的智能体系统故障诊断场景。
基于大语言模型(LLM)的多智能体系统常出现复杂故障,导致错误结果。为此提出智能体故障归因任务:给定失败轨迹,识别故障智能体及其错误类型。现有方法多依赖LLM,通过提示、微调或复杂代理流程实现,但存在长上下文处理、昂贵后训练及人工流程等计算开销。实证表明,即使顶尖模型在基准上准确率也有限,说明单纯扩大模型规模无效。本文重新审视该任务,质疑生成式方案的必要性。提出AFANet,一种轻量级图网络框架,通过步骤级语义信号与智能体级关系建模交互轨迹。实验显示,相比基线,AFANet以显著更少参数和近乎零推理成本,(i) 在域内基准上匹配或超越微调后的LLM,(ii) 对不同GNN架构保持鲁棒性能,(iii) 可通过低成本测试时适配进一步提升跨域表现。结果表明,高效故障归因无需重型LLM推理,轻量结构化方法即可实现优异效果。
原文摘要 · Abstract (English)
Large language model (LLM)-based multi-agent systems (MAS) often exhibit complex failure modes, which frequently cause agents to produce incorrect outcomes. This motivates the task of Agent Failure Attribution: given a failed multi-agent trajectory, identify the faulty agents and their corresponding error types. Existing approaches predominantly rely on LLMs to perform failure attribution, either through direct prompting, fine-tuning on synthetic data or complex agentic pipelines. While effective, these methods incur substantial computational overhead due to long-context processing, expensive post-training and handcrafted workflows. Moreover, empirical evidence shows that even state-of-the-art models achieve limited accuracy on existing benchmarks, suggesting that scaling model size alone is insufficient. In this work, we revisit this task and question the necessity of such expensive generative solutions. We introduce AFANet, a lightweight graph-based framework that models interaction trajectories through step-level semantic signals and agent-level relationships. We show that with significantly fewer parameters and near-zero inference cost, AFANet (i) matches or outperforms LLM-based baselines, including fine-tuned models on in-domain benchmarks, (ii) maintains robust performance across different GNN architectures and (iii) can be further improved with inexpensive test-time adaptation on the OOD benchmark. Our results suggest that effective agent failure attribution does not require heavy LLM reasoning and a lightweight, structured approach can achieve strong performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。