用轻量框架精准定位多智能体系统失败根源,效率提升79%。
StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

- 先用LLM提取日志语义,再用时序注意力模型捕捉步骤依赖关系。
- 在Who&When数据集上准确率超纯LLM方法,推理时间减少79%。
- 适合需高效故障诊断的复杂多步任务系统开发者使用。
基于大语言模型的多智能体系统在复杂多步任务中展现出卓越协作能力,但对单步执行错误高度敏感,错误会通过智能体交互传播并导致级联失败。为理解失败原因并提升系统可靠性,故障归因任务旨在自动识别导致失败的根本步骤。现有方法主要依赖大语言模型对原始执行轨迹进行推理,不仅带来高昂的推理成本与延迟,还受冗余和噪声日志干扰,难以准确识别真实根因步骤。为此,我们提出StepFinder,一个轻量级故障归因框架。该框架仅在特征构建阶段使用大语言模型,将执行日志编码为时序语义序列;随后采用参数高效的时序建模与注意力模块,捕捉轨迹的序列演化与跨步骤依赖关系;最后通过多尺度差异与位置偏置对步骤级错误得分进行精炼,实现精准根因定位。在Who&When基准测试上的实验表明,StepFinder在步骤级故障归因性能上优于基于大语言模型的方法,同时推理效率显著提升,相比最快的大语言模型方法推理时间减少79%,且无文本生成开销。代码已公开于https://github.com/taiyu-zhu/StepFinder。
原文摘要 · Abstract (English)
LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks. However, these systems are highly sensitive to single-step execution errors that can propagate through agent interactions and lead to cascading failures. To understand the causes of failure and improve system reliability, failure attribution has been introduced as a task that aims to automatically identify the root cause step responsible for a failure. Existing failure attribution methods mainly rely on LLMs to reason over original execution trajectories, which not only incur high inference costs and latency, but also suffer from interference caused by redundant and noisy execution logs, causing LLMs to struggle in accurately identifying the true root cause step. To address this, we propose StepFinder, a lightweight failure attribution framework. We use LLMs solely during the feature construction phase to encode execution logs into temporal semantic sequences. Subsequently, a parameter-efficient combination of temporal modeling and attention modules is applied to capture the sequential evolution and cross-step dependencies of the trajectories. Finally, the step-level error score is refined through multi-scale differences and position bias, enabling precise root cause identification. Experimental results on the Who&When benchmark demonstrate that StepFinder outperforms LLM-based methods in step-level failure attribution while achieving substantially higher inference efficiency, reducing inference time by 79% compared with the fastest LLM-based method, with no text generation overhead. Our code is available at https://github.com/taiyu-zhu/StepFinder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。