用主动调查框架解决长序列智能体故障诊断难题
SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation
- 用工具增强的诊断循环替代全轨迹加载，突破上下文限制
- 在100万词元预算下比顶尖方法提升20%，2.5万词元下提升19%
- 可定位超出模型原生上下文5倍的故障，传统方法完全失效
随着自主智能体处理日益复杂的多步骤、多智能体任务，其执行轨迹已超出最大上下文窗口的限制。现有故障诊断方法将完整轨迹载入大模型上下文，导致注意力稀释，当智能体轨迹超出上下文容量时完全失效。为此，我们提出SAFARI（通过主动调查实现长周期智能体故障归因的可扩展框架），以工具增强的诊断循环取代线性上下文加载。通过为大模型配备专用工具箱以读取和搜索轨迹片段，并结合持久化短期记忆（STM）进行跨轮次推理，SAFARI有效将诊断精度与架构上下文限制解耦。实验表明，在100万词元预算下，SAFARI在Who&When数据集上比现有最佳方法提升20%；在TRAIL GAIA子集上，2.5万词元预算下提升19%。最显著的是，当目标故障位于模型原生上下文窗口5倍之外时，SAFARI仍保持0.58的精确率，而传统评估器完全失败。
原文摘要 · Abstract (English)
As autonomous agents tackle increasingly complex multi-step, multi-agent tasks, their execution trajectories have scaled beyond the constraints of even the largest context windows. Current methods for effectively diagnosing agent failures load the full trajectory into an LLM's context window, which suffers from attention dilution and fails when agentic traces inevitably exceed context limits. To address this, we introduce SAFARI (Scaling long-horizon Agentic Fault AttRibution via active Investigation), a framework that replaces linear context loading with a tool-augmented diagnostic loop. By equipping LLMs with a specialized toolbox to read and search trajectory segments alongside a persistent Short-Term Memory (STM) for cross-turn reasoning, SAFARI effectively decouples diagnostic accuracy from architectural context limits. Our experiments demonstrate that SAFARI outperforms state-of-the-art results by 20% on the Who&When dataset within a 1M token budget, and by 19% on TRAIL GAIA subset on a 25K token budget. Most significantly, SAFARI maintains a 0.58 precision even when the target fault resides 5x beyond the model's native context window, a scenario where traditional evaluators fail entirely.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。