arXiv:2605.08715cs.CLcs.AI2026-05被引 1

提出在线审计框架,提前发现多智能体系统中的致命错误。

AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems

论文配图:AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems
图 1 · 摘自论文原文
  • 构建在线审计机制,在每一步判断是否应报警,不依赖未来信息。
  • 在2000条轨迹上训练出7B模型,比GPT-4.1等领先模型提升19.9%准确率。
  • 可定位错误发生的具体步骤与责任智能体,适合部署时实时干预场景。

基于大语言模型的多智能体系统常用于长周期任务,但单个关键错误可能被下游智能体接受并引发整体失败。现有方法多为事后归因,无法在任务进行中干预。本文提出AgentForesight框架,将问题转为在线审计:每一步仅基于当前前缀,决定继续或报警,不访问未来信息。为此,构建AFTraj-2K数据集,涵盖编码、数学和智能体领域,通过多模型裁判共识标注致命错误发生步。在此基础上,训练AgentForesight-7B模型,采用粗到细强化学习策略,先建立风险预判先验,再通过三轴奖励精细定位错误的‘谁、何时、何事’。在AFTraj-2K和外部Who&When基准上,该模型超越GPT-4.1和DeepSeek-V4-Pro,性能最高提升19.9%,步骤定位误差降低3倍,实现从事后检测到部署期干预的闭环。

原文摘要 · Abstract (English)

LLM-based multi-agent systems are increasingly deployed on long-horizon tasks, but a single decisive error is often accepted by downstream agents and cascades into trajectory-level failure. Existing work frames this as \emph{post-hoc failure attribution}, diagnosing the responsible agent and step after the trajectory has ended. However, this paradigm forfeits any opportunity to intervene while trajectory is still unfolding. In this work, we introduce AgentForesight, a framework that reframes this problem as online auditing: at each step of an unfolding trajectory, an auditor observes only the current prefix and must either continue the run or alarm at the earliest decisive error, without access to future steps. To this end, we curate AFTraj-2K, a corpus of agentic trajectories across Coding, Math, and Agentic domains, in which safe trajectories are retained under a strict curation pipeline and unsafe trajectories are annotated at the step of their decisive error via consensus among multiple LLM judges. Built on that, we develop AgentForesight-7B, a compact online auditor trained with a coarse-to-fine reinforcement learning recipe that first equips it with a risk-anticipation prior at the failure boundary on adjacent safe/unsafe prefix pairs, then sharpens this prior into precise step-level localization under a three-axis reward jointly targeting the what, where, and who of an audit verdict. Across AFTraj-2K and an external Who\&When benchmark, AgentForesight-7B outperforms leading proprietary models, including GPT-4.1 and DeepSeek-V4-Pro, achieving up to +19.9% performance gain and 3$\times$ lower step localization error, opening the loop from post-hoc failures detection to enabling deployment-time intervention. Project page: https://zbox1005.github.io/agent-foresight/

多智能体在线审计故障预测LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。