让大模型自动定位沉默错误，还能通过实验验证并优化判断。
REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces
- 用特定修复方案重放可疑步骤，验证错误位置。
- 在四个基准上准确率领先，结构化工具调用场景提升显著。
- 无需真实答案也能给出可操作的错误定位，适合调试智能体。
大语言模型（LLM）代理通过长串计划与执行轨迹完成复杂任务，但对已完成轨迹中错误的定位能力仍严重不足，尤其在‘沉默失败’场景下。现有方法依赖分类器或大模型评判预测可疑步骤，或通过重试恢复正确答案，但均未将干预结果反馈以改进错误归因。本文提出 extit{REFLECT}，通过诊断候选错误步骤，使用针对性修复方案进行受控重放，并利用验证后结果的翻转作为对比证据，动态优化最终归因。在覆盖多跳推理跨领域的四个定位基准上， extit{REFLECT} 在同审计者方法中表现最优，尤其在结构化工具使用轨迹上提升最大；即使无真实答案时，也能提供可操作的定位结果。
原文摘要 · Abstract (English)
Large language model (LLM) agents now solve complex tasks through long plan-and-execution traces, yet the ability to locate errors in a completed traces still lags far behind, especially in the \emph{silent failure} regime. Existing approaches predict suspect steps via classifiers or LLM judges, or recover correct answers via retry, but none feed the intervention outcome back to \emph{refine the attribution itself}. We propose \methodname, a method that closes this gap by diagnosing a candidate error step, testing it through controlled replay with a diagnosis-specific patch, and using the verified outcome flip as contrastive evidence to refine the final attribution. Across four localization benchmarks spanning multi-hop reasoning across domains, \methodname achieves the highest localization accuracy among same-auditor methods across all four benchmarks, with the largest gains on structured tool-use traces, while providing actionable localization even when ground-truth answers are unavailable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。