大模型判官准确预测失败，未必能有效防止失败。
Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention
- 用50个任务的小规模测试预判干预效果，避免部署后性能崩溃。
- 高成功率任务上干预导致性能下降26个百分点，高失败任务仅提升2.8个百分点。
- 发现干预存在破坏与恢复的权衡，关键在识别何时不该干预。
大型语言模型（LLM）批评者常被认为能主动干预以提升可靠性，但其实际部署效果尚不明确。我们发现，一个离线准确率极高的二分类LLM批评者（AUROC 0.94）在实际部署中可能引发严重性能下降：在一个模型上导致26个百分点（pp）的性能坍塌，而在另一个模型上影响几乎为零。这种差异表明，仅靠批评者准确性不足以判断干预是否安全。我们揭示了‘干扰-恢复’权衡：干预虽可挽救失败轨迹，但也可能扰乱本可成功的轨迹。基于此，提出一种预部署测试方法，仅需50个任务小样本即可预测干预是否有益，无需全量部署。在多个基准测试中，该方法成功预判结果：高成功率任务上干预使性能下降0至-26 pp，而高失败的ALFWorld基准则获得+2.8 pp提升（p=0.014）。框架核心价值在于识别不应干预的场景，防止部署前出现严重退化。
原文摘要 · Abstract (English)
Proactive interventions by LLM critic models are often assumed to improve reliability, yet their effects at deployment time are poorly understood. We show that a binary LLM critic with strong offline accuracy (AUROC 0.94) can nevertheless cause severe performance degradation, inducing a 26 percentage point (pp) collapse on one model while affecting another by near zero pp. This variability demonstrates that LLM critic accuracy alone is insufficient to determine whether intervention is safe. We identify a disruption-recovery tradeoff: interventions may recover failing trajectories but also disrupt trajectories that would have succeeded. Based on this insight, we propose a pre-deployment test that uses a small pilot of 50 tasks to estimate whether intervention is likely to help or harm, without requiring full deployment. Across benchmarks, the test correctly anticipates outcomes: intervention degrades performance on high-success tasks (0 to -26 pp), while yielding a modest improvement on the high-failure ALFWorld benchmark (+2.8 pp, p=0.014). The primary value of our framework is therefore identifying when not to intervene, preventing severe regressions before deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。