用统计方法把验证器评分转为有可靠错误率控制的决策,提升智能体系统安全性
E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing
- 将验证评分转化为可在线监控的序列假设检验,每步都有统计保障
- 在6个数据集上比其他方法更少误报且检测力更强
- 可快速终止无效流程,节省计算资源,适合部署在复杂智能体中
智能体系统根据用户提示执行一系列操作(如推理步骤或工具调用)。为评估其行为轨迹的成功性,研究者开发了验证器(如大模型裁判和过程奖励模型),用于评分每个动作的质量。尽管这些启发式评分具有参考价值,但无法保证判断正确性。本文提出 e-valuator,一种将任意黑盒验证器评分转化为具有可证明误报率控制的决策规则的方法。我们将区分成功轨迹(最终能正确响应用户提示)与失败轨迹的问题建模为序列假设检验问题。e-valuator 借助 e-过程工具,构建出在智能体每一步操作时都保持统计有效的序列检验,支持对任意长操作序列进行在线监控。实验表明,在六个数据集和三种智能体上,e-valuator 比其他策略具有更高的统计功效和更好的误报率控制。此外,e-valuator 可用于快速终止异常轨迹,节省计算开销。e-valuator 提供了一种轻量、模型无关的框架,将验证器启发式评分转化为具备统计保障的决策规则,推动更可靠的智能体系统部署。
原文摘要 · Abstract (English)
Agentic AI systems execute a sequence of actions, such as reasoning steps or tool calls, in response to a user prompt. To evaluate the success of their trajectories, researchers have developed verifiers, such as LLM judges and process-reward models, to score the quality of each action in an agent's trajectory. Although these heuristic scores can be informative, there are no guarantees of correctness when used to decide whether an agent will yield a successful output. Here, we introduce e-valuator, a method to convert any black-box verifier score into a decision rule with provable control of false alarm rates. We frame the problem of distinguishing successful trajectories (that is, a sequence of actions that will lead to a correct response to the user's prompt) and unsuccessful trajectories as a sequential hypothesis testing problem. E-valuator builds on tools from e-processes to develop a sequential hypothesis test that remains statistically valid at every step of an agent's trajectory, enabling online monitoring of agents over arbitrarily long sequences of actions. Empirically, we demonstrate that e-valuator provides greater statistical power and better false alarm rate control than other strategies across six datasets and three agents. We additionally show that e-valuator can be used for to quickly terminate problematic trajectories and save tokens. Together, e-valuator provides a lightweight, model-agnostic framework that converts verifier heuristics into decisions rules with statistical guarantees, enabling the deployment of more reliable agentic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。