用真实轨迹自动生成可解释的评分标准，减少对失败任务的误判。
Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation
- 从少量真实标注轨迹中自动生成可读评分标准，基于环境奖励优化。
- 在tau-bench上误通过率降低至0.115，仅为传统方法的一半。
- 输出为可解释文本，每条判断都有明确依据，适合需可信评估的场景。
大规模评估语言模型智能体越来越依赖第二语言模型作为自动评判者，因为真实奖励信号（可执行环境奖励）成本高、速度慢或部署时不可用。这类评判者是无奖励的代理，其可信度取决于能否准确反映真实结果。现有方法要么人工编写评分标准（如G-Eval），要么微调评判模型权重，但都倾向于将流畅但失败的轨迹判定为成功。本文提出RubricForge，从少量带真实标签的轨迹中自动推导出评判标准，使其基于真实结果。该方法通过反射式进化不断优化评分标准，使其与环境奖励高度一致，冻结后仅需一次模型调用即可对未见轨迹进行评分，无需环境访问。最终生成的评分标准为可读文本，使每项判决均可追溯到具体标准。使用一个7B参数模型同时作为智能体和评判者，在tau-bench（173条标注轨迹，来自220次采样）和WebShop（160条）上测试显示，主要优势在于判断忠实性而非整体一致性。与通用的G-Eval相比，差异不显著（McNemar p = 0.248），绝对分数校准甚至略差（|err| 差异 -0.048，p = 2×10⁻⁴）。然而，RubricForge的误通过率显著更低（tau-bench上0.115 vs. 0.173），且在WebShop上的排序忠实度更高（斯皮尔曼相关系数0.410 vs. 0.370）。对于无奖励评估器而言，误通过率才是关键指标——误通过会部署故障智能体，而误拒绝仅导致重试。
原文摘要 · Abstract (English)
Evaluating language-model agents at scale increasingly relies on a second language model as an automatic judge, because the gold signal, an executable environment reward, is expensive, slow, or unavailable at deployment time. Such a judge is a reward-free proxy whose value depends on whether it can be trusted, yet existing judges either hand-write the scoring rubric, as in G-Eval, or fine-tune the judge's weights, and both tend to credit fluent but unsuccessful trajectories as successes. We instead induce the text of an agent-judging rubric from a small set of ground-truth-labeled trajectories, grounding it in true outcomes. We present RubricForge, which evolves a judge rubric by reflective evolution against labeled trajectories to maximize agreement with the environment reward, freezes it, and applies it to held-out trajectories in one model call with no environment access. The optimized artifact is human-readable text, so every verdict is attributable to named criteria. Using one frozen 7B model as both agent and judge, on tau-bench (173 labeled trajectories drawn from 220 rollouts) and WebShop (160), the principal gain is faithfulness rather than raw agreement. The edge over a generic G-Eval judge is not statistically significant (McNemar p = 0.248), and absolute-score calibration marginally favors the generic judge (|err| difference -0.048, p = 2x10^-4). Yet RubricForge over-credits failed trajectories roughly half as often (0.115 vs. 0.173 false-pass rate on tau-bench, with three over-credit catches and zero reversals) and ranks graded WebShop outcomes more faithfully (Spearman 0.410 vs. 0.370). For a reward-free evaluator the false-pass rate, not aggregate agreement, is the deployment-relevant quantity, since a false pass ships a broken agent whereas a false fail merely costs a retry.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。