arXiv:2607.12790cs.AIcs.CL2026-07

让评估指标和大模型技能一起进化,自动提升评估可靠性。

Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents

  • 用小型检测器组合演化评估函数,不依赖固定任务集
  • 在代码生成上与隐藏真值达成0.21一致率(p=0.014)
  • 适合需要可信自评系统的安全关键场景

自演化智能体系统可自主创建、修改和淘汰技能,但这一过程依赖可靠的评估指标。在多数真实应用中,这类指标并不存在。本文提出让评估指标本身也成为演化的对象:通过在完整演化生命周期中搜索小规模类型化缺陷检测器的组合,以与十项锚定参考集的一致性为目标,并通过对无标签输出的共识进行正则化。所演化的对象是评分函数,而非固定的评价任务集;结果是一个可解释的表达式,而非黑箱裁判。实验表明,该方法在代码生成任务上与隐藏真值达到0.21的一致性(配对检验,p=0.014),优于其内部的原始大模型评判者。有效性即安全性:移除锚定会致使评估器退化为始终通过的无效检测器,而移除检测器生命周期则不会,反转了技能演化的启示。这警示我们,下游任务得分无法验证自演化评估器的有效性,因失效评估器仍能有效训练技能。任务得分仅反映充分性,而演化的评估器已足够:双棘轮机制(Double Ratchet)在MBPP+、Spider~2.0-Snow和报告生成任务上,保持了88–110%的基准或人工评分标准带来的性能增益。当演化技能试图欺骗报告评分标准时,独立裁判发现异常,新增检测器修复问题。

原文摘要 · Abstract (English)

Self-evolving agent systems create, revise, and retire their own skills, but every such loop assumes a reliable evaluation metric already exists. In many real applications none does. We show the metric itself can be the evolving object: our loop searches compositions of small typed drawback detectors under a full evolutionary lifecycle, selecting for agreement with a ten-item anchored reference set and regularizing by consensus over unlabeled outputs. What evolves is the function that grades one output, never the fixed task sets it is scored on, and what comes out is an inspectable expression rather than an opaque judge. It is also valid: on code generation it gains 0.21 agreement with hidden ground truth on a locked set that metric selection never reads (paired $p=0.014$), beating the bare LLM judge it contains. Validity is where safety lives: removing the anchor guards collapses the metric into a vacuous always-pass detector while removing the detector lifecycle does not, inverting the lesson from skill evolution. That collapse warns this line of work that downstream task score cannot validate a self-evolved evaluator, since the collapsed metric trains skills just as well. Task score answers only sufficiency, and an evolved metric suffices: \emph{Double Ratchet}, co-evolving the metric with a lifecycle-managed skill loop, retains 88--110\% of the lift ground truth or a hand-written rubric buys, across MBPP+, Spider~2.0-Snow, and report generation. When evolved skills gamed the report rubric, an independent judge caught it and one added detector repaired it.

自进化评估演化大模型安全双棘轮

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。