提出可落地的持续评估框架，解决大模型评估中的系统性缺陷。
Beyond Static Snapshots: A Grounded Evaluation Framework for Language Models at the Agentic Frontier
- 用确定性验证器替代奖励模型，从设计上杜绝奖励黑客
- 在三个模型上实现最高25.6个百分点的能力提升，最差仅下降5.6个百分点
- 支持消费级设备运行，适合追求可靠评估的研究者与开发者
现有大语言模型评估框架存在分布、时间、范围和过程四类系统性失效，导致强化学习人类反馈（RLHF）中奖励黑客成为可预测结果而非偶然现象。我们提出基于现实的持续评估（GCE）框架，并以ISOPro为实现范例。ISOPro采用确定性验证器替代学习型奖励模型，在可验证领域从根本上消除奖励黑客；通过在CPU上更新LoRA适配器，将硬件门槛降低一个数量级。我们在Qwen 2.5 3B、Llama 3.2 3B、Gemma 2 2B三款模型及任务调度、MBPP两个领域验证，与GRPO-LoRA进行匹配算力对比。十二组测试中，ISOPro最大能力提升达+25.6、+22.2、+16.0个百分点，均值增益+9.0个百分点，最坏情况仅下降5.6个百分点；而GRPO-LoRA在消费级配置下峰值增益仅+8.5个百分点，最坏下降10个百分点，均值反而下降1.5个百分点。在MBPP组合泛化任务中，ISOPro在三款模型中有两款达到40%准确率（包括对Qwen 2.5 3B实现0%到40%的跃升），而GRPO-LoRA仅一款达20%。我们识别出一种缓冲区偏斜失效模式，其在三种前提下会削弱已有能力层级，并提出三种缓解策略。本工作与DeepSeek-R1的GRPO在大规模下达成相同架构共识：对于可验证奖励领域，验证器即奖励信号。
原文摘要 · Abstract (English)
We argue that current evaluation frameworks for large language models (LLMs) suffer from four systematic failures that make them structurally inadequate for deployed, agentic systems: distributional, temporal, scope, and process invalidity. These failures compound in RLHF, making reward hacking a predictable consequence of evaluation design rather than an unpredictable training pathology, and RLHF's dual-model architecture imposes a hardware barrier limiting evaluation reproducibility. We propose the Grounded Continuous Evaluation (GCE) framework and present ISOPro as a reference implementation. ISOPro replaces the learned reward model with a deterministic verifier, eliminating reward hacking by construction in verifiable-reward domains, and updates LoRA adapters on CPU, reducing the hardware barrier by an order of magnitude. We validate ISOPro across three architectures (Qwen 2.5 3B, Llama 3.2 3B, Gemma 2 2B) and two domains (scheduling, MBPP), with a head-to-head matched-compute comparison against GRPO-LoRA. Across twelve cells, ISOPro produces the largest absolute capability gains (+25.6, +22.2, +16.0pp) at mean delta +9.0pp and worst-case regression -5.6pp; GRPO-LoRA at consumer-budget hyperparameters reaches a smaller peak gain (+8.5pp), deeper worst-case regression (-10pp), and mean delta -1.5pp. Held-out compositional generalization on MBPP reaches 40% for ISOPro on two of three architectures (including a 0% to 40% bootstrap on Qwen 2.5 3B), against 20% for GRPO-LoRA on one of three. We characterize a buffer-skew failure mode in which the implicit curriculum can erode pre-existing tier capability under three preconditions, with three corresponding mitigations. The work is situated alongside DeepSeek-R1's GRPO, which arrived at the same architectural conclusion at scale: for verifiable-reward domains, the verifier is the reward signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。