用规则化延期判断提升大模型评估效率,比专用权重更优。
Share the Judge, Learn the Deferral: Where Specialization Helps LLM Evaluation
- 用轻量级规则判断替代专用模型权重,实现高效评估
- 规则延期使89.4%准确率仅用41.5%算力,满足95%风险约束
- 统一训练后微调分拆专家,比从零训练更可靠
智能体生成速度远超人工评审。本文对比两种大模型评估器专业化策略:专用判别器权重与基于规则的延期决策机制。在99,952个带评分标准的样本上,正确评分标准提升准确率2.11点,错误标准则降低2.66点。将训练数据分给八个特定标准的LoRA专家导致准确率下降10.05点,5%误差覆盖范围从24.44%降至5.43%,且该损失与模型规模和训练设置无关;通过统一判别器初始化专家可基本恢复性能。在不同数据预算下验证,从零开始的专业化无实证增益。温暖启动的分拆模型表现接近统一模型,但在数据有限时,统一训练仍更优,专业化仅在统一训练达到瓶颈后才有效。该结论在HealthBench上同样成立:医生制定的评分标准提升准确率,有缺陷的标准则降低性能。相比权重专业化,延期策略更高效。在RewardBench 2上,轻量级延期头构成0.6B-4B-8B奖励级联,无需修改核心评分,20次分裂中达89.40%准确率,优于独立8B判别器的84.75%准确率,仅需41.5%算力并满足95%风险约束。基于置信度的延期虽精度相当,但算力成本更高。该设计在Tulu-3-8B、Skywork-8B等模型上均有效,结合轻量DeBERTa前端提升性能。最终提出简单稳健的评估器设计原则:统一训练或温暖启动分拆模型,并使用经审计的延期级联实现低成本、高可靠的评估。
原文摘要 · Abstract (English)
Agentic systems generate outputs faster than human review. We contrast two LLM evaluator specialization strategies: specialized judge weights, or rule-based deferral policies for safe judgment acceptance. On 99,952 rubric-conditioned samples, correct rubrics improve accuracy by 2.11 points, while incorrect rubrics reduce performance by 2.66 points. Splitting training data across eight criterion-specific LoRA experts lowers accuracy by 10.05 points and reduces 5% error-bound coverage from 24.44% to 5.43%. This loss is independent of model size and training settings, with most performance recoverable by warm-starting experts from a unified judge. Sweeping data budgets confirms scratch expert specialization yields no empirical gains. Warm-started splits appear competitive with unified models, yet under limited data, unified training outperforms split training, with specialization beneficial only after unified training plateaus. Results hold on HealthBench, where physician rubrics improve accuracy while flawed rubrics degrade performance. Unlike weight specialization, deferral policies enable efficient evaluation. On RewardBench 2, lightweight deferral heads form a 0.6B-4B-8B reward cascade with no core scoring modification. Across 20 splits, the cascade achieves 89.40% accuracy versus 84.75% for a standalone 8B judge at 41.5% compute, satisfying 95% risk constraints. Margin-based deferral matches accuracy at far higher compute cost. The design generalizes across models, improving Tulu-3-8B and Skywork-8B performance with a lightweight DeBERTa frontend. We derive simple, robust evaluator design rules: unify judgment training or warm-start split models, and use audited deferral cascades for low-cost, reliable LLM evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。