提升大模型评分准确率的低成本方法，效果显著且适用广泛
On Cost-Effective LLM-as-a-Judge Improvement Techniques
- 用集成评分和任务特异性提示增强判断可靠性
- 在RewardBench 2上准确率达85.8%，比基线高13.5个百分点
- 小模型也能通过集成获高精度，适合资源有限场景
使用语言模型对候选回复进行打分或排序，已成为强化学习中人类反馈（RLHF）、基准测试及应用层评估中可扩展的替代方案。然而输出可靠性高度依赖提示设计与聚合策略。本文针对RewardBench 2，实证研究四种即插即用技术——集成评分、任务特异性标准注入、校准上下文与自适应模型升级，从随机评分器的噪声控制视角统一分析：集成作为单次调用噪声的蒙特卡洛平均，标准注入增强响应间差异判别力，单个响应得分方差作为不确定性信号。集成评分与任务特异性标准注入（后者几乎无成本）组合后准确率最高达85.8%，较基线提升13.5个百分点。校准上下文与自适应升级也优于基线，但在成本-精度帕累托前沿上被标准+集成组合超越。小模型在集成下受益更显著，使低成本高精度大模型评分器成为可能。这些技术在OpenAI GPT与Anthropic Claude系列模型间具有泛化能力。
原文摘要 · Abstract (English)
Using a language model to score or rank candidate responses has become a scalable alternative to human evaluation in reinforcement learning from human feedback (RLHF) pipelines, benchmarking, and application layer evaluations. However, output reliability depends heavily on prompting and aggregation strategy. We present an empirical investigation of four drop-in techniques -- ensemble scoring, task-specific criteria injection, calibration context, and adaptive model escalation -- for improving LLM judge accuracy on RewardBench 2, with a unifying lens of noise control on the stochastic judge: ensembling as Monte Carlo averaging over per-call noise, criteria injection as between-response discrimination sharpening, and per-response score variance as an uncertainty signal. Ensemble scoring and task-specific criteria injection (the latter virtually cost free) together reach up to 85.8% accuracy, +13.5pp over baseline. Calibration context and adaptive model escalation also improve over baseline but are dominated by criteria + ensembling on the cost-accuracy Pareto frontier. Small models benefit disproportionately from ensembling, making high-accuracy LLM judges accessible at low cost. We show that these techniques generalise across model providers, evaluating on both OpenAI GPT and Anthropic Claude families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。