用判断分布平均值替代贪婪解码，提升大模型判别文本质量的准确性。
Improving LLM-as-a-Judge Inference with the Judgment Distribution
- 用判断分布的均值代替模式（贪婪解码）提取偏好
- 均值法在点、对、列表三种评估场景下均优于传统方法
- 引入风险规避机制进一步提升性能，适合需高精度评估的场景
使用大语言模型自动评估文本质量（即 LLM-as-a-judge）已成为标准做法，广泛应用于各类任务。传统方法仅从模型输出文本中提取单一判断结果，通常采用贪婪解码。然而，大模型天然输出判断结果的分布，可支持更精细的推理方式。我们发现，在所有评估设置（点对、对对、列表）中，取判断分布的均值始终优于取模式（即贪婪解码）。我们进一步探索了从判断分布中推导偏好的新方法，发现引入风险规避策略常能提升性能。最后，我们分析了链式思维（CoT）提示对判别器的影响，发现其会压缩判断分布的范围，通常损害性能。结果表明，利用分布输出比仅依赖文本接口更具优势。
原文摘要 · Abstract (English)
Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) has become a standard practice applicable to many tasks. A judgment is often extracted from the judge's textual output alone, typically with greedy decoding. However, LLM judges naturally provide distributions over judgment tokens, inviting a breadth of inference methods for extracting fine-grained preferences. We find that taking the mean of the judgment distribution consistently outperforms taking the mode (i.e. greedy decoding) in all evaluation settings (i.e. pointwise, pairwise, and listwise). We further explore novel methods of deriving preferences from judgment distributions, and find that methods incorporating risk aversion often improve performance. Lastly, we analyze LLM-as-a-judge paired with chain-of-thought (CoT) prompting, showing that CoT can collapse the spread of the judgment distribution, often harming performance. Our findings show that leveraging distributional output improves LLM-as-a-judge, as opposed to using the text interface alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。