用回归模型让AI裁判更准地预测人类评分。
Quantitative LLM Judges
- 用回归模型对齐AI裁判评分与人类偏好。
- 在四个数据集上提升原有裁判的评分预测能力。
- 适合缺乏人工标注时的高效评估场景。
LLM-as-a-judge 是一种由大型语言模型（LLM）评估另一 LLM 输出的框架。尽管 LLM 在生成定性文本评价方面表现优异，但其预测人类偏好和数值评分的能力通常较弱。我们提出定量 LLM 判官，通过回归模型将现有 LLM 判官的评分与特定领域中的人类评分对齐。模型利用原始判官的推理过程和评分进行训练以优化其得分。我们构建了四种针对不同类型绝对与相对反馈的定量判官，展示了该框架的通用性与灵活性。相比监督微调，本框架计算效率更高；在人工反馈有限的情况下，统计效率也更优，这在实际应用中尤为常见。我们在四个数据集上使用两个基础判官进行了实验验证，结果表明定量判官可通过后处理建模显著提升现有判官的预测能力。
原文摘要 · Abstract (English)
LLM-as-a-judge is a framework where a large language model (LLM) evaluates the output of another LLM. While LLMs excel at producing qualitative textual evaluations, they often struggle to predict human preferences and numeric scores. We propose quantitative LLM judges, which align evaluation scores of existing LLM judges to humans in a given domain using regression models. The models are trained to improve the score of the original judge using its rationale and score. We present four quantitative judges for different types of absolute and relative feedback, which showcases the generality and versatility of our framework. Our framework is more computationally efficient than supervised fine-tuning and can be more statistically efficient when human feedback is limited, which is expected in practice. We validate these claims empirically on four datasets using two base judges. Our experiments show that quantitative judges can improve the predictive power of existing judges through post-hoc modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。