系统梳理奖励模型的构建与应用，助你快速入门强化学习对齐技术。
A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future
- 从偏好收集、建模到应用，构建奖励模型完整技术链条
- 总结主流评估基准与实际应用场景，涵盖对齐与生成任务
- 适合刚入门强化学习对齐或想了解奖励模型研究方向的人
奖励模型（Reward Model, RM）在提升大语言模型（LLM）性能方面展现出巨大潜力，能够作为人类偏好的代理信号，引导模型在各类任务中行为优化。本文全面综述相关研究，从偏好数据收集、奖励建模到实际应用展开分析。同时介绍奖励模型的典型应用场景及评估基准。深入探讨当前领域面临的关键挑战，并展望未来可能的研究方向。本综述旨在为初学者提供关于奖励模型的系统性入门指南，并推动后续研究发展。相关资源已公开于 GitHub（https://github.com/JLZhong23/awesome-reward-models）。
原文摘要 · Abstract (English)
Reward Model (RM) has demonstrated impressive potential for enhancing Large Language Models (LLM), as RM can serve as a proxy for human preferences, providing signals to guide LLMs' behavior in various tasks. In this paper, we provide a comprehensive overview of relevant research, exploring RMs from the perspectives of preference collection, reward modeling, and usage. Next, we introduce the applications of RMs and discuss the benchmarks for evaluation. Furthermore, we conduct an in-depth analysis of the challenges existing in the field and dive into the potential research directions. This paper is dedicated to providing beginners with a comprehensive introduction to RMs and facilitating future studies. The resources are publicly available at github\footnote{https://github.com/JLZhong23/awesome-reward-models}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。