系统梳理大模型后训练技术，揭示提升推理与对齐的关键方法。
LLM Post-Training: A Deep Dive into Reasoning Large Language Models
- 从微调、强化学习到测试时扩展，构建后训练完整技术体系。
- 指出灾难性遗忘、奖励劫持等核心挑战，提出应对策略。
- 适合关注模型对齐与推理优化的研究者与工程师参考。
大语言模型（LLMs）已重塑自然语言处理领域并催生多样应用。尽管在海量网络数据上预训练奠定了语言基础，研究界正逐步将重心转向后训练技术以实现进一步突破。预训练提供广泛语言基础，而后训练方法使模型能够细化知识、提升推理能力、增强事实准确性，并更有效地对齐用户意图与伦理考量。微调、强化学习及测试时扩展已成为优化模型性能、确保鲁棒性并提升跨实际任务适应性的关键策略。本综述系统探讨后训练方法，分析其在预训练之外优化大模型的作用，解决灾难性遗忘、奖励劫持和推理时权衡等关键挑战。文章还指出模型对齐、可扩展适配与推理时推理等新兴方向，并展望未来研究路径。同时提供公开仓库以持续追踪该快速演进领域的进展：https://github.com/mbzuai-oryx/Awesome-LLM-Post-training。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications. Pretraining on vast web-scale data has laid the foundation for these models, yet the research community is now increasingly shifting focus toward post-training techniques to achieve further breakthroughs. While pretraining provides a broad linguistic foundation, post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations. Fine-tuning, reinforcement learning, and test-time scaling have emerged as critical strategies for optimizing LLMs performance, ensuring robustness, and improving adaptability across various real-world tasks. This survey provides a systematic exploration of post-training methodologies, analyzing their role in refining LLMs beyond pretraining, addressing key challenges such as catastrophic forgetting, reward hacking, and inference-time trade-offs. We highlight emerging directions in model alignment, scalable adaptation, and inference-time reasoning, and outline future research directions. We also provide a public repository to continually track developments in this fast-evolving field: https://github.com/mbzuai-oryx/Awesome-LLM-Post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。