模型自博弈，无需新数据就能持续提升性能。
Language Self-Play For Data-Free Training
- 让模型与自己对弈，通过竞争机制自我优化。
- 在指令遵循、数学和编码任务上，模型性能显著提升。
- 适合想在无额外数据下优化大模型的研究者。
近年来，大语言模型的快速发展依赖于规模、高质量训练数据和强化学习。然而，这一进展面临根本瓶颈：需要不断获取更多数据以持续学习。本文提出一种强化学习方法，通过自博弈（Language Self-Play, LSP）框架，使模型在不依赖新增数据的情况下实现自我提升。该方法将模型能力视为博弈中的表现，通过让模型与自身对抗，逐步产生更强策略。在 Llama-3.2-3B-Instruct 模型上进行的实验表明，仅通过自博弈即可有效提升其在指令遵循、数学推理和代码生成等基准上的表现。
原文摘要 · Abstract (English)
Large language models (LLMs) have advanced rapidly in recent years, driven by scale, abundant high-quality training data, and reinforcement learning. Yet this progress faces a fundamental bottleneck: the need for ever more data from which models can continue to learn. In this work, we propose a reinforcement learning approach that removes this dependency by enabling models to improve without additional data. Our method leverages a game-theoretic framework of self-play, where a model's capabilities are cast as performance in a competitive game and stronger policies emerge by having the model play against itself-a process we call Language Self-Play (LSP). Experiments with Llama-3.2-3B-Instruct on instruction-following, mathematics, and coding benchmarks show that pretrained models can be effectively improved with self-play alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。