调温度提升多样性常失效，换损失函数更有效。
Improving Diversity in Language Models: When Temperature Fails, Change the Loss
- 用精度-召回框架重设计损失函数，替代单纯调温度。
- 新方法在精度与召回平衡上显著优于传统温度缩放。
- 适合追求生成多样性和质量平衡的研究者。
提升语言模型的多样性是一项挑战性且至关重要的任务。常用方法是提高解码温度，但本研究通过一个简单却常见的案例揭示：降低温度可提升生成质量（精度），而提高温度往往无法有效增加覆盖范围（召回）。分析表明，模型若未针对覆盖率进行训练，则难以通过温度调节实现有效调控。为此，我们提出重新思考语言模型的损失函数设计，基于精度-召回框架优化。实验结果表明，该方法在精度与召回的权衡上显著优于仅使用负对数似然训练结合温度缩放的方案。这些发现为构建更灵活、更鲁棒的语言建模技术提供了新路径。
原文摘要 · Abstract (English)
Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach through a simplistic yet common case to provide insights into why decreasing temperature can improve quality (Precision), while increasing it often fails to boost coverage (Recall). Our analysis reveals that for a model to be effectively tunable through temperature adjustments, it must be trained toward coverage. To address this, we propose rethinking loss functions in language models by leveraging the Precision-Recall framework. Our results demonstrate that this approach achieves a substantially better trade-off between Precision and Recall than merely combining negative log-likelihood training with temperature scaling. These findings offer a pathway toward more versatile and robust language modeling techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。