动态切换采样策略，让大模型生成更聪明且多样的答案。
Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs
- 根据错误风险动态选择贪心或高温采样
- 数学推理任务准确率提升，多样性不下降
- 轻量分类器几乎无延迟，适合实际部署
多样性是衡量语言模型创造力的关键指标。温度采样常用于提升输出多样性，但在需要高精度的任务（如数学推理）中，高温度采样（如min-p或top-p）会降低推理质量。我们发现，准确率下降源于在敏感解码位置采样了错误的延续。为此，本文提出「选择性采样」：基于采样风险度量，动态切换贪婪采样与高温采样。该风险度量评估在当前词元位置使用高温采样时出错的可能性。我们用少量可验证问题训练一个轻量级分类器，其可与基础语言模型集成，仅引入极小延迟。在数学推理任务上的实验表明，选择性采样显著改善了质量与多样性之间的权衡，即使在高温度设置下依然有效。
原文摘要 · Abstract (English)
Diversity is an essential metric for evaluating the creativity of outputs generated by language models. Temperature-based sampling is a common strategy to increase diversity. However, for tasks that require high precision, e.g., mathematical reasoning, uncontrolled high temperature sampling, e.g., min-$p$ or top-$p$, degrades reasoning quality. We demonstrate that the loss of accuracy is caused by sampling incorrect continuations in sensitive decoding positions. To address this, in this paper, we propose \textbf{selective sampling}, a method that dynamically switches between greedy and high-temperature sampling based on a sampling risk metric. This risk metric estimates the likelihood of output errors when applying high-temperature sampling on the current token position. To predict sampling risk, we train a lightweight classifier on a small subset of verifiable problems. The trained classifier can be integrated with the base language model with minimal latency overhead. Experiments on mathematical reasoning tasks demonstrate that selective sampling enhances the quality-diversity trade-off, even in high-temperature settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。