提出一种新采样方法，让大模型推理更准且稳定。
Top-$nσ$: Not All Logits Are You Need
- 基于前softmax logits的统计阈值筛选，自动分离噪声与有效信息
- 在4个推理数据集上优于现有方法，高温下仍保持稳定性能
- 适合追求高精度和鲁棒性的大模型推理应用
大型语言模型通常在推理任务中采用贪婪解码或低温采样，反映出多样性与准确性的权衡。本文挑战这一常规，提出一种名为 top-$nσ$ 的新型采样方法，直接作用于前softmax logits，利用统计阈值进行高效筛选。核心洞察是：logits天然分为服从高斯分布的噪声区域和独立的有意义区域，可无需复杂概率变换实现精准过滤。与现有方法（如 top-$p$、min-$p$）不同，top-$nσ$ 在高温下不会意外引入更多噪声词，保持采样空间稳定。我们还提供了 top-$nσ$ 的理论分析以深入理解其行为。在四个聚焦推理的数据集上的广泛实验表明，该方法不仅超越现有采样策略，还优于贪婪解码，并在高温度下维持一致性能。
原文摘要 · Abstract (English)
Large language models (LLMs) typically employ greedy decoding or low-temperature sampling for reasoning tasks, reflecting a perceived trade-off between diversity and accuracy. We challenge this convention by introducing top-$nσ$, a novel sampling method that operates directly on pre-softmax logits by leveraging a statistical threshold. Our key insight is that logits naturally separate into a Gaussian-distributed noisy region and a distinct informative region, enabling efficient token filtering without complex probability manipulations. Unlike existing methods (e.g., top-$p$, min-$p$) that inadvertently include more noise tokens at higher temperatures, top-$nσ$ maintains a stable sampling space regardless of temperature scaling. We also provide a theoretical analysis of top-$nσ$ to better understand its behavior. The extensive experimental results across four reasoning-focused datasets demonstrate that our method not only outperforms existing sampling approaches but also surpasses greedy decoding, while maintaining consistent performance even at high temperatures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。