用动态调制解决大模型生成重复单调问题
Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
- 通过上下文感知与自适应去偏，动态调整生成概率分布
- 在高温解码下显著提升数学推理准确率，多样性与连贯性俱增
- 无需训练、兼容现有策略，适合高阶文本生成任务
在开放式生成中，大语言模型常陷入‘似然陷阱’，表现为重复性退化与词汇单调，导致机器生成文本与人类写作存在差异。尽管事后尾部截断（如Top-$p$、Min-$p$）可避免采样不可靠尾部，但可能过度采样未校准的头部，偏离人类词汇偏好；固定标量重复惩罚则忽略不同推理步长的对数几率尺度变化，可能破坏语义连贯性。为此，我们提出无训练预解码干预方法——方差校准调制（VCM），通过两项动态机制重塑截断前的概率分布：(1) 基于PMI的上下文搜索灯，抑制全局停用词，增强上下文触发词元；(2) 自适应自去偏，利用实时对数几率标准差实现尺度无关的惩罚。在开放式生成、事实问答与数学推理任务中，VCM持续缓解似然陷阱。计算开销极小，可集成至现有解码策略，提升多样性、连贯性，尤其在高解码温度下显著改善推理准确率。
原文摘要 · Abstract (English)
In open-ended generation, LLMs frequently fall into the "likelihood trap", marked by repetitive degeneration and vocabulary dullness, creating a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-$p$, Min-$p$) avoids sampling from the unreliable tail, it can over-sample from the uncalibrated head and misalign generation with human lexical preferences; fixed scalar repetition penalties likewise ignore variation in logit scale across inference steps, potentially disrupting semantic coherence. To address both limitations, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention that reshapes the probability distribution before truncation through two dynamic mechanisms: (1) Contextual Searchlight via PMI, which suppresses global stopwords while elevating context-evoked tokens, and (2) Adaptive Self-Debiasing, which uses real-time logit standard deviation for scale-invariant penalization. Across open-ended generation, factual QA, and mathematical reasoning, VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity, coherence, and, particularly at higher decoding temperatures, reasoning accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。