通过熵对齐提升语言模型生成质量，让写作更自然、推理更准确。
Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
- 用未来轨迹熵控制生成过程，避免传统方法的短视偏差。
- 在写作和摘要任务中胜过主流解码策略，多样性与忠实度双提升。
- 无需调参，适用于需要高质量生成的场景，如创作与数学推理。
语言模型虽在数十亿文本上训练，但直接采样仍生成质量低下。现有解码方法依赖贪心启发式，导致句子同质、重复且不连贯。本文提出EPIC，一种无超参数的解码方法，将未来轨迹熵纳入生成过程，显式调节每一步的不确定性，使其与数据固有随机性对齐。通过熵感知的懒惰Gumbel-Max采样，EPIC实现精确且高效，每步仅需亚线性熵评估次数。相比基线，其采样分布更贴近真实数据分布熵。在创意写作与摘要任务中，EPIC显著提升模型评判偏好胜率；自动指标也显示其生成更多样、摘要更忠实。在数学推理任务中，性能超越所有基线。
原文摘要 · Abstract (English)
Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM distribution to a set of high-probability continuations, but rely on greedy heuristics that introduce myopic distortions, yielding sentences that are homogeneous, repetitive and incoherent. In this paper, we introduce EPIC, a hyperparameter-free decoding approach that incorporates the entropy of future trajectories into LM decoding. EPIC explicitly regulates the amount of uncertainty expressed at every step of generation, aligning the sampling distribution's entropy to the aleatoric (data) uncertainty. Through Entropy-Aware Lazy Gumbel-Max sampling, EPIC manages to be exact, while also being efficient, requiring only a sublinear number of entropy evaluations per step. Unlike current baselines, EPIC yields sampling distributions that are empirically well-aligned with the entropy of the underlying data distribution. Across creative writing and summarization tasks, EPIC consistently improves LM-as-judge preference win-rates over widely used decoding strategies. These preference gains are complemented by automatic metrics, showing that EPIC produces more diverse generations and more faithful summaries. We also evaluate EPIC on mathematical reasoning, where it outperforms all baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。