arXiv:2605.30844cs.CLcs.AI2026-05

微调让语言模型生成更高效,信息量更大。

Fine-Tuning Improves Information Conveyance in Language Models

  • 提出树状视角的熵度量Canopy Entropy,同时捕捉输出长度与内容的不确定性。
  • 微调后模型在输出长度与语义多样性间相关性提升近三倍。
  • 适合研究模型生成效率、对齐效果或优化提示工程的研究者。

微调常被认为会降低大语言模型的不确定性和多样性,但现有分析忽略了输出长度这一关键混杂因素,未能完整刻画不确定性在整个生成过程中的分布。为此,我们提出Canopy Entropy(CE*),从树形视角看待语言生成,将‘树冠’定义为所有可能生成路径的集合,使CE*自然衡量生成空间的有效规模。该度量联合捕捉输出长度N和序列Y₁:N的不确定性,其值等于总香农熵H(N, Y₁:N∣X),其中X为提示。该公式导出可解释指标,包括长度-熵率相关项ρ(N, r_N),量化每令牌的信息传递效率。实验表明,在多种任务和模型家族中,微调模型虽总熵下降,但ρ(N, r_N)显著增强;控制模型家族、任务、提示和输出长度后,微调使熵率与语义多样性间的相关性强度几乎提升三倍,说明对齐模型能更高效地将词元不确定性转化为语义多样性。结果表明,微调并非简单减少不确定性,而是从根本上重组为更具信息量和语义意义的生成。

原文摘要 · Abstract (English)

Fine-tuning is often believed to reduce uncertainty and diversity in large language models, but existing analyses overlook output length, a key confounder, and therefore fail to capture how uncertainty is distributed across an entire generation rollout. To address this, we propose Canopy Entropy ($\mathrm{CE}^\star$), a measure that views language generation from a tree perspective, where ``canopy'' represents the space of all possible rollouts, making $\mathrm{CE}^\star$ naturally quantify the effective size of the generation space. $\mathrm{CE}^\star$ jointly captures uncertainty in both the output length $N$ and the generated sequence $Y_{1:N}$ -- indeed, we show that it equals to total Shannon entropy $H(N, Y_{1:N}\mid X)$, where $X$ denotes the prompt. This formulation yields interpretable metrics, including a length-entropy correlation term $ρ(N, r_N)$, where $r_N$ is the entropy rate, quantifying information conveyance efficiency by indicating whether longer outputs are more or less informative per token. Empirically, across tasks and model families, we find that fine-tuned models consistently exhibit stronger positive correlation $ρ(N, r_N)$, even when total entropy decreases. Furthermore, after controlling for model family, task, prompt, and output-length effects, we find that fine-tuning nearly triples the correlation strength between entropy rate and semantic diversity, suggesting that aligned models convert token uncertainty into semantic diversity more efficiently. Overall, these results demonstrate that fine-tuning does not simply reduce uncertainty, but fundamentally reorganizes it into more informative and semantically meaningful generations. Our code is available at https://github.com/WeiyiTian/canopy-entropy.

语言模型信息效率微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。