用幂律衰减损失提升大模型生成文本的多样性与信息量
Power-Law Decay Loss for Large Language Model Finetuning: A Theory Perspective
- 按词频幂律衰减重加权，低频词权重更高
- 理论推导支持该设计，可改善生成内容质量
- 适合摘要、对话、风格迁移等文本生成任务
在文本生成任务的微调阶段，标准交叉熵损失对所有词一视同仁，导致模型过度关注高频但信息量低的词汇，忽略对内容独特性和信息性至关重要的低频词。本文提出一种新型损失函数——幂律衰减损失（Power-Law Decay Loss, PDL），其核心思想源于信息论与语言学观察：词的信息量与其出现频率成反比。PDL根据训练语料中词的频率，对标准交叉熵损失中的每个词进行幂律衰减加权，降低高频词权重，提高低频、信息密集词的权重。这一机制引导模型在微调过程中更关注学习和生成具有特定含义的词汇，从而提升生成文本的质量、多样性和信息密度。论文从理论上阐述了PDL的设计动机与构造逻辑，并探讨其在抽象摘要、对话系统、风格迁移等文本生成任务中的潜在应用与优势。
原文摘要 · Abstract (English)
During the finetuning stage of text generation tasks, standard cross-entropy loss treats all tokens equally. This can lead models to overemphasize high-frequency, low-information tokens, neglecting lower-frequency tokens crucial for specificity and informativeness in generated content. This paper introduces a novel loss function, Power-Law Decay Loss (PDL), specifically designed to optimize the finetuning process for text generation. The core motivation for PDL stems from observations in information theory and linguistics: the informativeness of a token is often inversely proportional to its frequency of occurrence. PDL re-weights the contribution of each token in the standard cross-entropy loss based on its frequency in the training corpus, following a power-law decay. Specifically, the weights for high-frequency tokens are reduced, while low-frequency, information-dense tokens are assigned higher weights. This mechanism guides the model during finetuning to focus more on learning and generating tokens that convey specific and unique information, thereby enhancing the quality, diversity, and informativeness of the generated text. We theoretically elaborate on the motivation and construction of PDL and discuss its potential applications and advantages across various text generation finetuning tasks, such as abstractive summarization, dialogue systems, and style transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。