用f散度构建新损失函数，提升语言模型训练效果。
Loss Functions and Operators Generated by f-Divergences
- 基于f散度构造新型凸损失函数，推广逻辑损失。
- α=1.5的α散度损失在预训练、微调和蒸馏中表现优异。
- 提出f-softargmax计算算法，支持并行化高效求解。
逻辑损失（即交叉熵损失）是多分类任务中最常用的损失函数，也是语言建模中下一个词预测的首选。它与KL散度及softargmax算子相关联。本文提出基于f散度构建新的凸损失函数，从两个方向推广逻辑损失：一是用f散度替代KL散度，二是允许非均匀参考测度。我们为多种f散度实例化该框架，既恢复了已有损失，也生成了新损失。与逻辑损失类似，每种由f散度生成的损失函数对应一个算子，称为f-softargmax。我们推导出一种新颖的可并行化二分查找算法，用于计算任意f散度对应的f-softargmax。实验方面，本文旨在评估超越传统交叉熵的损失函数在语言模型中的有效性，涵盖预训练、后训练（SFT）和知识蒸馏。结果表明，当参考测度为单位测度时，α=1.5的α散度损失（等价于Tsallis α负熵）在多个任务上表现良好。
原文摘要 · Abstract (English)
The logistic loss (a.k.a. cross-entropy loss) is one of the most popular loss functions used for multiclass classification. It is also the loss function of choice for next-token prediction in language modeling. It is associated with the Kullback--Leibler (KL) divergence and the softargmax operator. In this work, we propose to construct new convex loss functions based on $f$-divergences. Our loss functions generalize the logistic loss in two directions: i) by replacing the KL divergence with $f$-divergences and ii) by allowing non-uniform reference measures. We instantiate our framework for numerous $f$-divergences, recovering existing losses and creating new ones. By analogy with the logistic loss, the loss function generated by an $f$-divergence is associated with an operator, that we dub $f$-softargmax. We derive a novel parallelizable bisection algorithm for computing the $f$-softargmax associated with any $f$-divergence. On the empirical side, one of the goals of this paper is to determine the effectiveness of loss functions beyond the classical cross-entropy in a language model setting, including on pre-training, post-training (SFT) and distillation. We show that the loss function generated by the $α$-divergence (which is equivalent to Tsallis $α$-negentropy in the case of unit reference measures) with $α=1.5$ performs well across several tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。