用信息增益选训练数据，让大模型微调更省力有效。
FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain
- 基于信息增益筛选最有效的训练样本
- 相同计算量下，性能优于随机采样和均匀采样
- 适合资源有限但需高效微调的场景
监督微调（SFT）是将大语言模型（LLM）适配到新领域的标准方法。本文通过选择具有信息量的训练样本，提升SFT的统计效率。在固定训练样本预算（决定微调计算成本）的前提下，我们选取信息量最大的样本。核心思想是选择能最大化对数似然的海森矩阵信息增益。我们通过在线性化LLM最后一层并使用多项式逻辑回归模型，高效近似该信息增益。该方法计算高效、可分析且实证表现良好。我们在多个任务上验证了该方法，定量结果与大语言模型评估均支持其有效性。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is a standard approach to adapting large language models (LLMs) to new domains. In this work, we improve the statistical efficiency of SFT by selecting an informative subset of training examples. Specifically, for a fixed budget of training examples, which determines the computational cost of fine-tuning, we determine the most informative ones. The key idea in our method is to select examples that maximize information gain, measured by the Hessian of the log-likelihood of the LLM. We approximate it efficiently by linearizing the LLM at the last layer using multinomial logistic regression models. Our approach is computationally efficient, analyzable, and performs well empirically. We demonstrate this on several problems, and back our claims with both quantitative results and an LLM evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。