通过选择性合并参数缓解大模型微调中的数据顺序偏差
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
- 按不同数据顺序训练多个模型后，选择性合并参数
- 在五个数据集上优于传统加权平均法，提升微调效果
- 适合关注大模型微调稳定性和性能优化的研究者
监督微调（SFT）对将大语言模型（LLM）适配到具体任务至关重要。本文发现，训练数据的顺序会导致显著的训练不平衡，可能引发性能下降。为此，我们提出通过合并以不同数据顺序微调的SFT模型来缓解该问题，从而提升SFT的整体有效性。此外，我们引入一种新方法——‘参数选择性合并’，在五个数据集上的表现优于传统加权平均方法。通过分析与消融实验，验证了该方法的有效性，并识别出性能提升的来源。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is crucial for adapting Large Language Models (LLMs) to specific tasks. In this work, we demonstrate that the order of training data can lead to significant training imbalances, potentially resulting in performance degradation. Consequently, we propose to mitigate this imbalance by merging SFT models fine-tuned with different data orders, thereby enhancing the overall effectiveness of SFT. Additionally, we introduce a novel technique, "parameter-selection merging," which outperforms traditional weighted-average methods on five datasets. Further, through analysis and ablation studies, we validate the effectiveness of our method and identify the sources of performance improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。