用一次微调解决通用与领域学习分离问题，提升效率与性能。
Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
- 先用通用数据零学习率训练获取全局梯度，用于后续领域训练
- 相比原始微调，联合性能平均提升5.8%，且无需反复调参
- 仅需领域数据，减少91%的GPU耗时，适合资源受限场景
大型语言模型微调效果显著，但传统方法常需复杂数据混合和多次实验以获得最佳泛化性能。为解决此问题并简化训练流程，我们提出一种高效通用方案——动态增强退火（DBA）。通过在通用数据上进行零学习率训练获取全局梯度，该梯度随后用于梯度增强与动态训练步长校正，结合退火机制，在仅使用领域数据的情况下实现稳定微调且无性能坍塌。在多个主流基座模型上的多任务评估表明，DBA相较原始微调在联合性能上平均提升5.8%。由于通用数据不再参与退火过程，依赖数据混合的重复实验也被消除。测试显示，该方法可减少91.0%的GPU小时数。
原文摘要 · Abstract (English)
Large language models (LLMs) fine-tuning shows excellent implications. However, vanilla fine-tuning methods often require intricate data mixture and repeated experiments for optimal generalization. To address these challenges and streamline the training process, we propose an efficient and universal solution, Dynamic Boosted Annealing (DBA). We obtain a global gradient through zero-learning-rate training on general data, which is subsequently employed for gradient boosting and dynamic training step correction during domain training. In conjunction with annealing learning, we end up establishing a fine-tuning pipeline that relies solely on domain data without collapse. By evaluating both general and domain-specific performance across multiple tasks on several popular base models, DBA achieves an average improvement of 5.8% in joint performance over vanilla fine-tuning. Furthermore, since general data is no longer involved in annealing, repeated experiments led by data mixture are also eliminated. According to our tests, the DBA method can reduce GPU hours by 91.0% compared to the vanilla method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。