回收训练中被弃用的模型，通过优化合并提升性能。
If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs
- 用线性加权合并多个训练阶段的模型权重
- 在100B级模型上实现超越单个模型的综合性能
- 即使表现差的检查点也能为最优合并贡献价值
模型合并在整合专家模型方面展现出巨大潜力，但在合并训练于多任务的“通用”模型时，其优势尚不明确。本文研究了约1000亿参数规模的大模型在合并中的应用，聚焦于那些在不同任务间存在性能权衡的检查点。这些检查点通常在前沿模型开发过程中产生，次优者常被丢弃。针对来自不同训练阶段、目标、超参数和数据混合的模型检查点池，我们探索是否可通过合并将这些表现不佳的模型转化为帕累托最优模型。所提出的优化算法调整每个检查点在线性组合中的权重，生成的最终模型在性能上优于所有个体模型及基于合并的基线方法。进一步分析表明，优秀的合并往往包含几乎所有具有非零权重的检查点，说明即使是看似较差的初始检查点也对最终合并有积极贡献。
原文摘要 · Abstract (English)
Model merging has shown great promise at combining expert models, but the benefit of merging is unclear when merging "generalist" models trained on many tasks. We explore merging in the context of large (~100B) models, by recycling checkpoints that exhibit tradeoffs among different tasks. Such checkpoints are often created in the process of developing a frontier model, and the suboptimal ones are usually discarded. Given a pool of model checkpoints obtained from different training runs (e.g., different stages, objectives, hyperparameters, and data mixtures), which naturally show tradeoffs across different language capabilities (e.g., instruction following vs. code generation), we investigate whether merging can recycle such suboptimal models into a Pareto-optimal one. Our optimization algorithm tunes the weight of each checkpoint in a linear combination, resulting in such an optimal model that outperforms both individual models and merge-based baselines. Further analysis shows that good merges tend to include almost all checkpoints with non-zero weights, indicating that even seemingly bad initial checkpoints can contribute to good final merges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。