解决大模型合并时的参数干扰问题，提升合并后性能。
Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
- 基于损失变化评估参数重要性，保留关键权重减少内部干扰。
- 通过二值掩码分步融合模型，避免直接平均参数降低外部干扰。
- 在微调和预训练检查点上均显著优于现有方法，适合模型定制场景。
大型语言模型（LLMs）展现出强大能力，但其高昂的计算成本给定制化带来挑战。模型合并提供了一种低成本替代方案，但现有方法存在参数间干扰问题，导致性能下降。本文提出最优脑迭代合并（OBIM），旨在缓解模型内与模型间的干扰。OBIM包含两个核心组件：(1) 一种显著性度量机制，通过个体权重变动引发的损失变化评估参数重要性，仅保留高显著性参数以减少模型内干扰；(2) 一种互斥的迭代合并框架，采用二值掩码逐步整合模型，避免直接参数平均，从而减轻模型间干扰。我们在监督微调（SFT）模型和后续预训练检查点上验证了OBIM的有效性，实验结果表明其显著优于现有合并技术。总体而言，OBIM为提升大模型合并效果提供了高效且实用的解决方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated impressive capabilities, but their high computational costs pose challenges for customization. Model merging offers a cost-effective alternative, yet existing methods suffer from interference among parameters, leading to performance degradation. In this work, we propose Optimal Brain Iterative Merging (OBIM), a novel method designed to mitigate both intra-model and inter-model interference. OBIM consists of two key components: (1) A saliency measurement mechanism that evaluates parameter importance based on loss changes induced by individual weight alterations, reducing intra-model interference by preserving only high-saliency parameters. (2) A mutually exclusive iterative merging framework, which incrementally integrates models using a binary mask to avoid direct parameter averaging, thereby mitigating inter-model interference. We validate OBIM through experiments on both Supervised Fine-Tuned (SFT) models and post-pretrained checkpoints. The results show that OBIM significantly outperforms existing merging techniques. Overall, OBIM provides an effective and practical solution for enhancing LLM merging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。