解决大模型合并中的安全与性能冲突问题,提升多任务模型可靠性。
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint
- 通过梯度定位、重要性融合与参数隔离三阶段,精准识别并保护关键神经元。
- 在Llama-3-8B上使有害响应率下降31.4%,同时保持95%的任务性能。
- 适合需要安全可靠的多任务大模型部署的开发者和研究者。
为特定任务微调大型语言模型(LLMs)带来高昂的计算与数据成本。模型合并虽可免训练整合多个专用模型,但现有方法存在安全-性能冲突:通用能力增强会削弱安全防护。本文识别出两大根源:基于参数幅度的简单选择导致神经元误判,以及合并过程中的跨任务神经元干扰。为此提出LED-Merging框架,包含三阶段:通过梯度归因定位任务专属神经元,利用多模型重要性融合动态筛选关键神经元,通过参数隔离实现冲突更新解耦。在Llama-3-8B、Mistral-7B和Llama2-13B上的实验表明,该方法有效降低有害响应率,在Llama-3-8B-Instruct上于HarmBench测试中减少31.4%;同时保留95%的实用性,如在GSM8K上达到52.39%准确率。LED-Merging成功缓解安全-性能冲突,提供一种轻量、免训练的可靠多任务大模型构建范式。代码已开源。
原文摘要 · Abstract (English)
Fine-tuning pre-trained Large Language Models (LLMs) for specialized tasks incurs substantial computational and data costs. While model merging offers a training-free solution to integrate multiple task-specific models, existing methods suffer from safety-utility conflicts where enhanced general capabilities degrade safety safeguards. We identify two root causes: $\textbf{neuron misidentification}$ due to simplistic parameter magnitude-based selection, and $\textbf{cross-task neuron interference}$ during merging. To address these challenges, we propose $\textbf{LED-Merging}$, a three-stage framework that $\textbf{L}$ocates task-specific neurons via gradient-based attribution, dynamically $\textbf{E}$lects critical neurons through multi-model importance fusion, and $\textbf{D}$isjoints conflicting updates through parameter isolation. Extensive experiments on Llama-3-8B, Mistral-7B, and Llama2-13B demonstrate that LED-Merging effectively reduces harmful response rates, showing a 31.4\% decrease on Llama-3-8B-Instruct on HarmBench, while simultaneously preserving 95\% of utility performance, such as achieving 52.39\% accuracy on GSM8K. LED-Merging resolves safety-utility conflicts and provides a lightweight, training-free paradigm for constructing reliable multi-task LLMs. Code is available at $\href{https://github.com/MqLeet/LED-Merging}{GitHub}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。