用模型合并技术低成本添加新技能，还能更好保持安全特性。
Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging
- 先独立训练新技能，再与原模型合并，避免重复训练
- 在科学理解、安全、编程任务上效果接近全量重训，成本低得多
- 相比持续微调，更有效提升模型对安全指令的遵守能力
将通用语言模型适配到新技能通常成本高昂，且新增技能可能引发旧技能遗忘。本文研究通过在新技能上独立训练后，再与通用模型合并（如使用任务向量）来添加新技能的有效性。在科学文献理解、安全性和编码任务上的实验表明，这种‘并行训练-合并’流程显著低于重新训练的成本，且效果相当。尤其在安全性方面，该方法比持续微调和重训更能显著提升模型对安全提示的遵从性，同时保持拒绝有害提示的能力。
原文摘要 · Abstract (English)
Adapting general-purpose language models to new skills is currently an expensive process that must be repeated as new instruction datasets targeting new skills are created, or can cause the models to forget older skills. In this work, we investigate the effectiveness of adding new skills to preexisting models by training on the new skills in isolation and later merging with the general model (e.g. using task vectors). In experiments focusing on scientific literature understanding, safety, and coding, we find that the parallel-train-then-merge procedure, which is significantly cheaper than retraining the models on updated data mixtures, is often comparably effective. Our experiments also show that parallel training is especially well-suited for enabling safety features in LMs relative to continued finetuning and retraining, as it dramatically improves model compliance with safe prompts while preserving its ability to refuse dangerous or harmful prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。