对比数据混合与模型合并，发现后者在多语言多任务中更优。
Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
- 用目标函数驱动的模型合并比数据混合更有效
- 安全性能提升10%，通用能力提高8%
- 按语言单独微调后合并，效果优于统一训练
大型语言模型已广泛应用于各类场景，但保障其安全使用仍是重大挑战。现有偏好训练和安全措施多基于西方中心数据集，难以适应多语言环境。本文在多语言多任务背景下探索模型合并方法，将安全与通用任务结合。实验表明，基于目标的模型合并优于数据混合，在通用性能上提升最高达8%，安全性能提升最高达10%。按语言分别微调后合并，相比相同数据下的数据混合方法，通用性能提升4%，跨语言危害减少7%。研究为构建强健且安全的多语言模型提供了有效框架。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have been adopted and deployed worldwide for a broad variety of applications. However, ensuring their safe use remains a significant challenge. Preference training and safety measures often overfit to harms prevalent in Western-centric datasets, and safety protocols frequently fail to extend to multilingual settings. In this work, we explore model merging in a diverse multi-task setting, combining safety and general-purpose tasks within a multilingual context. Each language introduces unique and varied learning challenges across tasks. We find that objective-based merging is more effective than mixing data, with improvements of up to 8% and 10% in general performance and safety respectively. We also find that language-based merging is highly effective -- by merging monolingually fine-tuned models, we achieve a 4% increase in general performance and 7% reduction in harm across all languages on top of the data mixtures method using the same available data. Overall, our comprehensive study of merging approaches provides a useful framework for building strong and safe multilingual models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。