梳理模型合并方法,帮研究者快速掌握统一建模技术脉络。
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches
- 按合并机制分类,建立统一方法框架
- 解决多模型融合中术语混乱与对比困难问题
- 适合刚入行的研究者和跨领域融合需求者
模型合并已取得显著进展,诸多创新方法通过整合多个模型提升能力。然而,由于缺乏统一的分类框架与系统性比较分析,术语和分类标准不一,导致研究不一致。同时,随着大量微调模型公开,其原始训练数据常因隐私或知识产权限制无法获取,使得基于共享训练数据的传统多任务学习变得不可行。在无法直接访问训练数据的情况下,通过参数合并构建具备跨领域泛化能力的统一模型变得至关重要,凸显了模型合并技术的重要性。尽管该领域发展迅速,但系统性分类、近期进展综述及未来方向预测仍显不足。本文通过建立新的模型合并方法分类体系,系统比较不同方法,并概述关键进展,为该快速发展领域的初学者提供清晰视角,助力理解与进一步创新。
原文摘要 · Abstract (English)
Model merging has achieved significant success, with numerous innovative methods proposed to enhance capabilities by combining multiple models. However, challenges persist due to the lack of a unified framework for classification and systematic comparative analysis, leading to inconsistencies in terminologies and categorizations. Meanwhile, as an increasing number of fine-tuned models are publicly available, their original training data often remain inaccessible due to privacy concerns or intellectual property restrictions. This makes traditional multi-task learning based on shared training data impractical. In scenarios where direct access to training data is infeasible, merging model parameters to create a unified model with broad generalization across multiple domains becomes crucial, further underscoring the importance of model merging techniques. Despite the rapid progress in this field, a comprehensive taxonomy and survey summarizing recent advances and predicting future directions are still lacking. This paper addresses these gaps by establishing a new taxonomy of model merging methods, systematically comparing different approaches, and providing an overview of key developments. By offering a structured perspective on this evolving area, we aim to help newcomers quickly grasp the field's landscape and inspire further innovations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。