arXiv:2508.16204cs.AIcs.NE2025-08中稿 · GECCO 2025 as a fu…被引 3

用自然演化思想自动融合模型,无需手动分组。

Competition and Attraction Improve Model Fusion

  • 基于生态竞争与吸引机制,动态调整参数合并边界。
  • 从零训练MNIST分类器,性能媲美CMA-ES且更高效。
  • 可扩展融合多模态模型,保留未优化的关键能力。

模型融合能将多个机器学习模型的专业知识整合到单一模型中。但现有方法需人工将参数划分为固定组别进行融合,限制了组合探索并影响性能。为此,我们提出自然生态模型融合(M2N2),一种具备三项核心特性的进化算法:(1) 动态调整融合边界,逐步探索更广的参数组合;(2) 受自然界资源竞争启发的多样性保持机制,维持一组多样且高性能的模型以利于融合;(3) 基于启发式的吸引度量,识别最适配融合的模型对。实验首次证明模型融合可完全从零开始演化模型。我们将M2N2应用于从零训练MNIST分类器,性能接近CMA-ES,同时计算效率更高。此外,该方法可扩展用于融合专用语言与图像生成模型,达到当前最优性能。值得注意的是,它还能保留未被适应度函数显式优化的关键模型能力,体现其鲁棒性与通用性。代码已开源:https://github.com/SakanaAI/natural_niches

原文摘要 · Abstract (English)

Model merging is a powerful technique for integrating the specialized knowledge of multiple machine learning models into a single model. However, existing methods require manually partitioning model parameters into fixed groups for merging, which restricts the exploration of potential combinations and limits performance. To overcome these limitations, we propose Model Merging of Natural Niches (M2N2), an evolutionary algorithm with three key features: (1) dynamic adjustment of merging boundaries to progressively explore a broader range of parameter combinations; (2) a diversity preservation mechanism inspired by the competition for resources in nature, to maintain a population of diverse, high-performing models that are particularly well-suited for merging; and (3) a heuristicbased attraction metric to identify the most promising pairs of models for fusion. Our experimental results demonstrate, for the first time, that model merging can be used to evolve models entirely from scratch. Specifically, we apply M2N2 to evolve MNIST classifiers from scratch and achieve performance comparable to CMA-ES, while being computationally more efficient. Furthermore, M2N2 scales to merge specialized language and image generation models, achieving state-of-the-art performance. Notably, it preserves crucial model capabilities beyond those explicitly optimized by the fitness function, highlighting its robustness and versatility. Our code is available at https://github.com/SakanaAI/natural_niches

模型融合进化算法多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。