用进化算法合并模型，零训练成本提升性能。
Mergenetic: a Simple Evolutionary Model Merging Library
- 将进化算法与模型合并结合，灵活配置融合策略。
- 在普通显卡上实现跨任务跨语言的高效性能提升。
- 支持快速实验，适合想优化模型的开发者使用。
模型合并可在不进行额外训练的情况下，将多个现有模型的能力整合为一个新模型，因其低成本和对消费级GPU的支持而日益流行。近期研究发现，将合并与进化算法结合可进一步提升性能，但目前尚无框架支持语言模型中灵活的此类策略实验。我们提出 Mergenetic，一个开源的进化式模型合并库，支持轻松组合多种合并方法与进化算法，并集成轻量级适应度评估器以降低评估开销。我们阐述其设计并验证，在仅使用适度硬件条件下，Mergenetic 能在多任务和多语言场景下生成具有竞争力的结果。
原文摘要 · Abstract (English)
Model merging allows combining the capabilities of existing models into a new one - post hoc, without additional training. This has made it increasingly popular thanks to its low cost and the availability of libraries that support merging on consumer GPUs. Recent work shows that pairing merging with evolutionary algorithms can boost performance, but no framework currently supports flexible experimentation with such strategies in language models. We introduce Mergenetic, an open-source library for evolutionary model merging. Mergenetic enables easy composition of merging methods and evolutionary algorithms while incorporating lightweight fitness estimators to reduce evaluation costs. We describe its design and demonstrate that Mergenetic produces competitive results across tasks and languages using modest hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。