不训练就能提升大模型推理能力,通过智能组合现有模型权重。
Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning

- 用基因式重组方法在权重空间无梯度融合多个模型。
- 270亿参数模型在GPQA钻石题上达86.9%,超越原模型。
- 支持跨架构融合,适合需要快速迭代推理模型的研究者。
我们提出Darwin Family,一种无需训练的大型语言模型进化融合框架,通过无梯度的权重空间重组实现模型能力提升。核心思想包括:(i) 14维自适应融合基因,支持组件与模块级精细重组;(ii) MRI-Trust融合机制,通过可学习的信任参数动态平衡诊断层重要性信号与进化搜索;(iii) 架构映射器,实现异构模型家族间的跨架构杂交。实验证明,旗舰模型Darwin-27B-Opus在GPQA Diamond任务中达到86.9%准确率,排名1,252个模型中的第6位,优于其全量训练的基础模型且未进行任何梯度训练。从40亿到350亿参数规模,达尔文系列模型持续超越父模型,支持多代递归进化,并实现Transformer与Mamba结构组件的训练自由融合。结果表明,诊断引导的进化融合是推理导向语言模型替代昂贵后训练流程的一种可行且可复现的方案。
原文摘要 · Abstract (English)
We present Darwin Family, a framework for training-free evolutionary merging of large language models via gradient-free weight-space recombination. We ask whether frontier-level reasoning performance can be improved without additional training, by reorganizing latent capabilities already encoded in existing checkpoints. Darwin introduces three key ideas: (i) a 14-dimensional adaptive merge genome enabling fine-grained component- and block-level recombination; (ii) MRI-Trust Fusion, which adaptively balances diagnostic layer-importance signals with evolutionary search through a learnable trust parameter; and (iii) an Architecture Mapper that enables cross-architecture breeding between heterogeneous model families. Empirically, the flagship Darwin-27B-Opus achieves 86.9% on GPQA Diamond, ranking #6 among 1,252 evaluated models, and outperforming its fully trained foundation model without any gradient-based training. Across scales from 4B to 35B parameters, Darwin models consistently improve over their parents, support recursive multi-generation evolution, and enable a training-free evolutionary merge that combines Transformer- and Mamba-based components. Together, the Darwin Family demonstrates that diagnostic-guided evolutionary merging is a practical and reproducible alternative to costly post-training pipelines for reasoning-centric language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。