直接插值权重可实现高效推理，效果优于复杂融合方法。
Revisiting Model Interpolation for Efficient Reasoning
- 直接对模型权重进行线性插值，简单却有效。
- 插值后模型在效率与性能上均超越现有复杂融合方法。
- 揭示了推理过程中的三阶段演化规律，适合精准控制推理能力。
模型合并，尤其在指令和思维类模型上，已展现出高效的推理能力。本文系统重访最简单的合并方法——直接对两个模型权重进行插值。我们发现，模型插值遵循一种三阶段演化范式，在推理轨迹上表现出显著不同的行为特征，为性能与成本的权衡提供了理论指导。实验证明，经过策略性插值的模型在效率和有效性上均优于复杂的模型合并基线。通过在模型层、模块和解码策略上的大量消融实验，进一步验证了结论。本工作揭示了模型插值的本质，提供了一个可精确设计推理能力的实用框架。代码已开源至Github。
原文摘要 · Abstract (English)
Model merging, typically on Instruct and Thinking models, has shown remarkable performance for efficient reasoning. In this paper, we systematically revisit the simplest merging method that interpolates two weights directly. Particularly, we observe that model interpolation follows a three-stage evolutionary paradigm with distinct behaviors on the reasoning trajectory. These dynamics provide a principled guide for navigating the performance-cost trade-off. Empirical results demonstrate that a strategically interpolated model surprisingly surpasses sophisticated model merging baselines on both efficiency and effectiveness. We further validate our findings with extensive ablation studies on model layers, modules, and decoding strategies. Ultimately, this work demystifies model interpolation and offers a practical framework for crafting models with precisely targeted reasoning capabilities. Code is available at \href{https://github.com/wutaiqiang/MI}{Github}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。