提出可调融合方法MeRoTune,解决模型合并时位置编码不兼容问题。
MeRoTune: RoPE-Safe Merging with a Tunable Dial

- 设计与旋转位置编码兼容的校正矩阵,确保注意力对齐
- 支持事后调节融合比例,像旋钮一样灵活控制结果
- 在保持基础权重冻结的前提下实现高效可调合并
通过简单平均权重合并两个基于同一基线检查点微调的模型时,隐含假设其注意力子空间仍对齐。近期工作通过为每个模型学习一个可逆校正矩阵 $M$ 来修复错位,在查询侧使用 $M$、键侧使用 $M^{-T}$,使校正项在点积前相互抵消。然而,这种抵消仅在投影与点积之间无其他操作时精确成立。现实中,几乎所有现代开源语言模型都在此位置插入了旋转位置编码(RoPE)。本文证明:该抵消仅当 $M$ 与 RoPE 的逐位置旋转可交换时才精确成立。我们推导出满足条件的矩阵类:在每个 RoPE 频率对内独立缩放的旋转矩阵。这构成了当前方法通常训练的无约束矩阵的严格低维子集。基于此,我们提出新合并方法 MeRoTune:保持基础权重完全冻结,两个微调模型各自学习其符合 RoPE 的校正矩阵,并针对选定混合比例进行优化,使最终结果可事后调节如旋钮一般,而非固定于单一合并结果。默认方法在固定混合比例下训练,类似 LoRA 提前设定缩放超参数;我们也实验了每步随机重采样混合比例的方法,并报告两种方案的结果。
原文摘要 · Abstract (English)
When you merge two fine-tuned models from the same base checkpoint by simply averaging their weights, you implicitly assume their attention subspaces are still aligned. Recent work attempts to fix misalignments by learning an invertible correction matrix, $M$, for each model's query and key projections. This correction cancels out---using $M$ on the query side and $M^{-T}$ on the key side---right before the dot product. However, this cancellation is only exact if nothing sits between the projection and the dot product. In reality, almost all modern open-weight language models put a rotary position embedding (RoPE) exactly there. In this paper, we show that this cancellation is exact under RoPE if and only if $M$ commutes with RoPE's per-position rotation. We derive the specific class of matrices where this holds: a scaled rotation acting independently within each RoPE frequency pair. This forms a strict, low-dimensional subset of the unconstrained matrices that current methods normally train. Building on this, we turn this constrained matrix class into a new merging method. While keeping the base weights entirely frozen, two fine-tunes each learn their own RoPE-compliant correction matrices. We optimize these corrections against a chosen blend ratio so the final result can be adjusted post-hoc like a dial, rather than locked into a single fixed merge. Our default approach trains at one fixed blend ratio, similar to how LoRA sets its scaling hyperparameter in advance. We also experiment with resampling the blend ratio randomly at every training step, and we report the results of both approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。