arXiv:2604.06377cs.LGcs.AI2026-04

不重训练即可跨模型迁移推理能力,靠对齐低维潜在方向实现。

The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

论文配图:The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment
图 1 · 摘自论文原文
  • 通过对比源模型有无特定能力时的激活差异,提取能力方向。
  • 用低秩线性变换对齐目标模型,推理时直接注入方向提升性能。
  • 在不同规模模型间转移思维链和数学推理能力,效果显著优于原模型。

我们研究了后训练能力是否可在不重新训练的情况下跨模型迁移,尤其关注不同规模模型间的迁移。提出‘主键假说’:模型能力对应于低维潜在子空间中的方向,能通过线性对齐跨模型传递。基于此,我们提出UNLOCK框架——无需训练、无需标签,通过对比具备与不具备特定能力的源模型激活,提取能力方向,经低秩线性变换对齐目标模型,在推理时注入以激发行为。实验表明,对推理能力如思维链(CoT)和数学推理的迁移带来显著提升。例如,将Qwen1.5-14B的CoT能力迁移到Qwen1.5-7B,在MATH数据集上准确率提升12.1%;将Qwen3-4B-Base的数学推理方向迁移到Qwen3-14B-Base,AGIEval Math准确率从61.1%提升至71.3%,超过该规模后训练模型的67.8%表现。分析显示,迁移成功取决于预训练阶段学习到的能力,干预通过聚焦输出分布至成功推理路径来增强隐含能力。

原文摘要 · Abstract (English)

We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories.

能力迁移推理增强零样本迁移子空间对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。