arXiv:2602.16189cs.CLcs.AI2026-02被引 1

不训练也能提升模型性能,通过移植内部模块实现即时功能增强。

Beyond Learning: A Training-Free Alternative to Model Adaptation

  • 从模型内部识别出任务专用的局部激活模块并移植到目标模型。
  • 跨生成场景下性能最高提升至基线两倍,差距恢复超100%。
  • 适合需要快速优化模型且无资源训练的场景,如部署调试。

尽管语言模型持续演进,但有时表现反而不如旧版本。现有方法成本高昂,亟需无需训练的替代方案。本文假设每个语言模型内含针对特定任务的局部模块。通过激活分析,识别出在推理负载下表现出一致且局部激活变化的一组模块。随后将适配特定任务的内部模块移植至目标模型,实现无需额外训练或微调即可立即产生可测量的功能改进。为验证移植技术有效性,我们量化了不同条件下移植强度与性能提升的关系。在跨生成设置中,移植激活选择的模块可显著提升表现不佳的模型,最高达基线的两倍,差距恢复超过100%。在基础模型与其指令微调版本间的移植实验中,表现较弱模型性能提升至基线约2.33倍,最大差距恢复达100%。结果表明,语言模型中可通过植入高度局部化的模块实现有效能力迁移。本研究为语言模型的任务局部模块化提供了实证支持,并开创了‘模型移植’这一新方向。

原文摘要 · Abstract (English)

Despite the continuous research and evolution of language models, they sometimes underperform previous versions. Existing approaches to overcome these challenges are resource-intensive, highlighting the need for alternatives that enable immediate action. We assume that each language model has a local module inside that is suitable for a specific function. First, this work identifies a set of modules showing consistent and local activation changes under an inference workload through activation-based analysis. Subsequently, we transplant an internal module that is properly activated for a specific task into the target model, leading to immediate and measurable functional changes without additional training or fine-tuning. To experimentally demonstrate the effectiveness of the transplant technique, we quantify the relationship between transplant strength and performance improvement under different conditions for two language models. In the cross-generation setting, we find that transplanting activation-selected modules can substantially improve the underperforming model, reaching up to twice the target baseline and achieving gap-based recovery above 100%. Moreover, in transplant experiments between a base model and its instruction-tuned counterpart, transplantation improves the underperforming model toward the stronger baseline, yielding up to about 2.33 times the target baseline with gap-based recovery reaching up to 100% in the best case. These results show that meaningful capacity transfer can be realized through the implantation of highly localized modules implied by language models. Overall, this work provides empirical evidence for task-localized modularity in language models and presents a new research area: model transplantation.

模型移植零训练模块化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。