用任务向量消除模型融合中的干扰，无需额外数据即可提升性能。
Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors
- 基于线性层任务向量近似共线的特性，设计无干扰融合方法。
- 在多任务视觉与语言任务上平均提升10.9%，超越主流测试时适配方法。
- 仅需极少计算资源，适合资源受限场景下的模型集成应用。
模型融合旨在将特定任务的专家模型整合到统一架构中，同时保持多任务泛化能力，但各模型间的参数干扰常导致性能下降。尽管已有多种融合策略，但在不依赖额外训练数据或测试时计算的前提下解决干扰问题仍具挑战。本文理论证明，线性层的任务向量在其对应输入下构成近似线性子空间，据此提出WUDI-Merging方法，通过任务向量引导实现无干扰融合，无需额外数据或缩放系数。在视觉与语言基准上的全面实验表明，该方法在数据无感模型融合场景中表现卓越，平均较基线提升10.9%，甚至优于主流测试时适应方法3.3%，且仅需极少量计算资源。代码即将公开。
原文摘要 · Abstract (English)
Model merging seeks to integrate task-specific expert models into a unified architecture while preserving multi-task generalization capabilities, yet parameter interference between constituent models frequently induces performance degradation. Although prior work has explored many merging strategies, resolving interference without additional data for retraining or test-time computation remains challenging. In this paper, we theoretically demonstrate that the task vectors of the linear layer constitute an approximate linear subspace for its corresponding input. Therefore, we can minimize interference under the guidance of task vectors. Based on this insight, we propose \textbf{WUDI-Merging} (\textbf{W}hoever started the interference sho\textbf{U}ld en\textbf{D} \textbf{I}t), a simple yet effective model merging method that eliminates interference without any additional data or rescaling coefficients. Comprehensive empirical evaluations across vision and language benchmarks demonstrate our method's superiority, achieving state-of-the-art performance in data-free model merging scenarios (average 10.9\% improvement versus baseline methods) while even outperforming mainstream test-time adaptation approaches by 3.3\%, and only very few computing resources are required. The code will be publicly available soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。