通过单层适配实现模型合并中的任务协同，提升多任务性能
SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation
- 仅调整一个任务专属层，激发跨任务协同效应
- 在视觉、密集预测和NLP任务上达到当前最佳效果
- 适用于不同初始化的模型合并，传统方法在此失效
模型合并将独立训练的模型整合为单一多任务模型。现有方法多关注避免任务干扰，而我们主张其更大潜力在于实现任务协同，即任务间相互促进。我们提出以跨任务性能（由不同任务的编码器与预测器兼容性决定）作为合并质量的关键指标，并证明仅需调整单个任务特定层即可诱导协同。本文提出轻量级框架SyMerge，联合优化合并系数与单个任务特定层。采用专家引导的自标签目标，提供超越熵最小化的稳定监督。有趣的是，我们进一步证明SyMerge可成功合并不同初始化训练的模型，而标准方法在此情形下失效。该极简但原理清晰的方法在视觉、密集预测和NLP基准上均取得当前最优表现。代码已开源。
原文摘要 · Abstract (English)
Model merging combines independently trained models into a single multi-task model. However, most existing approaches focus primarily on avoiding task interference. We argue that its greater potential lies in enabling task synergy, where tasks actively improve one another. We identify cross-task performance, defined by compatibility between encoders and predictors across tasks, as a key indicator of merge quality. We demonstrate that adapting only a single task-specific layer is sufficient to induce such synergy. This study proposes SyMerge, a lightweight framework that jointly optimizes merging coefficients and a single task-specific layer. We adopt an expert-guided self-labeling objective, providing stable supervision beyond entropy minimization. Intriguingly, we further show that SyMerge successfully merges models trained from different initializations, a regime where standard methods break down. Our minimalist yet principled method achieves state-of-the-art results across vision, dense prediction, and NLP benchmarks. Our code is available at https://aim-skku.github.io/SyMerge
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。