模型合并时共享知识会重复累加，导致性能下降。
When Shared Knowledge Hurts: Spectral Over-Accumulation in Model Merging
- 通过量化子空间重叠并校正膨胀的奇异值来修复合并偏差
- 在视觉与语言任务上显著提升现有合并方法性能
- 无需训练、不需数据，适合快速优化多模型合并
模型合并通过加总权重更新将多个微调模型整合为单一模型，是一种轻量级替代重训练的方法。现有方法主要解决任务更新间的冲突问题，但未关注共享知识过度累积的失败模式。我们发现，当任务共享对齐的谱方向（即重叠的奇异向量）时，简单线性组合会反复累积这些方向，放大奇异值，使合并模型偏向共享子空间。为此，我们提出奇异值校准（SVC），一种无需训练、不需数据的后处理方法，可量化子空间重叠并重标定膨胀的奇异值，恢复平衡的谱分布。在视觉与语言基准测试中，SVC持续提升强基线合并方法性能，并达到当前最优表现。此外，仅修改奇异值即可使任务算术性能提升13.0%。代码已公开于https://github.com/lyymuwu/SVC。
原文摘要 · Abstract (English)
Model merging combines multiple fine-tuned models into a single model by adding their weight updates, providing a lightweight alternative to retraining. Existing methods primarily target resolving conflicts between task updates, leaving the failure mode of over-counting shared knowledge unaddressed. We show that when tasks share aligned spectral directions (i.e., overlapping singular vectors), a simple linear combination repeatedly accumulates these directions, inflating the singular values and biasing the merged model toward shared subspaces. To mitigate this issue, we propose Singular Value Calibration (SVC), a training-free and data-free post-processing method that quantifies subspace overlap and rescales inflated singular values to restore a balanced spectrum. Across vision and language benchmarks, SVC consistently improves strong merging baselines and achieves state-of-the-art performance. Furthermore, by modifying only the singular values, SVC improves the performance of Task Arithmetic by 13.0%. Code is available at https://github.com/lyymuwu/SVC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。