无需数据即可合并多个模型，提升性能且不依赖调参。
Weight Weaving: Parameter Pooling for Data-Free Model Merging
- 通过加权池化不同参数组合，自动融合多个模型。
- 在无数据环境下，最高提升15.9个百分点准确率。
- 适配多种模型合并方法，适合资源受限场景使用。
模型合并通过参数整合实现专业化深度神经网络的低成本、高效组合，无需重新训练即可利用下游任务中的专家模型。现有方法严重依赖全局或个体化的缩放超参数 $λ$，而缺乏在无数据情况下合理设定 $λ$ 的有效方法，通常需借助评估集的特权数据进行调优，这在实践中不可行。为此，我们提出 Weight Weaving，一种即插即用技术，通过用户定义的池化函数（如平均、随机选择或现有合并方法）对 $λ$ 搜索空间内的模型权重进行聚合。该方法高度模块化，对搜索空间约束极少，与现有合并方法正交，且完全无需评估数据。我们在三种 ViT 变体上验证了该方法，涵盖视觉多任务学习、持续学习和域泛化三种设置。结果表明，Weight Weaving 持续提升多种合并方法的性能，在无数据条件下平均准确率提升高达15.9个百分点。
原文摘要 · Abstract (English)
Model merging provides a cost-effective and data-efficient combination of specialized deep neural networks through parameter integration. This technique leverages expert models across downstream tasks without requiring retraining. Most model merging approaches critically depend on scaling hyper-parameters $λ$, which weight each model's contribution globally or individually. Principled approaches for setting scaling factors without accessing any data (data-free) are scarce, often leading researchers to tune $λ$ using privileged data from the evaluation set, which is obviously unfeasible in practice. To address this limitation, we introduce Weight Weaving, a plug-and-play technique that pools model weights across $λ$ values search space using user-defined pooling functions, such as averaging, random selection, or even existing model merging methods. Our method demonstrates high modularity, imposing minimal constraints on the search space. It operates orthogonally to existing model merging methods and eliminates evaluation data requirements. We validate Weight Weaving across three ViT variants in three experimental setups: vision multi-task learning, vision continual learning, and domain generalization. Our method consistently improves the performance of several model merging methods, achieving average accuracy gains of up to 15.9 percentage points in a data-free setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。