无需数据共享，融合多个领域模型知识，提升性能。
Domain-Adaptive Model Merging Across Disconnected Modes
- 通过伪数据合成与轻量微调，融合差异大的模型。
- 在多模态和单模态任务上均超越现有方法。
- 适合隐私敏感、数据分散场景下的模型整合。
当数据因隐私或异构性无法集中时，跨领域学习极具挑战，限制了单一综合模型的训练。模型合并提供了一种替代方案，可通过整合多个专用模型的知识来避免数据共享并降低重训练成本。本文提出DMM，一种无需数据的模型合并框架，用于处理高度差异化的模型。DMM包含三个步骤：首先独立训练各领域模型；其次使用标准方法合并相似模型以保证稳定性；最后通过归一化统计量生成伪数据，并利用这些样本引导轻量级微调，将差异模型的知识蒸馏到合并模型中。该方法既保留了罕见但关键的知识，又维持了稳定性。在单模态和多模态基准上的大量实验表明，DMM在性能上优于现有合并方法。
原文摘要 · Abstract (English)
Learning across domains is challenging when data cannot be centralized due to privacy or heterogeneity, which limits the ability to train a single comprehensive model. Model merging provides an appealing alternative by consolidating knowledge from multiple specialized models into one, avoiding data sharing and reducing retraining cost. In this work, we present DMM, a data-free model merging framework designed to handle highly divergent models. DMM proceeds in three steps. First, domain-specific models are trained independently. Second, models with high similarity are merged using standard techniques to ensure stability. Third, we synthesize pseudo-data from normalization statistics and distill knowledge from divergent models into the merged model through a lightweight refinement guided by these samples. This approach preserves rare but critical knowledge while maintaining stability. Extensive experiments on unimodal and multimodal benchmarks show that DMM achieves state-of-the-art performance over existing merging methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。