arXiv:2409.15813cs.CVcs.AI2024-09被引 4

通过分层合并模型,低成本提升分割任务的无监督域适应性能。

Layer-wise Model Merging for Unsupervised Domain Adaptation in Segmentation Tasks

  • 分层合并:统一前层特征提取,保留后层任务特异性。
  • 同架构模型融合提升2.6% mIoU,异构模型融合提升6.8% mIoU。
  • 适用于多种分割任务与域适应策略,无需额外训练成本。

参数合并近年来被重新视为提升任务性能与鲁棒性的有效方法,但现有工作受限于集成模型创建和推理的高成本。本文利用大量免费可获取的预训练模型,提出一种零成本的模型合并方法。该方法采用分层整合策略,保留任务特异性的最终层,统一主要负责特征提取的初始层,确保各层参数一致性,从而提升性能。同时,该方法促进知识无缝融合,可有效合并来自不同数据集和任务的模型。我们首次将该方法应用于无监督域适应(UDA)中的语义分割与全景分割任务。实验表明,对于同架构模型,跨数据集合并可带来2.6%的mIoU提升;对于共享主干网络的异构模型,提升达6.8%;合并语义与全景分割模型使mPQ提升7%。这些成果在多种UDA策略、模型架构和数据集上均得到验证。

原文摘要 · Abstract (English)

Merging parameters of multiple models has resurfaced as an effective strategy to enhance task performance and robustness, but prior work is limited by the high costs of ensemble creation and inference. In this paper, we leverage the abundance of freely accessible trained models to introduce a cost-free approach to model merging. It focuses on a layer-wise integration of merged models, aiming to maintain the distinctiveness of the task-specific final layers while unifying the initial layers, which are primarily associated with feature extraction. This approach ensures parameter consistency across all layers, essential for boosting performance. Moreover, it facilitates seamless integration of knowledge, enabling effective merging of models from different datasets and tasks. Specifically, we investigate its applicability in Unsupervised Domain Adaptation (UDA), an unexplored area for model merging, for Semantic and Panoptic Segmentation. Experimental results demonstrate substantial UDA improvements without additional costs for merging same-architecture models from distinct datasets ($\uparrow 2.6\%$ mIoU) and different-architecture models with a shared backbone ($\uparrow 6.8\%$ mIoU). Furthermore, merging Semantic and Panoptic Segmentation models increases mPQ by $\uparrow 7\%$. These findings are validated across a wide variety of UDA strategies, architectures, and datasets.

模型合并无监督学习分割任务域适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。