arXiv:2508.05213cs.CV2025-08中稿 · ACMMM2025被引 3

无需源数据,用图文信息实现跨域少样本分割的精准适配。

Textual and Visual Guided Task Adaptation for Source-Free Cross-Domain Few-Shot Segmentation

  • 通过图文对齐机制,在无源数据下适应目标域任务。
  • 1-shot和5-shot下分别提升2.18%和4.11%分割准确率。
  • 适合注重隐私与低资源部署的跨域分割场景。

少样本分割(FSS)旨在用少量标注样例高效分割新物体,但当训练与部署域存在差异时性能显著下降。跨域少样本分割(CD-FSS)被提出以缓解此问题。现有方法主要在源域上训练具备跨域泛化能力的模型,但随着数据隐私担忧加剧及降低数据传输与训练成本的需求,无源域数据的CD-FSS方法变得至关重要。本文提出一种无源域的CD-FSS方法,利用文本与视觉信息实现目标域任务适配。首先在预训练主干的特征金字塔中加入任务特定注意力适配器(TSAA),用于适配多层级特征;然后通过视觉-视觉嵌入对齐(VVEA)与文本-视觉嵌入对齐(TVEA)模块训练其参数。VVEA利用全局-局部视觉特征对齐不同视角图像特征,TVEA则借助预对齐的多模态特征(如CLIP)中的文本先验引导跨模态适配。通过密集比较与跳跃连接融合输出,生成优化的预测掩码。在四个跨域数据集上,1-shot与5-shot设置下平均分割准确率分别提升2.18%和4.11%,显著优于现有先进方法。代码已公开于https://github.com/ljm198134/TVGTANet。

原文摘要 · Abstract (English)

Few-Shot Segmentation(FSS) aims to efficient segmentation of new objects with few labeled samples. However, its performance significantly degrades when domain discrepancies exist between training and deployment. Cross-Domain Few-Shot Segmentation(CD-FSS) is proposed to mitigate such performance degradation. Current CD-FSS methods primarily sought to develop segmentation models on a source domain capable of cross-domain generalization. However, driven by escalating concerns over data privacy and the imperative to minimize data transfer and training expenses, the development of source-free CD-FSS approaches has become essential. In this work, we propose a source-free CD-FSS method that leverages both textual and visual information to facilitate target domain task adaptation without requiring source domain data. Specifically, we first append Task-Specific Attention Adapters (TSAA) to the feature pyramid of a pretrained backbone, which adapt multi-level features extracted from the shared pre-trained backbone to the target task. Then, the parameters of the TSAA are trained through a Visual-Visual Embedding Alignment (VVEA) module and a Text-Visual Embedding Alignment (TVEA) module. The VVEA module utilizes global-local visual features to align image features across different views, while the TVEA module leverages textual priors from pre-aligned multi-modal features (e.g., from CLIP) to guide cross-modal adaptation. By combining the outputs of these modules through dense comparison operations and subsequent fusion via skip connections, our method produces refined prediction masks. Under both 1-shot and 5-shot settings, the proposed approach achieves average segmentation accuracy improvements of 2.18\% and 4.11\%, respectively, across four cross-domain datasets, significantly outperforming state-of-the-art CD-FSS methods. Code are available at https://github.com/ljm198134/TVGTANet.

少样本分割跨域迁移图文对齐无源学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。