arXiv:2511.15515cs.CV2025-11

用多段文本增强少样本分割的语义精度,提升复杂物体识别能力。

Multi-Text Guided Few-Shot Semantic Segmentation

  • 通过多文本提示融合,优化文本先验并引导视觉特征对齐。
  • 1次采样下在PASCAL-5i达76.8% mIoU,COCO-20i达57.4% mIoU。
  • 适合处理类内差异大的复杂类别分割任务。

基于CLIP的少样本语义分割方法通常使用单一文本提示(如类别图片)来辅助分割,但单个提示难以覆盖复杂类别完整的语义多样性,导致目标区域激活不全。同时,缺乏显式的跨模态交互且易受噪声支持特征干扰,进一步降低视觉先验质量。为此,本文提出多文本引导少样本语义分割网络MTGNet,一个双分支框架,通过融合多样化文本提示来优化文本先验,并指导视觉先验的跨模态优化。具体设计了多文本先验精炼(MTPR)模块,抑制干扰并聚合互补语义线索,增强前景激活并扩展结构复杂物体的语义覆盖。引入文本锚点特征融合(TAFF)模块,利用多文本嵌入作为语义锚点,促进支持图像中判别性局部原型向查询图像迁移,提升语义一致性并缓解类内差异。此外,提出前景置信度加权注意力(FCWA)模块,通过利用支持前景特征内部自相似性,自适应降低不一致区域权重,有效抑制查询分割过程中的干扰。在标准少样本分割基准上的大量实验验证了MTGNet的有效性。1次采样设置下,在PASCAL-5i上达到76.8% mIoU,COCO-20i上达57.4%,在类内变化较大的类别上表现尤为突出。

原文摘要 · Abstract (English)

Recent CLIP-based few-shot semantic segmentation methods introduce class-level textual priors to assist segmentation by typically using a single prompt (e.g., a photo of class). However, these approaches often result in incomplete activation of target regions, as a single textual description cannot fully capture the semantic diversity of complex categories. Moreover, they lack explicit cross-modal interaction and are vulnerable to noisy support features, further degrading visual prior quality. To address these issues, we propose the Multi-Text Guided Few-Shot Semantic Segmentation Network (MTGNet), a dual-branch framework that enhances segmentation performance by fusing diverse textual prompts to refine textual priors and guide the cross-modal optimization of visual priors. Specifically, we design a Multi-Textual Prior Refinement (MTPR) module that suppresses interference and aggregates complementary semantic cues to enhance foreground activation and expand semantic coverage for structurally complex objects. We introduce a Text Anchor Feature Fusion (TAFF) module, which leverages multi-text embeddings as semantic anchors to facilitate the transfer of discriminative local prototypes from support images to query images, thereby improving semantic consistency and alleviating intra-class variations. Furthermore, a Foreground Confidence-Weighted Attention (FCWA) module is presented to enhance visual prior robustness by leveraging internal self-similarity within support foreground features. It adaptively down-weights inconsistent regions and effectively suppresses interference in the query segmentation process. Extensive experiments on standard FSS benchmarks validate the effectiveness of MTGNet. In the 1-shot setting, it achieves 76.8% mIoU on PASCAL-5i and 57.4% on COCO-20i, with notable improvements in folds exhibiting high intra-class variations.

少样本分割文本引导跨模态融合语义分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。