arXiv:2506.08772cs.CV2025-06被引 3

用多个视觉大模型指导遥感图像分割,提升小样本下的精度和泛化能力。

RS-MTDF: Multi-Teacher Distillation and Fusion for Remote Sensing Semi-Supervised Semantic Segmentation

  • 引入多个冻结的视觉大模型作为教师,通过特征级知识蒸馏引导学生网络
  • 在爱达数据集上不同标签比例下均达到领先性能,多数类别交并比最高
  • 适合遥感领域标注数据少、需强泛化能力的语义分割任务

遥感图像语义分割对多种应用至关重要,但其性能严重依赖大规模高质量像素级标注,而这类标注成本高、耗时长。半监督语义分割(SSS)可缓解数据依赖问题。然而现有方法常因少量有标签数据与大量无标签数据间分布不一致,导致泛化能力不足。为此,本文将预训练于海量多样化数据的视觉基础模型(VFMs)引入SSS任务,因其具备强泛化能力,可有效弥合分布差距并提供有力语义先验。受此启发,提出RS-MTDF(多教师蒸馏与融合)框架,利用多个冻结的VFMs(如DINOv2和CLIP)作为专家教师,通过特征级蒸馏使学生特征与之对齐,并将蒸馏知识无缝融合至学生解码器以增强判别力。在三个挑战性遥感数据集上的大量实验表明,RS-MTDF始终达到最先进性能。尤其在LoveDA数据集上,不同标签比例下均表现优异,多数语义类别取得最高交并比(IoU)。消融实验证明了各模块的有效性。

原文摘要 · Abstract (English)

Semantic segmentation in remote sensing images is crucial for various applications, yet its performance is heavily reliant on large-scale, high-quality pixel-wise annotations, which are notoriously expensive and time-consuming to acquire. Semi-supervised semantic segmentation (SSS) offers a promising alternative to mitigate this data dependency. However, existing SSS methods often struggle with the inherent distribution mismatch between limited labeled data and abundant unlabeled data, leading to suboptimal generalization. To alleviate this issue, we attempt to introduce the Vision Foundation Models (VFMs) pre-trained on vast and diverse datasets into the SSS task since VFMs possess robust generalization capabilities that can effectively bridge this distribution gap and provide strong semantic priors for SSS. Inspired by this, we introduce RS-MTDF (Multi-Teacher Distillation and Fusion), a novel framework that leverages the powerful semantic knowledge embedded in VFMs to guide semi-supervised learning in remote sensing. Specifically, RS-MTDF employs multiple frozen VFMs (e.g., DINOv2 and CLIP) as expert teachers, utilizing feature-level distillation to align student features with their robust representations. To further enhance discriminative power, the distilled knowledge is seamlessly fused into the student decoder. Extensive experiments on three challenging remote sensing datasets demonstrate that RS-MTDF consistently achieves state-of-the-art performance. Notably, our method outperforms existing approaches across various label ratios on LoveDA and secures the highest IoU in the majority of semantic categories. These results underscore the efficacy of multi-teacher VFM guidance in significantly enhancing both generalization and semantic understanding for remote sensing segmentation. Ablation studies further validate the contribution of each proposed module.

遥感分割半监督学习知识蒸馏视觉大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。