arXiv:2412.12798cs.CV2024-12被引 16

让模型识别训练中没见过的航拍物体,突破遥感实例分割的零样本瓶颈。

ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentation

  • 用优化文本嵌入增强类别差异感知,提升分类辨识力。
  • 不直接微调,通过解耦语义信息保留预训练对齐,适应遥感视觉特征。
  • 引入航空图像原型缓存,补足文本语义,适配遥感领域需求。

遥感领域的实例分割算法通常依赖传统方法,仅适用于已见场景和封闭集预测。本文提出零样本遥感实例分割新任务,旨在识别训练数据中不存在的空中目标。面对类别间相似度高、类内差异大的挑战,以及视觉-语言模型预训练数据与遥感数据之间的域差距,本文提出名为ZoRI的框架。该框架采用增强判别力的分类器,利用优化的文本嵌入提升类别差异感知;提出知识保持式适配策略,解耦语义信息,在保留预训练视觉-语言对齐的同时调整特征以捕捉遥感特有的视觉线索;并引入先验注入预测机制,结合航空图像原型缓存,丰富文本嵌入语义,无缝融合遥感表征。我们建立了新的实验协议与基准,大量实验证明ZoRI在零样本遥感实例分割任务上达到当前最优性能。代码已开源:https://github.com/HuangShiqi128/ZoRI。

原文摘要 · Abstract (English)

Instance segmentation algorithms in remote sensing are typically based on conventional methods, limiting their application to seen scenarios and closed-set predictions. In this work, we propose a novel task called zero-shot remote sensing instance segmentation, aimed at identifying aerial objects that are absent from training data. Challenges arise when classifying aerial categories with high inter-class similarity and intra-class variance. Besides, the domain gap between vision-language models' pretraining datasets and remote sensing datasets hinders the zero-shot capabilities of the pretrained model when it is directly applied to remote sensing images. To address these challenges, we propose a $\textbf{Z}$ero-Sh$\textbf{o}$t $\textbf{R}$emote Sensing $\textbf{I}$nstance Segmentation framework, dubbed $\textbf{ZoRI}$. Our approach features a discrimination-enhanced classifier that uses refined textual embeddings to increase the awareness of class disparities. Instead of direct fine-tuning, we propose a knowledge-maintained adaptation strategy that decouples semantic-related information to preserve the pretrained vision-language alignment while adjusting features to capture remote sensing domain-specific visual cues. Additionally, we introduce a prior-injected prediction with cache bank of aerial visual prototypes to supplement the semantic richness of text embeddings and seamlessly integrate aerial representations, adapting to the remote sensing domain. We establish new experimental protocols and benchmarks, and extensive experiments convincingly demonstrate that ZoRI achieves the state-of-art performance on the zero-shot remote sensing instance segmentation task. Our code is available at https://github.com/HuangShiqi128/ZoRI.

遥感零样本实例分割视觉语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。