arXiv:2504.16433cs.CV2025-04CVPR被引 4

通过保留傅里叶低频特征,提升遥感图像的跨域分类能力

FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing

  • 利用傅里叶滤波保留图像低频结构,抑制背景噪声
  • 在四个遥感数据集上优于现有方法,跨域准确率提升显著
  • 适合遥感场景分类与模型泛化需求的研究者

近年来,像CLIP这样的大规模视觉语言模型因其零样本推理能力而受到关注,可使用文本提示进行推理。尽管这些模型在通用计算机视觉中表现优异,但其在遥感(RS)领域中的跨域泛化潜力仍待探索。现有方法通过生成视觉提示词来增强提示学习,但依赖全图特征,引入了随类别变化的噪声和背景干扰,导致误分类。为此,我们提出FrogDogNet,一种结合傅里叶频率滤波与自注意力机制的新提示学习框架,以提升遥感场景分类与跨域泛化能力。FrogDogNet选择性保留不变的低频成分,消除噪声和无关背景,确保跨域的鲁棒特征表示。模型首先通过投影和自注意力提取关键特征,再应用基于频率的滤波以保留提示学习所需的核心结构信息。在四个遥感数据集和三个跨域泛化任务上的大量实验表明,FrogDogNet持续优于当前最优提示学习方法,展现出更强的域迁移适应能力。研究结果突显了基于频率的不变特征保留对泛化的有效性,为更广泛应用铺平道路。代码已开源:https://github.com/HariseetharamG/FrogDogNet

原文摘要 · Abstract (English)

In recent years, large-scale vision-language models (VLMs) like CLIP have gained attention for their zero-shot inference using instructional text prompts. While these models excel in general computer vision, their potential for domain generalization in remote sensing (RS) remains underexplored. Existing approaches enhance prompt learning by generating visual prompt tokens but rely on full-image features, introducing noise and background artifacts that vary within a class, causing misclassification. To address this, we propose FrogDogNet, a novel prompt learning framework integrating Fourier frequency filtering and self-attention to improve RS scene classification and domain generalization. FrogDogNet selectively retains invariant low-frequency components while eliminating noise and irrelevant backgrounds, ensuring robust feature representation across domains. The model first extracts significant features via projection and self-attention, then applies frequency-based filtering to preserve essential structural information for prompt learning. Extensive experiments on four RS datasets and three domain generalization tasks show that FrogDogNet consistently outperforms state-of-the-art prompt learning methods, demonstrating superior adaptability across domain shifts. Our findings highlight the effectiveness of frequency-based invariant feature retention in generalization, paving the way for broader applications. Our code is available at https://github.com/HariseetharamG/FrogDogNet

遥感图像跨域泛化视觉提示傅里叶特征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。