arXiv:2504.05644cs.CV2025-04被引 8

通过先筛选后对齐与关键词显式推理,提升遥感图文检索精度。

iEBAKER: Improved Remote Sensing Image-Text Retrieval Framework via Eliminate Before Align and Keyword Explicit Reasoning

  • 先过滤弱相关样本再对齐,减少错误干扰
  • 在三个基准数据集上超越现有模型,且训练数据更少
  • 适合需要高效精准遥感图文匹配的科研与应用者

近期研究聚焦于遥感图像-文本检索(RSITR),旨在根据给定查询搜索对应目标。尽管基于基础模型(如CLIP)的方法已取得良好效果,但现有方法忽略了弱相关样本对的影响,且未充分考虑遥感文本间的细微差异,导致嵌入空间偏差和浅层匹配。为此,本文提出iEBAKER框架,采用创新的“先消除后对齐”(EBA)策略,剔除弱相关样本对,降低对齐过程中的偏离。进一步地,设计了考虑局部与全局相似性相互影响的两种方案,并引入反向检索优化相似度矩阵的“排序后反向检索”(SAR)策略。同时,加入关键词显式推理(KER)模块,增强对关键概念差异的捕捉能力。无需额外遥感数据预训练,即可直接适配基础模型至遥感任务。在三个主流基准数据集上的大量实验表明,iEBAKER显著优于现有最优模型,且训练数据需求更低。代码将开源于https://github.com/zhangy0822/iEBAKER。

原文摘要 · Abstract (English)

Recent studies focus on the Remote Sensing Image-Text Retrieval (RSITR), which aims at searching for the corresponding targets based on the given query. Among these efforts, the application of Foundation Models (FMs), such as CLIP, to the domain of remote sensing has yielded encouraging outcomes. However, existing FM based methodologies neglect the negative impact of weakly correlated sample pairs and fail to account for the key distinctions among remote sensing texts, leading to biased and superficial exploration of sample pairs. To address these challenges, we propose an approach named iEBAKER (an Improved Eliminate Before Align strategy with Keyword Explicit Reasoning framework) for RSITR. Specifically, we propose an innovative Eliminate Before Align (EBA) strategy to filter out the weakly correlated sample pairs, thereby mitigating their deviations from optimal embedding space during alignment.Further, two specific schemes are introduced from the perspective of whether local similarity and global similarity affect each other. On this basis, we introduce an alternative Sort After Reversed Retrieval (SAR) strategy, aims at optimizing the similarity matrix via reverse retrieval. Additionally, we incorporate a Keyword Explicit Reasoning (KER) module to facilitate the beneficial impact of subtle key concept distinctions. Without bells and whistles, our approach enables a direct transition from FM to RSITR task, eliminating the need for additional pretraining on remote sensing data. Extensive experiments conducted on three popular benchmark datasets demonstrate that our proposed iEBAKER method surpasses the state-of-the-art models while requiring less training data. Our source code will be released at https://github.com/zhangy0822/iEBAKER.

遥感图像图文检索基础模型关键词推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。