arXiv:2512.10596cs.CVcs.AI2025-12

无需训练,用文本匹配实现遥感图像高效语义检索

Beyond Pixels: A Training-Free, Text-to-Text Framework for Remote Sensing Image Retrieval

  • 将跨模态检索转为纯文本匹配,直接用结构化描述作为查询
  • 在RSITMD上达42.62%召回率,近乎翻倍于CLIP基线的23.86%
  • 适合追求低成本、零训练部署的遥感图像检索应用

遥感图像的语义检索面临核心挑战——语义鸿沟,即模型低层视觉特征与人类高层概念之间的差异。尽管大视觉语言模型(VLMs)有望弥合此鸿沟,但现有方法常依赖昂贵的领域特定训练,且缺乏评估VLM生成文本在零样本检索中实用性的基准。为此,我们提出遥感丰富文本(RSRT)数据集,包含每幅图像的多条结构化描述。基于该数据集,我们构建完全免训练、仅文本的检索框架TRSLLaVA。方法将跨模态检索重构为文本到文本(T2T)匹配问题,利用丰富文本描述作为查询,在统一文本嵌入空间中与VLM生成的描述库进行匹配,彻底规避模型训练或微调。在RSITMD和RSICD基准上的实验表明,该免训练方法性能媲美顶尖监督模型。例如,在RSITMD上,本方法达到42.62%平均召回率,近乎翻倍于标准零样本CLIP基线的23.86%,并超越多个顶级监督模型。这验证了通过结构化文本生成高质量语义表示是一种强大且经济高效的遥感图像检索范式。

原文摘要 · Abstract (English)

Semantic retrieval of remote sensing (RS) images is a critical task fundamentally challenged by the \textquote{semantic gap}, the discrepancy between a model's low-level visual features and high-level human concepts. While large Vision-Language Models (VLMs) offer a promising path to bridge this gap, existing methods often rely on costly, domain-specific training, and there is a lack of benchmarks to evaluate the practical utility of VLM-generated text in a zero-shot retrieval context. To address this research gap, we introduce the Remote Sensing Rich Text (RSRT) dataset, a new benchmark featuring multiple structured captions per image. Based on this dataset, we propose a fully training-free, text-only retrieval reference called TRSLLaVA. Our methodology reformulates cross-modal retrieval as a text-to-text (T2T) matching problem, leveraging rich text descriptions as queries against a database of VLM-generated captions within a unified textual embedding space. This approach completely bypasses model training or fine-tuning. Experiments on the RSITMD and RSICD benchmarks show our training-free method is highly competitive with state-of-the-art supervised models. For instance, on RSITMD, our method achieves a mean Recall of 42.62\%, nearly doubling the 23.86\% of the standard zero-shot CLIP baseline and surpassing several top supervised models. This validates that high-quality semantic representation through structured text provides a powerful and cost-effective paradigm for remote sensing image retrieval.

遥感图像文本检索零样本免训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。