arXiv:2605.05344cs.CVcs.AI2026-05被引 1

用大模型动态优化遥感图像检索的文本嵌入,无需训练即可提升准确率。

Open-SAT: LLM-Guided Query Embedding Refinement for Open-Vocabulary Object Retrieval in Satellite Imagery

论文配图:Open-SAT: LLM-Guided Query Embedding Refinement for Open-Vocabulary Object Retrieval in Satellite Imagery
图 1 · 摘自论文原文
  • 利用大模型在推理时动态优化查询文本嵌入,增强与遥感图像的语义对齐。
  • 在三个公开数据集上最高提升F1分数16.04%,检索结果数量相当。
  • 无需额外训练,适合需要快速适应新查询场景的遥感应用开发者。

在遥感应用中,用户查询常以开放式自然语言形式出现,超出预定义类别范围。这种开放词汇特性给图像片元检索带来挑战,因系统需泛化至大量未见物体和概念。尽管视觉-语言模型(如CLIP)广泛用于图文检索,其微调变体仍难以精准对齐此类查询与卫星影像。为此,我们提出Open-SAT,一种无需训练的推理时查询嵌入优化算法,通过大型语言模型(LLMs)结合目标对象及其环境的上下文信息,改进文本嵌入以提升与卫星图像内容的匹配度。该方法将图像片元的嵌入存入向量数据库,实现高效检索,并采用无阈值机制进一步提高准确性和效率。在三个公开基准上的实验表明,Open-SAT在保持相似检索数量的前提下,最高使F1得分提升16.04%。结果证明,Open-SAT在开放词汇遥感图像检索中有效,借助大模型引导而无需额外训练或监督。

原文摘要 · Abstract (English)

In satellite applications, user queries often take the form of open-ended natural language, extending beyond a fixed set of predefined categories. This open-vocabulary nature poses significant challenges for retrieving relevant image tiles, as the retrieval system must generalize to a wide range of unseen objects and concepts. While vision-language models (VLMs) such as CLIP are widely used for text-image retrieval, even fine-tuned variants often struggle to accurately align such queries with satellite imagery. To address this, we propose Open-SAT, a training-free query embedding refinement algorithm that operates at inference time to improve alignment between user queries and satellite image content. Open-SAT uses VLMs to compute embeddings for image tiles, which are stored in a vector database for efficient retrieval. At query time, it leverages Large Language Models (LLMs) to refine the text embeddings by incorporating contextual information about objects of interest and their surroundings. A threshold-free retrieval mechanism further enhances accuracy and efficiency. Experimental results in three public benchmarks demonstrate that Open-SAT improves the F1 score by up to 16.04%, while retrieving a comparable number of image tiles. These results demonstrate the effectiveness of Open-SAT in open-vocabulary satellite image retrieval, leveraging LLM guidance without the need for additional training or supervision.

遥感图像开放词汇大模型检索优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。