让视觉语言模型根据查询动态调整特征空间，提升大规模图像检索效果。
QuARI: Query Adaptive Retrieval Improvement
- 通过线性变换将查询映射到特定特征空间，实现个性化检索优化。
- 在百万级图像库中显著超越现有方法，且查询耗时仅增加极少。
- 适合需要快速高精度检索的大规模应用，如跨域图像搜索。
大规模预训练使视觉语言模型在多领域图像到图像、文本到图像检索中广泛应用。然而，在超大规模图像集合中的实例检索等挑战任务上表现不佳。近期研究发现，对视觉语言模型特征进行线性变换以强调目标领域相关子空间，可提升检索性能。本文探索更激进的专化策略：学习将给定查询映射到查询相关的特征空间变换。由于该变换为线性，可低开销地应用于数百万图像嵌入，适用于大规模检索或重排序。实验表明，该方法持续优于当前最优方案，包括那些需查询时间耗费数个数量级更多的方法。
原文摘要 · Abstract (English)
Massive-scale pretraining has made vision-language models increasingly popular for image-to-image and text-to-image retrieval across a broad collection of domains. However, these models do not perform well when used for challenging retrieval tasks, such as instance retrieval in very large-scale image collections. Recent work has shown that linear transformations of VLM features trained for instance retrieval can improve performance by emphasizing subspaces that relate to the domain of interest. In this paper, we explore a more extreme version of this specialization by learning to map a given query to a query-specific feature space transformation. Because this transformation is linear, it can be applied with minimal computational cost to millions of image embeddings, making it effective for large-scale retrieval or re-ranking. Results show that this method consistently outperforms state-of-the-art alternatives, including those that require many orders of magnitude more computation at query time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。