用局部描述符做全局重排序，提升图像检索精度。
LOCORE: Image Re-ranking with Long-Context Sequence Modeling
- 基于长序列建模，统一处理查询与候选图的局部特征
- 在多个数据集上优于现有重排序方法，延迟接近配对式方法
- 适合需要高精度检索的场景，如商品、鸟类识别
我们提出LOCORE——一种基于长上下文建模的图像重排序模型。输入为图像查询的局部描述符和一组候选图像，输出查询与每张候选图的相似度分数。该模型用于图像检索，通常先用高效相似度度量进行初步排序，再对前若干名候选图使用更精细的重排序。相比以往基于局部描述符的成对相似性估计或基于全局描述符的列表级重排序，LOCORE是首个采用局部描述符进行列表级重排序的方法。为此，我们利用高效的长上下文序列模型，在局部描述符层面有效捕捉查询与候选图之间的依赖关系。测试时，通过滑动窗口策略处理长候选列表，克服序列模型的上下文长度限制。在地标（ROxf、RPar）、商品（SOP）、时尚物品（In-Shop）和鸟类物种（CUB-200）等标准图像检索基准上，该方法表现显著优于其他重排序器，且延迟与成对式局部描述符重排序器相当。
原文摘要 · Abstract (English)
We introduce LOCORE, Long-Context Re-ranker, a model that takes as input local descriptors corresponding to an image query and a list of gallery images and outputs similarity scores between the query and each gallery image. This model is used for image retrieval, where typically a first ranking is performed with an efficient similarity measure, and then a shortlist of top-ranked images is re-ranked based on a more fine-grained similarity measure. Compared to existing methods that perform pair-wise similarity estimation with local descriptors or list-wise re-ranking with global descriptors, LOCORE is the first method to perform list-wise re-ranking with local descriptors. To achieve this, we leverage efficient long-context sequence models to effectively capture the dependencies between query and gallery images at the local-descriptor level. During testing, we process long shortlists with a sliding window strategy that is tailored to overcome the context size limitations of sequence models. Our approach achieves superior performance compared with other re-rankers on established image retrieval benchmarks of landmarks (ROxf and RPar), products (SOP), fashion items (In-Shop), and bird species (CUB-200) while having comparable latency to the pair-wise local descriptor re-rankers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。