arXiv:2410.07022cs.IR2024-10ICLR被引 4

用约束自编码器提升大模型图像检索性能,兼顾效率与扩展性。

Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval

  • 引入方差强约束的自编码器,优化大模型嵌入空间分布
  • 在SoP和Pittsburgh30k上提升最多16%检索准确率
  • 支持自适应嵌入大小,小尺寸下仍保持高效率

图像检索在机器人定位和视觉推荐中至关重要。现有系统面临可扩展性与效率双重挑战:通常为每个数据集训练专用神经网络,难以扩展;且检索速度随嵌入维度增大而下降。虽有研究采用现成的基础模型跨数据集应用,但性能仍不及定制模型。本文发现,基础模型虽具备必要语义信息,但其嵌入空间分布不利于余弦相似度搜索。为此提出带强方差约束的自编码器(AE-SVC),用于嵌入投影,显著提升性能。进一步提出单次相似性空间蒸馏((SS)²D),学习可自适应调整大小的嵌入,实现尺寸与性能更好平衡。在四个数据集(包括Stanford Online Products和Pittsburgh30k)上,使用DinoV2、CLIP等四种基础模型进行实验,结果表明,AE-SVC使检索性能最高提升16%,(SS)²D在小嵌入尺寸下再提升10%。

原文摘要 · Abstract (English)

Image retrieval is crucial in robotics and computer vision, with downstream applications in robot place recognition and vision-based product recommendations. Modern retrieval systems face two key challenges: scalability and efficiency. State-of-the-art image retrieval systems train specific neural networks for each dataset, an approach that lacks scalability. Furthermore, since retrieval speed is directly proportional to embedding size, existing systems that use large embeddings lack efficiency. To tackle scalability, recent works propose using off-the-shelf foundation models. However, these models, though applicable across datasets, fall short in achieving performance comparable to that of dataset-specific models. Our key observation is that, while foundation models capture necessary subtleties for effective retrieval, the underlying distribution of their embedding space can negatively impact cosine similarity searches. We introduce Autoencoders with Strong Variance Constraints (AE-SVC), which, when used for projection, significantly improves the performance of foundation models. We provide an in-depth theoretical analysis of AE-SVC. Addressing efficiency, we introduce Single-shot Similarity Space Distillation ((SS)$_2$D), a novel approach to learn embeddings with adaptive sizes that offers a better trade-off between size and performance. We conducted extensive experiments on four retrieval datasets, including Stanford Online Products (SoP) and Pittsburgh30k, using four different off-the-shelf foundation models, including DinoV2 and CLIP. AE-SVC demonstrates up to a $16\%$ improvement in retrieval performance, while (SS)$_2$D shows a further $10\%$ improvement for smaller embedding sizes.

图像检索基础模型嵌入优化高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。