arXiv:2602.17099cs.DBcs.IR2026-02

提出高效索引合并方法,大幅提升大规模近邻搜索速度

Multiple Index Merge for Approximate Nearest Neighbor Search

  • 采用反向邻居滑动策略,利用图结构信息加速两个索引合并
  • 通过优化合并顺序减少冗余操作,最高提速5.48倍
  • 适用于超大规模数据(1亿向量、50分区),适合向量数据库场景

高维空间中的近似k近邻(AKNN)搜索是向量数据库的核心问题,应用广泛。基于邻近图的索引在多个基准测试中表现最佳,但其高维向量的大量距离计算导致构建慢、内存开销大。受限于内存容量,大规模数据通常需分批构建多个子索引。然而直接在分离索引上查询会严重降低效率,因无法利用跨图连接。因此,高效的索引合并对多索引搜索至关重要。本文聚焦于两索引合并及多索引合并顺序问题,提出反向邻居滑动合并(RNSM)方法,利用结构信息提升合并效率;进一步研究合并顺序选择(MOS),通过消除冗余合并操作降低开销。实验表明,该方法相较现有合并方法最高提速5.48倍,相较重建索引提速9.92倍,且在1亿向量、50分区规模下仍保持稳定加速效果。

原文摘要 · Abstract (English)

Approximate $k$ nearest neighbor (AKNN) search in high-dimensional space is a foundational problem in vector databases with widespread applications. Among the numerous AKNN indexes, Proximity Graph-based indexes achieve state-of-the-art search efficiency across various benchmarks. However, their extensive distance computations of high-dimensional vectors lead to slow construction and substantial memory overhead. The limited memory capacity often prevents building the entire index at once when handling large-scale datasets. A common practice is to build multiple sub-indexes separately. However, directly searching on these separated indexes severely compromises search efficiency, as queries cannot leverage cross-graph connections. Therefore, efficient graph index merging is crucial for multi-index searching. In this paper, we focus on efficient two-index merging and the merge order of multiple indexes for AKNN search. To achieve this, we propose a reverse neighbor sliding merge (RNSM) that exploits structural information to boost merging efficiency. We further investigate merge order selection (MOS) to reduce the merging cost by eliminating redundant merge operations. Experiments show that our approach yields up to a 5.48$\times$ speedup over existing index merge methods and 9.92$\times$ speedup over index reconstruction, while maintaining expected superior search performance. Moreover, our method scales efficiently to 100 million vectors with 50 partitions, maintaining consistent speedups.

近邻搜索向量数据库索引合并图结构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。