对比12种图结构向量搜索方法,验证其在百亿级数据下的性能差异。
Graph-Based Vector Search: An Experimental Evaluation of the State-of-the-Art
- 按五类设计范式分类,构建统一评估框架。
- 增量插入与邻域多样化方法表现最佳,基图选择影响扩展性。
- 适合大规模向量检索系统选型与算法研究者参考。
向量数据在商业和科学应用中广泛存在,随着学习嵌入的普及,其规模已达数十亿向量,维度达数千。向量搜索是众多分析任务的核心,图基方法成为无需答案质量保证场景下的首选。本文简要回顾内存中图基向量搜索的发展,梳理不同方法的时间脉络,并根据五类设计范式(种子选择、增量插入、邻域传播、邻域多样化、分治)进行分类。对十二种前沿方法在七个真实数据集(最大达10亿向量)上进行全面实验评估,揭示关键洞察:最优方法通常基于增量插入与邻域多样化,且基图选择显著影响可扩展性。最后讨论开放研究方向,如设计更智能的数据自适应种子选择与多样化策略。
原文摘要 · Abstract (English)
Vector data is prevalent across business and scientific applications, and its popularity is growing with the proliferation of learned embeddings. Vector data collections often reach billions of vectors with thousands of dimensions, thus, increasing the complexity of their analysis. Vector search is the backbone of many critical analytical tasks, and graph-based methods have become the best choice for analytical tasks that do not require guarantees on the quality of the answers. We briefly survey in-memory graph-based vector search, outline the chronology of the different methods and classify them according to five main design paradigms: seed selection, incremental insertion, neighborhood propagation, neighborhood diversification, and divide-and-conquer. We conduct an exhaustive experimental evaluation of twelve state-of-the-art methods on seven real data collections, with sizes up to 1 billion vectors. We share key insights about the strengths and limitations of these methods; e.g., the best approaches are typically based on incremental insertion and neighborhood diversification, and the choice of the base graph can hurt scalability. Finally, we discuss open research directions, such as the importance of devising more sophisticated data-adaptive seed selection and diversification strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。