让图搜索更快:通过自适应选择入口点提升近邻查询效率
Empowering Graph-based Approximate Nearest Neighbor Search with Adaptive Awareness Capabilities
- 构建可自适应选择入口点的轻量模块,利用聚类特性筛选候选节点
- 在真实数据上实现1.2至2.0倍查询加速,显著优于现有图索引
- 适合需要高维近邻搜索的推荐系统与信息检索场景
高维空间中的近似最近邻搜索(ANNS)广泛应用于数据库、信息检索和推荐系统。尽管基于图的方法因查询性能优越成为主流,但仍面临陷入局部最优和冗余计算等问题,主要源于未能充分挖掘邻近图G的拓扑信息,且实际中基础数据与查询间存在严重分布差异。为此,本文提出GATE——一种位于图索引之上的轻量级自适应模块,具备拓扑感知与查询感知能力,以加速ANNS。GATE首先利用高维数据的可聚类性,提取少量枢纽节点V作为候选入口点;随后通过基于对比学习的双塔模型,将图结构语义与查询相关特征编码至这些枢纽节点的隐表示中;最终构建枢纽节点上的导航图索引,降低模型推理开销。大量实验表明,相比最先进的图索引,GATE在查询性能上实现了1.2至2.0倍的加速。
原文摘要 · Abstract (English)
Approximate Nearest Neighbor Search (ANNS) in high-dimensional spaces finds extensive applications in databases, information retrieval, recommender systems, etc. While graph-based methods have emerged as the leading solution for ANNS due to their superior query performance, they still face several challenges, such as struggling with local optima and redundant computations. These issues arise because existing methods (i) fail to fully exploit the topological information underlying the proximity graph G, and (ii) suffer from severe distribution mismatches between the base data and queries in practice. To this end, this paper proposes GATE, high-tier proximity Graph with Adaptive Topology and Query AwarEness, as a lightweight and adaptive module atop the graph-based indexes to accelerate ANNS. Specifically, GATE formulates the critical problem to identify an optimal entry point in the proximity graph for a given query, facilitating faster online search. By leveraging the inherent clusterability of high-dimensional data, GATE first extracts a small set of hub nodes V as candidate entry points. Then, resorting to a contrastive learning-based two-tower model, GATE encodes both the structural semantics underlying G and the query-relevant features into the latent representations of these hub nodes V. A navigation graph index on V is further constructed to minimize the model inference overhead. Extensive experiments demonstrate that GATE achieves a 1.2-2.0X speed-up in query performance compared to state-of-the-art graph-based indexes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。