用少量预存图动态构建查询范围邻近搜索索引,兼顾效率与内存。
iRangeGraph: Improvising Range-dedicated Graphs for Range-filtering Nearest Neighbor Search
- 预存少量基础图(elemental graphs),查询时动态组合生成目标范围索引
- 实验表明在真实数据集上空间开销适中,查询性能稳定且优于现有方法
- 适合需要高效处理范围约束最近邻查询的数据库系统或推荐场景
范围过滤近似最近邻(RFANN)搜索在学术界和工业界日益受到关注。给定一组数据对象(每个包含一个高维向量和一个数值),以向量和数值范围为参数的RFANN查询需返回数值在查询范围内的、向量最接近查询向量的数据对象。近期研究提出为所有可能的查询范围构建O(n²)专用图索引以实现高效处理,但存储全部索引代价过高。为此,该研究采用压缩索引以降低内存开销,但因压缩导致性能下降。本文不预先构建所有范围的压缩索引,而是仅预存少量基础图(elemental graphs),针对有限数量的范围。查询时,通过高效算法利用这些基础图动态构建任意查询范围所需的索引。我们证明该构建过程耗时极低。实验基于真实数据集验证:预存的基础图仅占用适度空间,且所提方法在不同查询负载下均能实现更优且稳定的查询性能。
原文摘要 · Abstract (English)
Range-filtering approximate nearest neighbor (RFANN) search is attracting increasing attention in academia and industry. Given a set of data objects, each being a pair of a high-dimensional vector and a numeric value, an RFANN query with a vector and a numeric range as parameters returns the data object whose numeric value is in the query range and whose vector is nearest to the query vector. To process this query, a recent study proposes to build $O(n^2)$ dedicated graph-based indexes for all possible query ranges to enable efficient processing on a database of $n$ objects. As storing all these indexes is prohibitively expensive, the study constructs compressed indexes instead, which reduces the memory consumption considerably. However, this incurs suboptimal performance because the compression is lossy. In this study, instead of materializing a compressed index for every possible query range in preparation for querying, we materialize graph-based indexes, called elemental graphs, for a moderate number of ranges. We then provide an effective and efficient algorithm that during querying can construct an index for any query range using the elemental graphs. We prove that the time needed to construct such an index is low. We also cover an experimental study on real-world datasets that provides evidence that the materialized elemental graphs only consume moderate space and that the proposed method is capable of superior and stable query performance across different query workloads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。