arXiv:2506.03437cs.IR2025-06被引 24

Quake动态调整向量索引,应对数据变化与访问模式波动。

Quake: Adaptive Indexing for Vector Search

  • 多级分区自适应更新与访问模式变化
  • 查询延迟降低1.5-38倍,更新延迟降低4.5-126倍
  • 适合高动态性、分布不均的向量搜索场景

向量搜索是找到查询向量在高维向量数据库中的k近邻,广泛应用于检索增强生成、推荐系统和信息检索。然而,现有近似最近邻(ANN)方法在数据分布动态变化的负载下表现不佳。我们提出Quake,一种自适应索引系统,在此类环境中保持低延迟和高召回率。Quake采用多级分区方案,根据更新频率和访问模式动态调整,由成本模型预测查询延迟。它还利用新型召回率估计模型,动态设置查询参数以满足召回目标。此外,通过NUMA感知的查询内并行化提升内存带宽利用率。为评估系统,我们构建了维基百科向量搜索工作负载,并开发了可配置访问模式的工作负载生成器。实验表明,在动态工作负载下,Quake相比最新索引(SVS、DiskANN、HNSW、SCANN)实现1.5-38倍的查询延迟降低和4.5-126倍的更新延迟降低。

原文摘要 · Abstract (English)

Vector search, the task of finding the k-nearest neighbors of a query vector against a database of high-dimensional vectors, underpins many machine learning applications, including retrieval-augmented generation, recommendation systems, and information retrieval. However, existing approximate nearest neighbor (ANN) methods perform poorly under dynamic and skewed workloads where data distributions evolve. We introduce Quake, an adaptive indexing system that maintains low latency and high recall in such environments. Quake employs a multi-level partitioning scheme that adjusts to updates and changing access patterns, guided by a cost model that predicts query latency based on partition sizes and access frequencies. Quake also dynamically sets query execution parameters to meet recall targets using a novel recall estimation model. Furthermore, Quake utilizes NUMA-aware intra-query parallelism for improved memory bandwidth utilization during search. To evaluate Quake, we prepare a Wikipedia vector search workload and develop a workload generator to create vector search workloads with configurable access patterns. Our evaluation shows that on dynamic workloads, Quake achieves query latency reductions of 1.5-38x and update latency reductions of 4.5-126x compared to state-of-the-art indexes such as SVS, DiskANN, HNSW, and SCANN.

向量搜索自适应索引近似最近邻动态负载

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。