提出自适应增量更新方法，提升向量检索索引的实时维护效率。
Incremental IVF Index Maintenance for Streaming Vector Search
- 根据性能下降情况动态选择需重分区的索引块
- 局部重聚类实现高效更新，吞吐量最高提升5倍
- 适合需要持续更新的实时向量搜索场景
向量相似性搜索在现代机器学习应用中广泛应用，而这些应用处理的数据持续变化，亟需高效有效的索引维护技术。现有向量索引主要针对静态负载设计，数据更新后搜索质量与性能会下降，除非进行昂贵的重建。为此，我们提出Ada-IVF，一种面向倒排文件（IVF）索引的增量式维护方法。该方法包含：1）自适应维护策略，判断哪些索引分区性能下降需重分区；2）局部重聚类机制，决定如何重分区。相比当前最先进的动态IVF索引维护策略，Ada-IVF在多种基准工作负载下平均提升2倍、最高达5倍的更新吞吐量。
原文摘要 · Abstract (English)
The prevalence of vector similarity search in modern machine learning applications and the continuously changing nature of data processed by these applications necessitate efficient and effective index maintenance techniques for vector search indexes. Designed primarily for static workloads, existing vector search indexes degrade in search quality and performance as the underlying data is updated unless costly index reconstruction is performed. To address this, we introduce Ada-IVF, an incremental indexing methodology for Inverted File (IVF) indexes. Ada-IVF consists of 1) an adaptive maintenance policy that decides which index partitions are problematic for performance and should be repartitioned and 2) a local re-clustering mechanism that determines how to repartition them. Compared with state-of-the-art dynamic IVF index maintenance strategies, Ada-IVF achieves an average of 2x and up to 5x higher update throughput across a range of benchmark workloads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。