arXiv:2607.13728cs.IRstat.ML2026-07

通过拍卖机制联合优化向量搜索的聚类与查询探查,提升异分布场景下的检索效率。

Cluster with Auctions for Vector Search

  • 用拍卖算法动态分配数据库向量到平衡聚类,同时训练神经探查函数。
  • 在分布不一致时,吞吐量最高达现有方法的4.7倍,相同召回率下性能更优。
  • 适合处理查询与数据分布不同的大规模向量搜索场景。

大规模近似最近邻搜索通常依赖于分区索引:将数据库向量划分为若干聚类,查询时通过探查函数选择需扫描的聚类。现有方法通常将查询与数据库向量使用相同的分配函数,当两者分布不同时效果不佳。本文提出CwA(Cluster with Auctions),通过联合学习均衡的数据库分区和神经探查函数,直接优化查询分布下的搜索性能。其优化过程交替进行:(i) 对探查函数的神经网络进行梯度下降;(ii) 通过可并行的拍卖算法求解数据库向量的聚类分配,实现分区平衡。为进一步扩展,引入聚类的笛卡尔积以提高划分粒度。当数据库与查询分布不一致时,CwA在相同召回率下达到最高4.7倍的吞吐量优势;在同分布设置下,仅用线性探查函数训练的CwA也超越了复杂深度神经方法。

原文摘要 · Abstract (English)

Large-scale approximate nearest neighbor search commonly relies on partitions for indexing: database vectors are partitioned into clusters, and for each query a probing function selects the clusters to be scanned. The query probing function and the database partition are rarely treated as separate entities: most techniques assign queries with the same assignment function as the database vectors, which is suboptimal especially when database and query distributions differ. This paper introduces CwA (Cluster with Auctions), which addresses this limitation by jointly learning a balanced database partition and a neural probing function. CwA optimizes search performance directly for the query distribution. It minimizes its objective by alternating two steps: (i) gradient descent on the neural network of the probing function, and (ii) a large-scale combinatorial optimization of the cluster assignment for the database vectors. We solve the latter with a parallelizable auction algorithm that balances the partition by design. To further scale CwA, we extend the method to a Cartesian product of clusters that increases the partition's granularity. When database and query distributions differ, CwA achieves up to 4.7$\times$ throughput over the state-of-the-art at equal recall. In the in-distribution (ID) setting, even a simple linear probing function trained with CwA outperforms competing deep neural methods.

向量搜索聚类优化拍卖算法高效检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。