根据查询频率动态分配计算资源，提升高维相似搜索效率
Adaptive Prefiltering for High-Dimensional Similarity Search: A Frequency-Aware Approach
- 按查询频率分层并差异化分配搜索策略
- 相同召回率下减少20.4%距离计算量
- 适合需要低延迟的图像检索系统
高维相似搜索支撑现代检索系统，但统一策略难以应对真实查询分布的异质性。本文提出一种自适应预过滤框架，利用查询频率模式与聚类一致性指标，动态分配计算预算。方法依据齐普夫分布对查询空间分层，结合历史访问模式与局部密度特征分配不同搜索策略。在ImageNet-1k上使用CLIP嵌入的实验表明，频率感知的预算分配在保持亚毫秒延迟的同时，相比静态nprobe选择可减少20.4%的距离计算量且达到相同召回率。框架通过轻量级频率追踪引入极小开销，并通过基于一致性的回退策略实现对未见查询的平滑降级。
原文摘要 · Abstract (English)
High-dimensional similarity search underpins modern retrieval systems, yet uniform search strategies fail to exploit the heterogeneous nature of real-world query distributions. We present an adaptive prefiltering framework that leverages query frequency patterns and cluster coherence metrics to dynamically allocate computational budgets. Our approach partitions the query space into frequency tiers following Zipfian distributions and assigns differentiated search policies based on historical access patterns and local density characteristics. Experiments on ImageNet-1k using CLIP embeddings demonstrate that frequency-aware budget allocation achieves equivalent recall with 20.4% fewer distance computations compared to static nprobe selection, while maintaining sub-millisecond latency on GPU-accelerated FAISS indices. The framework introduces minimal overhead through lightweight frequency tracking and provides graceful degradation for unseen queries through coherence-based fallback policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。