通过谱衰减特性加速近邻搜索验证，显著提升检索速度。
Panorama: Fast-Track Nearest Neighbors
- 利用PCA压缩信号能量，分步计算距离下界并动态剪枝。
- 在任意召回率下实现最高每秒查询数，最快提速28.9倍。
- 兼容量化索引，适合大规模向量检索系统部署。
高维神经嵌入的近似最近邻搜索（ANNS）中，候选验证占查询时间主要部分，成为主要瓶颈。本文提出PANORAMA，一种基于嵌入固有谱衰减特性的先进精炼技术。通过主成分分析（PCA）压缩信号能量，分步计算完整向量距离的严格下界，并在该下界超过当前第k近邻距离时立即剪枝候选。虽PCA破坏了产品量化（PQ）所需的均匀方差假设，但通过能量重分配的方差整形步骤解决了此问题，使累积精炼兼容量化索引。针对现代内存层次结构优化，采用向量化批量剪枝和缓存友好的数据布局。PANORAMA已集成至FAISS库，支持多种索引类型（IVFPQ/Flat、HNSW、Refine）。其性能在任意目标召回率下均优于现有方法，每秒查询数（QPS）更高，且加速比与数据集谱衰减成反比，端到端速度提升最高达28.9倍，在所有召回率区间均超越概率性方法。
原文摘要 · Abstract (English)
Approximate Nearest-Neighbor Search (ANNS) pipelines for high-dimensional neural embeddings spend the bulk of their query time in candidate verification, making it the primary bottleneck in the search process. In this paper, we present PANORAMA, a state-of-the-art refinement technique that accelerates verification by exploiting the inherent spectral decay of these embeddings. Using PCA to compact signal energy, PANORAMA evaluates candidate distances incrementally, computing at each step a strict lower bound on the full-vector distance and dynamically pruning candidates the moment this bound exceeds the running k-th nearest neighbor distance. While PCA's concentration of variance facilitates pruning, it breaks the uniform-variance assumption required by Product Quantization (PQ); we resolve this with a variance-shaping step that redistributes energy across subvectors, rendering accretive refinement compatible with quantized indexes. Optimized for modern memory hierarchies via vectorized bulk-pruning and cache-conscious data layouts, PANORAMA has been upstreamed into the FAISS library across major index families (IVFPQ/Flat, HNSW, and Refine). PANORAMA achieves higher QPS at any target recall with a cost that provably scales inversely with dataset spectral decay, delivering end-to-end speedups of up to 28.9x and outperforming probabilistic methods across all recall bands.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。