用稀疏编码替代聚类,实现高效精准的多向量检索。
No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval

- 用稀疏自编码器将词元嵌入转为高维稀疏表示,避免聚类。
- 索引时间减少15倍,检索延迟减半,性能优于现有基线。
- 适合需要高速高精度检索的场景,如大规模语料库搜索。
多向量检索(MVR)模型,如ColBERT,通过保留细粒度的词元级交互实现了检索准确性的新基准。然而,这种细粒度带来了存储和检索效率的巨大瓶颈:为应对百亿级词元向量带来的内存和计算开销,现有系统不得不依赖激进的降维和复杂聚类(如K-means)。这引入了两个关键限制:大规模语料聚类导致的过高索引延迟,以及压缩固有的语义信息损失。本文提出单阶段稀疏检索(SSR),彻底改变范式,以高效稀疏编码替代昂贵的聚类。不同于将特征压缩为低维稠密向量,我们使用稀疏自编码器(SAE)将词元嵌入映射到高维但高度稀疏的表示。该转换使我们能够完全跳过向量聚类,并利用倒排索引实现精确、高吞吐的检索。在BEIR基准上的大量实验表明,SSR实现了三重提升:相比ColBERTv2,索引时间减少15倍,检索延迟降低一半,同时在检索性能上超越领先基线。
原文摘要 · Abstract (English)
Multi-vector retrieval (MVR) models, exemplified by ColBERT, have established new benchmarks in retrieval accuracy by preserving fine-grained token-level interactions. However, this granularity imposes prohibitive storage and retrieval efficiency bottlenecks: to manage the immense memory footprint and computational overhead of billion-scale token vectors, state-of-the-art systems are forced to rely on aggressive dimension reduction and complex clustering (e.g., K-means). This compromise introduces two critical limitations: excessive indexing latency of clustering large-scale corpora and semantic information loss inherent to compression. In this paper, we propose Single-stage Sparse Retrieval (SSR}, a paradigm shift that replaces expensive clustering with efficient sparse coding. Instead of compressing features into low-dimensional dense vectors, we utilize Sparse Autoencoder (SAE) to project token embeddings into a high-dimensional but highly sparse representation. This transformation enables us to bypass vector clustering entirely and leverage inverted indexing for precise, high-throughput retrieval. Extensive experiments on the BEIR benchmark demonstrate that SSR achieves a "trifecta" of improvements: it reduces indexing time by 15x compared to ColBERTv2, halves retrieval latency, and simultaneously improves retrieval performance over leading baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。