统一了从随机投影到RAG时代的哈希学习方法,揭示压缩检索的核心机制。
Projection and Quantisation: A Unifying View of Learning to Hash, from Random Projections to the RAG Era
- 用投影-量化-组织框架解析各类检索方法的设计选择。
- 一比特编码加全精度重排,6/7嵌入器达到无压缩效果,代码仅占原大小1/32。
- 标签指导下8字节有监督编码,检索质量超2KB无监督浮点编码一倍以上。
近似最近邻搜索支撑大规模检索与检索增强生成,但相关研究分散在互不交流的领域。我们提出投影-量化-组织视角:所有方法均在投影、量化阈值和编码组织方式上做设计选择。通过可复现的测试,发布开源BitBudget基准,得出三方面发现:第一,量化轴带来最大内存节省——一比特编码结合全精度重排序,在七种嵌入器中六种达到无压缩质量,编码尺寸仅为浮点的1/32;第二,该框架预测的编码顺序(如二值码在大模型下超越倒排文件乘积量化)随嵌入规模增大持续出现;第三,若有类别标签,8字节有监督编码的检索性能超过2KB无任务感知浮点编码两倍以上。我们还将生成式检索中的语义标识重新解释为量化码。核心贡献是首次以实证方式统一解释从随机投影到检索增强时代的紧凑编码搜索。
原文摘要 · Abstract (English)
Approximate nearest-neighbour search underpins large-scale retrieval and retrieval-augmented generation, yet its methods are studied in communities that seldom read one another. We argue that they form one field with three design choices. We develop the projection-quantisation-organisation lens: every method places its projections, places its quantisation thresholds, and organises the resulting codes for search. We test the lens with a reproducible measurement, released as the open BitBudget benchmark, and report three findings. First, the quantisation axis delivers the largest memory savings: a one-bit code with full-precision re-ranking matches uncompressed quality for six of seven embedders, the scanned code one thirty-second of the float's size. Second, the orderings the lens anticipates, including a learned-embedding regime where binary codes overtake an inverted-file product quantiser at a matched byte budget, recur as the embedding is enlarged. Third, given class labels, an eight-byte supervised code more than doubles the retrieval quality of the two-kilobyte task-agnostic float it replaces. We also recast the semantic identifiers of generative retrieval as quantisation codes. The main contribution is a single, tested account of compact-code search, from random projections to the retrieval-augmented era.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。