改进神经码本的向量压缩与搜索，大幅提升精度与效率。
Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks
- 用预选和束搜索优化编码，提升压缩质量。
- 8字节编码下搜索准确率提升24%，16字节压缩重建误差降34%。
- 适合需要高精度向量检索的大规模应用，如推荐系统。
向量量化是压缩与大规模最近邻搜索的基础技术。对于高精度场景，多码本量化将数据向量与多个码本中的元素关联。例如残差量化（RQ）通过迭代量化前一步的残差误差实现。但传统RQ忽略了各部分码本间的依赖关系，导致率失真性能不佳。QINCo通过神经网络根据前序步骤的重构结果动态确定码本，缓解了该问题。本文提出QINCo2，进一步改进：(i) 采用码字预选与束搜索优化编码；(ii) 设计快速近似解码器，利用码字对生成精准候选列表用于搜索；(iii) 优化训练流程与网络结构。在四个数据集上评估了其在向量压缩与百亿级最近邻搜索的表现。结果显示，在BigANN上16字节压缩重建均方误差比现有最优降低34%，在Deep1M上8字节编码搜索准确率提升24%。
原文摘要 · Abstract (English)
Vector quantization is a fundamental technique for compression and large-scale nearest neighbor search. For high-accuracy operating points, multi-codebook quantization associates data vectors with one element from each of multiple codebooks. An example is residual quantization (RQ), which iteratively quantizes the residual error of previous steps. Dependencies between the different parts of the code are, however, ignored in RQ, which leads to suboptimal rate-distortion performance. QINCo recently addressed this inefficiency by using a neural network to determine the quantization codebook in RQ based on the vector reconstruction from previous steps. In this paper we introduce QINCo2 which extends and improves QINCo with (i) improved vector encoding using codeword pre-selection and beam-search, (ii) a fast approximate decoder leveraging codeword pairs to establish accurate short-lists for search, and (iii) an optimized training procedure and network architecture. We conduct experiments on four datasets to evaluate QINCo2 for vector compression and billion-scale nearest neighbor search. We obtain outstanding results in both settings, improving the state-of-the-art reconstruction MSE by 34% for 16-byte vector compression on BigANN, and search accuracy by 24% with 8-byte encodings on Deep1M.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。