通过学习降维提升向量量化精度与速度，实现更高召回率和更快检索。
ASH: Asymmetric Scalar Hashing With Learned Dimensionality Reduction for High-Fidelity Vector Quantization
- 用学习的正交投影降低数据库向量维度，查询保持原样
- 在相同压缩率下，召回率和速度均优于现有方法
- 适合需要高效相似性计算的实时部署场景
长期以来，加法型量化（如乘积量化）被视为精度与效率的黄金标准。近期，无数据依赖的标量量化技术重新兴起。在此框架下，我们转向数据驱动方法，发现通过减少维度但提高每维比特率，可实现更高的召回率和更快的速度。关键在于该降维必须基于数据学习。本文提出ASH（非对称标量哈希），一种数据驱动的编码-解码框架：通过学习的正交投影对数据库向量降维，再进行标量量化，而查询保持原始形式。这种非对称设计在同等压缩率下，性能超越最优加法型与标量量化器，同时支持基于SIMD的高效相似性计算。ASH训练与编码时间短，适合实际部署。大量实验表明，其在各类数据集上均实现了当前最佳的近似最近邻召回率与速度，覆盖所有压缩率区间。
原文摘要 · Abstract (English)
For a long time, additive quantizers, such as product quantization, have been considered the gold standard in terms of accuracy and efficiency. Recently, scalar quantization has re-emerged from the depths of history with a new wave of data-agnostic techniques. Inscribed in this general framework, we turn our attention to data-driven methods, showing that new highs in recall and speed can be achieved by reducing the number of dimensions while increasing the bitrate per dimension. Critically, this dimensionality reduction needs to be learned from data to be successful. We present ASH (Asymmetric Scalar Hashing), a data-driven encoder-decoder framework that applies dimensionality reduction to database vectors via a learned orthonormal projection, followed by scalar quantization, while keeping queries in their original form. This asymmetric design enables higher accuracy than the best additive and scalar quantizers at iso-compression, while admitting highly efficient similarity computations via SIMD operations. ASH has short learning and encoding times, making it attractive for real-world deployment. Extensive experiments on a variety of datasets demonstrate that ASH achieves state-of-the-art ANN recall and speeds across all compression regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。