提出可精准保持内积的量化方法,提升压缩效率与模型精度。
Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms

- 设计自适应量化方案,确保与任意输入向量的内积近似保留
- 理论证明算法在最坏和平均情况下的内积保真度,性能优于传统方法
- 实际应用中比现有最优方法快2-10倍,适合大规模高效推理场景
量化是压缩数据集、神经网络权重及内存使用的核心技术。许多下游任务需对任意输入执行向量内积,这促使研究能近似保持与未见向量内积的量化方法——而非仅最小化均方误差。本文提出符合自然期望的目标函数,并开发出自适应、无偏的量化算法,可在最坏情况与平均情况下近似保留内积。理论分析揭示其与经典自适应随机量化(ASQ)存在紧密联系。我们设计了可证明高效的精确与近似算法。理论结果启发了高效实用算法,在多种工作负载分布下表现优异。同时,这些成果也推动标准ASQ实现2-10倍加速,且保持高质量。本研究使自适应量化在实际应用中更高效、更可行。
原文摘要 · Abstract (English)
Quantization is a fundamental tool used to compress datasets, neural network weights, and memory usage in a range of computational tasks. Many downstream applications of vector quantization perform inner products with arbitrary inputs. This motivates the study of inner product aware quantization schemes that approximately preserve inner products with unseen vectors -- in contrast to simply minimizing the mean-squared error. In this work, we formulate objectives that capture natural desiderata and develop adaptive and unbiased quantization methods that approximately preserve inner products with worst-case and average-case inputs. An analysis of these objectives shows a tight connection with the well-studied notion of Adaptive Stochastic Quantization (ASQ). We develop provably fast exact and approximate algorithms for our objectives. Our theoretical results inspire efficient practical algorithms that perform well across a variety of workload distributions. They also lead to practical algorithms for standard ASQ which are 2-10$\times$ faster than prior state-of-the-art methods while maintaining quality. These theoretical and empirical results contribute towards making adaptive quantization techniques more efficient and tractable in practical settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。