用快速变换替代乘法，实现高效低功耗向量量化。
Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach
- 用快速沃尔什-哈达玛变换替代密集矩阵乘法
- 在DBpedia数据集上实现19.7倍算法加速
- 适合边缘设备部署的低硬件开销量化方案
随着大语言模型规模扩大，键值缓存和检索增强生成系统的内存带宽成为关键瓶颈。虽然1比特量化可缓解此问题，但近期的TurboQuant依赖密集随机旋转矩阵在量化前调节向量分布，每嵌入需数百万次浮点乘法，难以在资源受限的边缘芯片上部署。本文提出Fast-TurboQuant，一种无乘法器的投影架构，将密集矩阵替换为结构化的快速Johnson-Lindenstrauss变换。通过雷德马克相位反转结合快速沃尔什-哈达玛变换（FWHT），该方法利用次高斯集中性满足标量Lloyd-Max量化的前提条件，无需高斯投影。这一替换将计算复杂度降至仅加法操作，彻底消除硬件乘法器需求。在DBpedia OpenAI-3 Large嵌入上评估显示，顺序执行下算法速度提升19.7倍。此外，FWHT零填充带来的维度扩展降低了均方误差，并提升了Recall@10。
原文摘要 · Abstract (English)
As large language models scale, memory bandwidth for key-value caches and retrieval-augmented generation systems becomes a critical bottleneck. While 1-bit quantization addresses this constraint, recent TurboQuant relies on dense random rotation matrices to condition the vector distribution before quantization. This projection demands millions of floating-point multiplications per embedding, making it difficult to deploy on constrained edge silicon. We introduce Fast-TurboQuant, a multiplier-free projection architecture that replaces the dense matrix with a structured fast Johnson-Lindenstrauss transform. By applying a Rademacher phase inversion followed by a fast Walsh-Hadamard transform (FWHT), the method leverages sub-Gaussian concentration to satisfy the prerequisites of scalar Lloyd-Max quantization without Gaussian projections. This substitution reduces the arithmetic complexity to only additions, eliminating hardware multipliers. Evaluation on DBpedia OpenAI-3 Large embeddings demonstrates a 19.7 times algorithmic speedup under sequential execution. Furthermore, the dimension expansion due to the FWHT zero-padding reduces the mean squared error and improves Recall@10.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。