通过哈达玛旋转实现大模型权重近无损量化，显著提升压缩效果。
PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression
- 先将权重归一化至单位超球面，再用哈达玛变换使其近似高斯分布。
- 在Q5量化下，困惑度降至6.40，仅比全精度低0.03，近乎无损。
- 可作为INT4量化的预处理，提升性能同时保持高效推理速度。
我们提出PolarQuant，一种针对大语言模型（LLM）的后训练权重量化方法，利用神经网络权重的分布结构实现近无损压缩。该方法包含三个阶段：(1) 块级归一化至单位超球面；(2) 采用沃尔什-哈达玛旋转，将坐标转换为近似高斯随机变量；(3) 使用与高斯分布匹配的中心点进行量化。消融实验表明，仅哈达玛旋转就贡献了98%的质量提升，使Qwen3.5-9B的困惑度从6.90（absmax Q5）降至6.40（相比FP16仅+0.03），近乎无损且无需校准数据。此外，PolarQuant可有效作为下游INT4量化器的预处理步骤：经PolarQuant Q5去量化再重量化后，torchao INT4的困惑度为6.56，优于直接absmax INT4的6.68，同时维持6.5 GB显存下的43.1 tok/s吞吐。代码与模型已公开。
原文摘要 · Abstract (English)
We present PolarQuant, a post-training weight quantization method for large language models (LLMs) that exploits the distributional structure of neural network weights to achieve near-lossless compression. PolarQuant operates in three stages: (1) block-wise normalization to the unit hypersphere, (2) Walsh-Hadamard rotation to transform coordinates into approximately Gaussian random variables, and (3) quantization with centroids matched to the Gaussian distribution. Our ablation reveals that Hadamard rotation alone accounts for 98% of the quality improvement, reducing Qwen3.5-9B perplexity from 6.90 (absmax Q5) to 6.40 (Delta = +0.03 from FP16), making it practically lossless without any calibration data. Furthermore, PolarQuant functions as an effective preprocessing step for downstream INT4 quantizers: PolarQuant Q5 dequantized and re-quantized by torchao INT4 achieves perplexity 6.56 versus 6.68 for direct absmax INT4, while maintaining 43.1 tok/s throughput at 6.5 GB VRAM. Code and models are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。