超低比特量化压缩大模型权重与缓存，效果远超现有方法。
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
- 用哈达玛变换+最优格点量化，让权重分布更接近高斯分布。
- 4比特下权重压缩3.9倍，键值缓存压缩3.79倍，几乎无质量损失。
- 适用于大语言模型和视频生成模型，适合追求极致压缩的开发者。
我们提出HyperQuant（Hadamard, optimallY Packing, Entropy Rice-coding），一种统一的后训练量化流水线，用于大型语言模型和扩散模型的权重及键值缓存。在一系列自包含实验中（表1），HyperQuant在3至5比特每标量（bps）范围内，性能优于近期的HIGGS方案；在键值缓存量化上，其表现超越TurboQuant与OCTOPUS，最低可达1.7 bps。在非LLM场景下，该方法对190亿参数的LTX-2 DiT视频模型进行量化，未产生可察觉的逐帧伪影。在H100上端到端运行，4 bps下将线性权重压缩约3.9倍，键值缓存压缩约3.79倍，保持近无损质量。HyperQuant融合四种已有技术：(i) 每块的随机哈达玛变换，使权重与激活的坐标分布近似高斯；(ii) 低维最优格点（E8、D4、A2或Z）量化；(iii) 无损位剥离与近熵最优的可变长度Rice编码；(iv) 键值缓存的偏置修正方法，确保内积不变性，保留注意力语义。此外，该流水线集成8比特与4比特张量核心矩阵乘路径（fp8-e4m3、int8、nvfp4、mxfp4），发现int8在经哈达玛变换后的格点输出上优于fp8。
原文摘要 · Abstract (English)
We present HyperQuant (Hadamard, optimallY Packing, Entropy Rice-coding), a unified post-training quantization pipeline for the weights and the KV cache of large language and diffusion transformers. Across a suite of self-contained experiments (Table 1), HyperQuant outperforms the recent HIGGS scheme at every operating point from 3 to 5 bits per scalar (bps) on weights, and beats both TurboQuant and OCTOPUS on KV quantization down to 1.7 bps. Beyond the LLM setting, HyperQuant quantizes the 19B-parameter LTX-2 DiT video model with no observable per-frame artifacts. End-to-end on an H100 at 4 bps, HyperQuant compresses the linear weights ~3.9x and the KV cache ~3.79x at near-lossless quality. HyperQuant combines four known ideas into a single construction: (i) a per-tile Randomized Hadamard Transform that makes the per-coordinate distribution of weights and activations approximately Gaussian; (ii) quantization to a low-dimensional optimal lattice (E8, D4, A2, or Z); (iii) lossless bit-stripping and near-entropy-optimal variable-length Rice coding of the lattice indices; and (iv) bias-correction methods for the KV cache that keep the reconstruction unbiased under inner products, preserving attention semantics. We further integrate the pipeline with 8-bit and 4-bit Tensor-Core MMA paths (fp8-e4m3, int8, nvfp4, mxfp4), and find that int8 beats fp8 on the post-RHT lattice output. Project page: https://moonmath.ai/hyperquant/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。