HARP通过自适应旋转提升极端低比特大模型量化效果。
HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
- 用可学习的结构化正交变换替代固定哈达玛变换，自适应调整量化基。
- 在2-4比特下，相比固定变换提升困惑度与零样本准确率。
- 保持高效推理速度，达到128 tok/s，适合部署场景。
训练后量化（PTQ）对在内存和带宽受限下部署大语言模型至关重要。然而，极端低比特量化仍对激活异常值和权重曲率各向异性高度敏感。现有基于不一致性的PTQ方法采用固定的随机哈达玛变换（RHT），虽提升鲁棒性但无法根据层、校准分布或量化器自适应调整旋转基。本文提出HARP（哈达玛预处理自适应旋转处理器），一种可学习的双侧正交结构化处理器，取代固定哈达玛混合，同时保持全精度等价性。每个旋转表示为稀疏蝴蝶状块正交阶段的乘积，支持非2的幂次维度的混合进制调度，并以固定置换初始化至原生RHT。仅在校准数据上拟合，HARP可自适应各层与后端。在1B至70B参数模型上，2-4比特设置下，HARP显著优于固定RHT，且保持高部署效率，达到128 tok/s，远超FP16的61 tok/s。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) is essential for deploying LLMs under memory and bandwidth constraints. However, extreme low-bit quantization remains highly sensitive to activation outliers and anisotropic weight curvature. Existing incoherence-based PTQ methods mitigate this issue with fixed randomized Hadamard transforms (RHTs), which improve quantization robustness but cannot adapt the rotated basis to the layer, calibration distribution, or quantizer. We introduce HARP (Hadamard-preconditioned Adaptive Rotation Processor), a learnable structured two-sided orthogonal processor that replaces fixed Hadamard mixing while preserving exact full-precision equivalence. HARP represents each rotation as a product of sparse butterfly-like block-orthogonal stages, supports non-power-of-two dimensions via Mixed-Radix schedules, and initializes to the RHT processor up to a fixed permutation. Fitted only on calibration data, HARP adapts the quantization basis to each layer and backend. Across 2-4 bit settings on models ranging from 1B to 70B parameters, HARP improves perplexity and zero-shot accuracy over fixed RHT. Importantly, HARP preserves deployment efficiency, reaching 128 tok/s versus 61 tok/s for FP16.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。