arXiv:2506.15689cs.LGcs.AI2025-06被引 4

通过偏差修正与非对称缩放,显著降低大模型量化中的误差。

BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models

  • 引入偏差校正和非对称缩放,优化旋转量化机制。
  • 相比QuaRot等方法,精度差距缩小超29%以上。
  • 支持分块优化,无需全模型反向传播,节省内存。

旋转已成为大语言模型量化流水线的核心技术,能有效平滑权重和激活中的异常值。然而,进一步优化旋转参数带来的性能提升有限,并引入显著训练开销:由于旋转参数共享,反向传播需同时加载整个模型,导致内存消耗巨大,实用性受限。本文识别出当前旋转量化的两大根本缺陷:(i) 旋转无法对齐通道均值,导致量化范围变宽,增加舍入误差;(ii) 旋转使激活分布更趋近高斯分布,加剧截断误差引发的能量损失。为此,我们提出 extbf{BASE-Q},一种简单而强大的方法,结合偏差校正与非对称缩放,有效降低舍入与截断误差。此外,BASE-Q支持分块优化,无需内存密集的全模型反向传播。在多种LLM与基准测试上的实验表明,相比QuaRot、SpinQuant和OSTQuant,BASE-Q将精度差距分别缩小50.5%、42.9%和29.2%。代码即将发布。

原文摘要 · Abstract (English)

Rotations have become essential to state-of-the-art quantization pipelines for large language models (LLMs) by effectively smoothing outliers in weights and activations. However, further optimizing the rotation parameters offers only limited performance gains and introduces significant training overhead: due to rotation parameter sharing, full-model must be loaded simultaneously to enable backpropagation, resulting in substantial memory consumption and limited practical utility. In this work, we identify two fundamental limitations of current rotational quantization methods: (i) rotation fails to align channel means, resulting in wider quantization bounds and increased rounding errors; and (ii) rotation makes the activation distribution more Gaussian-like, increasing energy loss caused by clipping errors. To address these issues, we introduce \textbf{BASE-Q}, a simple yet powerful approach that combines bias correction and asymmetric scaling to effectively reduce rounding and clipping errors. Furthermore, BASE-Q enables blockwise optimization, eliminating the need for memory-intensive full-model backpropagation. Extensive experiments on various LLMs and benchmarks demonstrate the effectiveness of BASE-Q, narrowing the accuracy gap to full-precision models by 50.5\%, 42.9\%, and 29.2\% compared to QuaRot, SpinQuant, and OSTQuant, respectively. The code will be released soon.

量化大模型旋转量化高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。