arXiv:2602.04929cs.LG2026-02被引 2

提出无需反向传播的快速精准量化方法,加速三倍以上且精度更高。

TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation

  • 多输出通道联合量化+闭式误差补偿,打破逐层瓶颈。
  • 比BoA快3倍以上,低比特下仍保持高精度。
  • 适合部署大模型的高效量化,尤其适合资源受限场景。

大语言模型的快速发展使得训练后量化(PTQ)在降低内存与计算开销方面愈发重要。尽管GPTQ因高效性广受关注,能在数小时内在单个GPU上完成数十亿参数模型的量化,但其分层独立假设导致低比特下精度严重下降。近期的BoA通过引入注意力模块内的层间依赖关系提升了精度,但依赖所有输出通道的串行量化,效率显著降低。本文提出TurboBoA,一种无需反向传播的PTQ算法,在保持BoA精度优势的同时大幅加速。其核心创新包括:(i) 多输出通道联合量化与闭式误差补偿规则,消除串行瓶颈,实现三倍以上加速;(ii) 对前序量化层误差传播的修正机制;(iii) 自适应网格计算结合坐标下降精修,维持迭代更新中的对齐性。大量实验表明,TurboBoA相比BoA有显著加速,且持续提升精度。结合异常值抑制技术后,在仅权重与权重-激活量化中均达到当前最优表现。代码将公开于https://github.com/SamsungLabs/TurboBoA。

原文摘要 · Abstract (English)

The rapid growth of large language models (LLMs) has heightened the importance of post-training quantization (PTQ) for reducing memory and computation costs. Among PTQ methods, GPTQ has gained significant attention for its efficiency, enabling billion-scale LLMs to be quantized within a few GPU hours. However, GPTQ's assumption of layer-wise independence leads to severe accuracy drops in low-bit regimes. Recently, BoA improved upon GPTQ by incorporating inter-layer dependencies within attention modules, but its reliance on sequential quantization across all out-channels makes it substantially less efficient. In this paper, we propose TurboBoA, a new backpropagation-free PTQ algorithm that preserves the accuracy benefits of BoA while significantly accelerating the process. The proposed TurboBoA introduces three key innovations: (i) joint quantization of multiple out-channels with a closed-form error compensation rule, which reduces sequential bottlenecks and yields more than a three-fold speedup; (ii) a correction mechanism for errors propagated from preceding quantized layers; and (iii) adaptive grid computation with coordinate descent refinement to maintain alignment during iterative updates. Extensive experiments demonstrate that TurboBoA delivers substantial acceleration over BoA while consistently improving accuracy. When combined with outlier suppression techniques, it achieves state-of-the-art results in both weight-only and weight-activation quantization. The code will be available at https://github.com/SamsungLabs/TurboBoA.

量化大模型加速注意力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。