arXiv:2605.00422cs.LGcs.AI2026-05ACL被引 3

提出首个实现1比特权重量化与低比特激活的后训练压缩框架,显著提升大模型推理速度。

BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs

论文配图:BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs
图 1 · 摘自论文原文
  • 通过正交克罗内克变换与近端SVD投影,实现权重1比特化和激活6比特化
  • Qwen3-32B在6比特激活下困惑度降至11.92,比现有方法降低约70%
  • 推理速度提升3.26倍,适合部署高精度大模型的边缘设备

大语言模型(LLMs)推动了自然语言处理的快速发展,但其巨大的内存与计算需求仍制约实际部署。权重量化至1比特可从根本上降低计算与带宽开销,但现有方法难以处理激活值的长尾分布,导致激活必须保持高精度,无法实现端到端加速。为此,本文提出BWLA(Binarized Weights and Low-bit Activations),首个支持1比特权重量化与低比特激活(如6比特)的后训练量化框架。其核心是正交克罗内克变换(OKT),通过期望最大化(EM)优化学习正交映射,将单峰权重转化为对称双峰形式,抑制激活尾部与不一致性;再通过近端SVD投影(PSP)进行轻量级低秩精修,进一步提升可量化性且开销极小。在Qwen3-32B上,BWLA在6比特激活下达到11.92的Wikitext2困惑度(优于当前最优方案的38),零样本任务性能提升超70%,推理速度提升3.26倍,展现出强大的真实场景压缩与加速潜力。

原文摘要 · Abstract (English)

Large language models (LLMs) have driven major progress in NLP, yet their substantial memory and compute demands still hinder practical deployment. Binarization can compress weights to 1 bit, fundamentally lowering compute and bandwidth cost. However, existing methods cannot address activation heavy tails and thus must keep activations in high precision, preventing true end-to-end acceleration. To overcome this limitation, we propose BWLA (Binarized Weights and Low-bit Activations), the first post-training quantization framework that preserves high accuracy while achieving 1-bit weight quantization together with low-bit activations (e.g., 6 bits). The Orthogonal-Kronecker Transformation (OKT) learns an orthogonal mapping via EM minimization, converting unimodal weights into symmetric bimodal forms while suppressing activation tails and incoherence. The Proximal SVD Projection (PSP) then performs lightweight low-rank refinement through proximal SVD projection, further enhancing quantizability with minimal overhead. On Qwen3-32B, BWLA reaches a Wikitext2 perplexity of 11.92 under 6-bit activations (vs. 38 from SOTA), improves five zero-shot tasks by more than 70%, and delivers 3.26 times inference speedup, demonstrating strong potential for real-world LLM compression and acceleration.

大模型压缩量化推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。