提出DBellQuant框架，实现大模型近1比特权重量化与6比特激活量化。
DBellQuant: Breaking the Bell with Double-Bell Transformation for LLMs Post Training Binarization
- 通过双钟形变换将权重分布转为双峰形式，降低二值化误差
- 在LLaMA2-13B上实现14.39的困惑度（6比特激活），优于对比方法
- 适合需要极致压缩且保持性能的部署场景
大型语言模型虽表现卓越，但计算与内存开销限制了实际应用。量化成为有前景的解决方案，但常受非量化友好权重分布及激活异常值影响。本文提出DBellQuant，一种后训练量化框架，实现近1比特权重量化和6比特激活量化，性能损失极小。其核心是可学习双钟形变换（LTDB）算法，将单峰权重分布转化为双峰结构以减少二值化误差，并对激活进行逆变换平滑。在Wikitext2数据集上，该方法在LLaMA2-13B上达到14.39的困惑度，显著优于无激活量化的BiLLM（21.35），展示了在真实场景中压缩大模型的巨大潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) demonstrate remarkable performance but face substantial computational and memory challenges that limit their practical deployment. Quantization has emerged as a promising solution; however, its effectiveness is often limited by quantization errors arising from weight distributions that are not quantization-friendly and the presence of activation outliers. To address these challenges, we introduce DBellQuant, an innovative post-training quantization (PTQ) framework that achieves nearly 1-bit weight compression and 6-bit activation quantization with minimal performance degradation. DBellQuant uses Learnable Transformation for Dual-Bell (LTDB) algorithm, which transforms single-bell weight distributions into dual-bell forms to reduce binarization errors and applies inverse transformations to smooth activations. DBellQuant sets a new state-of-the-art by preserving superior model performance under aggressive weight and activation quantization. For example, on the Wikitext2 dataset, DBellQuant achieves a perplexity of 14.39 on LLaMA2-13B with 6-bit activation quantization, significantly outperforming BiLLM's 21.35 without activation quantization, underscoring its potential in compressing LLMs for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。