4-bit模型量化新方法，速度极快且精度更高。
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
- 为每层动态选择最优的两个可学习码本，提升权重编码精度。
- 仅需3-30分钟完成量化，比现有方法快数百倍。
- 零额外内存开销，适合资源受限场景快速部署。
训练后权重仅量化至4比特广泛用于降低大语言模型推理的内存与计算成本。现有方法如AWQ和GPTQ通过缩放、裁剪或误差补偿改进权重在固定4比特网格上的映射。为进一步提升精度，OmniQuant和QuIP#等方法采用梯度辅助算法，但需数小时量化时间。本文提出轻量级4比特量化方法AAAC（Activation-Aware Adaptive Codebooks），将标准量化中固定的标量码本替换为每层两个小的可学习标量码本（共64字节）。每组权重选择使激活加权重构误差最小的码本，通过正比例系数的未使用符号位编码选择，不增加存储开销。AAAC在单张GPU上耗时仅3–30分钟，且不引入额外内存。我们在多个模型家族上对比了AWQ、GPTQ、IF4、GPTVQ、OmniQuant、SqueezeLLM和QuIP#，结果表明，AAAC在数量级更低的量化时间内超越所有基线。
原文摘要 · Abstract (English)
Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference. Existing PTQ methods, such as AWQ and GPTQ, improve how weights are mapped onto a fixed 4-bit grid through scaling, clipping, or error compensation. To further improve accuracy, methods such as OmniQuant and QuIP\# uses gradient-assisted algorithms at the cost of hours of quantization time. In this work, we propose AAAC (Activation-Aware Adaptive Codebooks), a lightweight method for 4-bit LLM weight quantization. AAAC replaces the fixed scalar codebook used in standard quantization with two small learned scalar codebooks (64 bytes) per layer. Each group of weights selects the codebook that minimizes activation-weighted reconstruction error, encoding the choice in the unused sign bit of the group's positive scale and adding zero storage overhead. AAAC completes in 3--30 minutes on a single GPU, and adds no memory beyond the model itself. We evaluate against AWQ, GPTQ, IF4, GPTVQ, OmniQuant, SqueezeLLM, and QuIP\# across model families. AAAC outperforms baselines at orders-of-magnitude less quantization time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。