提出新型后训练量化方法，实现大模型权重和激活的极致压缩。
Achieving binary weight and activation for LLMs using Post-Training Quantization
- 通过海森感知分组与基于EM的量化方案，实现权重1比特+1位分组精细控制。
- 激活采用4倍通道扩展的1比特表示，量化误差显著降低。
- 在多个任务上超越现有4比特以下量化方法，推动全二值化大模型发展。
将大语言模型（LLMs）量化至1比特可大幅降低计算开销，但现有技术在权重与激活精度低于4比特（W4A4）时性能明显下降。本文提出一种后训练量化框架，采用W(1+1)A(1*4)配置：权重量化为1比特并附加1比特用于细粒度分组，激活量化为1比特且通道数扩大4倍。针对权重量化，提出基于海森矩阵感知的细粒度分组结合基于EM的量化方案；针对激活量化，将INT4量化激活等效分解为4×INT1格式，并同步平滑缩放因子以减少量化误差。实验表明，该方法在多任务上优于当前最优的W2A4量化基线，推动了大模型量化向全二值化方向迈进。代码已开源：https://github.com/JimmyCrave/LLM-PTQ-binarization。
原文摘要 · Abstract (English)
Quantizing large language models (LLMs) to 1-bit precision significantly reduces computational costs, but existing quantization techniques suffer from noticeable performance degradation when using weight and activation precisions below 4 bits (W4A4). In this paper, we propose a post-training quantization framework with W(1+1)A(1*4) configuration, where weights are quantized to 1 bit with an additional 1 bit for fine-grain grouping and activations are quantized to 1 bit with a 4-fold increase in the number of channels. For weight quantization, we propose utilizing Hessian-aware fine-grained grouping along with an EM-based quantization scheme. For activation quantization, we decompose INT4-quantized activations into a 4 * INT1 format equivalently and simultaneously smooth the scaling factors based on quantization errors, which further reduces the quantization errors in activations. Our method surpasses state-of-the-art (SOTA) LLM quantization baselines on W2A4 across multiple tasks, pushing the boundaries of existing LLM quantization methods toward fully binarized models. Code is available at https://github.com/JimmyCrave/LLM-PTQ-binarization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。