2-bit量化让大模型在单张显卡上运行，准确率接近16位原版。
BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
- 按比特平面分解构建可变量化网格，动态调整精度分布。
- 2比特下在RTX 3090上运行Qwen2.5-72B，GSM8K准确率达83.85%。
- 理论证明可变网格提升优化空间，适合资源受限的大模型部署。
大语言模型推理常受内存占用和带宽限制，量化是高效服务的关键。尽管后训练量化（PTQ）在4比特时保持高精度，但在2-3比特时性能下降明显。现有方法对每组采用形状不变的量化网格（如UINT2的固定均匀区间），严重限制了误差最小化的可行解空间。为此，我们提出比特平面分解量化（BPDQ），通过比特平面与标量系数构建可变量化网格，并利用二阶信息迭代优化，逐步补偿量化误差以最小化输出偏差。在2比特场景下，BPDQ实现Qwen2.5-72B在单张RTX 3090上的部署，GSM8K准确率达83.85%（16比特为90.83%）。此外，我们从理论上证明可变网格扩大了可行集，且量化过程始终沿海森几何方向逼近优化目标。代码已开源。
原文摘要 · Abstract (English)
Large language model inference is often bounded by memory footprint and bandwidth in resource-constrained deployments, making quantization fundamental to efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. In essence, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using second-order information while progressively compensating for quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85\% GSM8K accuracy (vs. 90.83\% at 16-bit). Moreover, we theoretically show that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. The code is available at https://github.com/KingdalfGoodman/BPDQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。