2-bit大模型量化精度提升11.88%，通过优化分组权值和残差补偿实现。
SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
- 基于舒尔补的分组离散优化，精确消除残差影响
- 2比特下对Qwen3-4B模型零样本准确率提升11.88个百分点
- 适用于无反向传播的高效量化部署，适合资源受限场景
权重仅后训练量化（PTQ）可在严苛内存约束下部署大语言模型，但2-3比特时精度常严重下降。现有无反向传播的PTQ优化器存在两个缺陷：分组决策忽略剩余连续后缀的校正能力，且离散优化通常固定仿射量化网格。本文提出SCHUROPT，通过解析消除后缀最优连续响应，获得精确的分组二次型（具有舒尔补曲率）。随后交替进行闭式行级缩放/零点重拟合与整数码坐标下降。在保持GPTQ目标不变的前提下，SCHUROPT使2比特Qwen3-4B模型的平均零样本准确率提升11.88个百分点。然而在更高精度下，更紧重构未必提升下游指标。因此，SCHURQUANT进一步结合量化前缀教师重建、参考权重正则化、残差添加目标及教师决策词加权。在八款Llama与Qwen模型上，该方法在所有无反向传播的基准中达到最高均值零样本准确率，在2比特下比最强基线高出9.65个百分点。
原文摘要 · Abstract (English)
Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits. Existing backpropagation-free PTQ optimizers have two limitations: group decisions ignore the correction that the remaining continuous suffix can absorb, and discrete refinements typically keep the affine quantization grid fixed. We introduce SCHUROPT, which analytically eliminates the suffix's optimal continuous response, yielding an exact groupwise quadratic with Schur-complement curvature. It then alternates closed-form row-wise scale/zero-point refitting with coordinate descent over integer codes. With the GPTQ objective fixed, SCHUROPT improves mean zero-shot accuracy on 2-bit Qwen3-4B by 11.88 percentage points (pp). At higher precision, however, tighter reconstruction does not consistently improve end-model metrics. SCHURQUANT therefore combines SCHUROPT with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and teacher-decision token weighting. Across eight Llama and Qwen models, SCHURQUANT achieves the highest mean zero-shot accuracy among the evaluated backpropagation free PTQ baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。