用向量编码压缩量化误差，让大模型推理更快更准。
Revisiting Adaptive Rounding with Vectorized Reparameterization for LLM Quantization
- 将复杂误差矩阵转为紧凑代码本，大幅降低参数量
- 仅用128样本即可端到端优化，训练效率提升99.8%
- 适用于千亿参数大模型，支持快速收敛
自适应量化（Adaptive Rounding）通过跨元素误差抵消，成为后训练量化的新选择。然而，其稠密的逐元素量化矩阵对千亿参数大语言模型（LLMs）而言成本过高。本文从效率角度重新审视该方法，提出参数高效的VQRound框架，将量化矩阵重参数化为紧凑代码本。与低秩替代方案不同，VQRound在$L_\infty$范数下最小化逐元素最坏情况误差，这对处理长尾权重分布至关重要。此外，我们发现量化初始化是关键因素，并设计了轻量级端到端微调流程，仅需128个样本即可在所有层优化代码本。在OPT、LLaMA、LLaMA2和Qwen3模型上的大量实验表明，VQRound在相同迭代次数下收敛优于传统自适应量化，且可训练参数少至0.2%。结果证明自适应量化可同时实现可扩展性与快速拟合。代码已开源：https://github.com/zhoustan/VQRound。
原文摘要 · Abstract (English)
Adaptive Rounding has emerged as an alternative to round-to-nearest (RTN) for post-training quantization by enabling cross-element error cancellation. Yet, dense and element-wise rounding matrices are prohibitively expensive for billion-parameter large language models (LLMs). We revisit adaptive rounding from an efficiency perspective and propose VQRound, a parameter-efficient optimization framework that reparameterizes the rounding matrix into a compact codebook. Unlike low-rank alternatives, VQRound minimizes the element-wise worst-case error under $L_\infty$ norm, which is critical for handling heavy-tailed weight distributions in LLMs. Beyond reparameterization, we identify rounding initialization as a decisive factor and develop a lightweight end-to-end finetuning pipeline that optimizes codebooks across all layers using only 128 samples. Extensive experiments on OPT, LLaMA, LLaMA2, and Qwen3 models demonstrate that VQRound achieves better convergence than traditional adaptive rounding at the same number of steps while using as little as 0.2% of the trainable parameters. Our results show that adaptive rounding can be made both scalable and fast-fitting. The code is available at https://github.com/zhoustan/VQRound.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。