用一套模型支持多种精度，训练快、省显存，适合部署灵活的LLM。
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
- 权重分基础低精度+逐层残差修正，单检查点实现多精度
- Qwen3-8B 2/4/6/8bit包仅需1293秒，比对手快3.3倍
- 免校准、无联合优化，适合资源受限场景快速部署
在不同部署条件下服务大语言模型（LLMs）需要在精度、内存和吞吐量之间灵活权衡。传统量化方法通常为每种目标位宽单独保存检查点。本文提出循环残差量化（RRQ），一种后训练量化（PTQ）框架，将权重表示为低比特量化基础与一系列量化残差修正之和，从而从单一检查点生成多种有效精度。从经后训练量化（PTQ）或四舍五入到最近（RTN）得到的2比特模型出发，RRQ通过添加轻量级2比特残差逐步构建出4、6、8比特表示。该方法无需校准，避免了联合多比特优化。在Qwen3-8B设置下，完整的全RTN 2/4/6/8比特包仅耗时1,293秒，比测量的MatGPTQ构建速度快3.3倍。六款近期主流LLM的实验表明，6比特和8比特下表现具有竞争力，4比特则呈现模型依赖性行为。代码将在论文发表后公开。
原文摘要 · Abstract (English)
Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput. However, conventional quantization methods typically require a separate checkpoint for each target bit-width. We introduce Recurrent Residual Quantization (RRQ), a post-training quantization (PTQ) framework that represents weights as a low-bit quantized base together with a sequence of quantized residual corrections, enabling multiple effective precisions from a single checkpoint. Starting from a 2-bit model obtained via post-training quantization (PTQ) or round-to-nearest (RTN), RRQ progressively adds lightweight 2-bit residuals generated via RTN to construct 4-, 6-, and 8-bit representations. The method is calibration-free and avoids joint multi-bit optimization. In our Qwen3-8B setup, the full all-RTN 2-/4-/6-/8-bit package is constructed in 1,293 seconds, 3.3 times faster than the measured MatGPTQ construction. Experiments on six recent LLMs show competitive accuracy at 6 and 8 bits, with model-dependent behavior at 4 bits. The code will be made publicly available upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。