Qronos通过迭代修正实现更精准的模型量化，显著提升压缩后性能。
Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization
- 分步优化：交替进行误差修正与权重扩散，逐步纠正量化误差。
- 在Llama3模型上，量化后性能超越现有最优方法，尤其在激活和键值缓存上表现突出。
- 兼容多种加速技术，可高效部署于实际推理场景。
我们提出Qronos——一种新的、达到当前最佳水平的后训练量化算法，通过顺序地对神经网络权重进行舍入并更新。Qronos不仅显式纠正由权重和激活量化带来的误差，还纠正前层量化引起的误差。其迭代算法基于可解释且严谨的优化框架，涵盖并超越了现有的数据驱动方法。每一步中，Qronos通过最优更新规则交替执行误差校正与扩散。重要的是，我们证明Qronos可通过乔列斯基分解高效实现，用于求解最小二乘问题。此外，我们展示了Qronos与现有变换技术（如基于哈达玛的非相干处理、权重-激活缩放均衡等）具有兼容性。我们在Llama3系列最新自回归语言生成模型上评估Qronos；在量化权重、激活和/或键值缓存时，其性能始终优于以往最先进的自适应舍入方法。
原文摘要 · Abstract (English)
We introduce Qronos -- a new state-of-the-art post-training quantization algorithm that sequentially rounds and updates neural network weights. Qronos not only explicitly corrects errors due to both weight and activation quantization, but also errors resulting from quantizing previous layers. Our iterative algorithm is based on an interpretable and disciplined optimization framework that subsumes and surpasses existing data-driven approaches. At each step, Qronos alternates between error correction and diffusion via optimal update rules. Importantly, we prove that Qronos admits an efficient implementation that uses the Cholesky decomposition for solving least-squares problems. We also demonstrate that Qronos is compatible with existing transformation techniques such as Hadamard-based incoherence processing and weight-activation scaling equalization, among others. We evaluate Qronos using recent autoregressive language generation models in the Llama3 family; Qronos consistently outperforms previous state-of-the-art adaptive rounding methods when quantizing the weights, activations, and/or KV caches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。