提出新型量化方法YAQA，直接优化模型输出误差，显著提升压缩后模型性能。
Model-Preserving Adaptive Rounding
- 通过分析海森矩阵近似结构，设计自适应舍入策略
- 理论证明可实现端到端误差上界，实测误差降低约30%
- 无需推理开销，优于训练感知量化，适合部署场景
量化的目标是生成一个输出分布尽可能接近原始模型的压缩模型。现有大多数量化算法为简化计算，仅最小化每层即时激活误差作为端到端误差的代理，但忽略了后续层的影响，导致该代理效果不佳。本文提出一种新型自适应舍入算法YAQA，直接考虑网络输出端的误差。通过一系列理论推导，首次为量化算法建立了端到端误差的理论边界。首先，我们基于海森矩阵近似的结构刻画了自适应舍入算法的收敛时间；随后证明，端到端误差可由近似矩阵与真实海森矩阵之间的余弦相似度界定。由此自然导出一种克罗内克分解近似，并得到近似最优的海森矩阵草图。实验表明，YAQA在理论上优于GPTQ/ LDLQ，且实测误差降低约30%；甚至低于量化感知训练的效果。在下游任务中达到当前最优表现，且不增加推理开销。
原文摘要 · Abstract (English)
The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by $\approx 30\%$ over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。