提出统计优化量化方法，提升低精度神经网络的精度与稳定性。
StatQAT: Statistical Quantizer Optimization for Deep Networks
- 基于统计误差分析设计迭代与解析量化器，适配不同数据分布。
- 在整数与浮点格式下均实现更高精度与训练稳定性。
- 适合需要高效低精度推理的模型部署场景。
量化是降低深度神经网络计算成本和内存占用的关键技术，使低精度硬件上的高效推理成为可能。尽管统一量化和浮点量化方案日益普及，但如何为多样化的训练与推理数据分布选择最优量化参数仍是核心挑战。本文提出一种针对统一量化和浮点量化的新型统计误差分析框架，揭示了不同量化配置下的误差行为规律。基于该分析，我们设计了适用于任意数据分布的迭代量化器，以及针对高斯类权值分布的解析量化器。这些方法可实现高效、低误差的激活与权重量化，并集成至量化感知训练中，在整数与浮点格式下均验证了其优越性能。实验表明，该方法显著提升了低精度神经网络的准确率与训练稳定性。
原文摘要 · Abstract (English)
Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision hardware. Despite the growing adoption of uniform and floating-point quantization schemes, selecting optimal quantization parameters remains a key challenge, particularly for diverse data distributions encountered during training and inference. This work presents a novel statistical error analysis framework for uniform and floating-point quantization, providing theoretical insight into error behavior across quantization configurations. Building on this analysis, we propose iterative quantizers designed for arbitrary data distributions and analytic quantizers tailored for Gaussian-like weight distributions. These methods enable efficient, low-error quantization suitable for both activations and weights. We incorporate our quantizers into quantization-aware training and evaluate them across integer and floating-point formats. Experiments demonstrate improved accuracy and stability, highlighting the effectiveness of our approach for training low-precision neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。