提出高效深度量化方法DQA，实现3-5比特激活值压缩且精度显著提升。
DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations
- 基于移位操作与哈夫曼编码，避免复杂计算，适合资源受限设备。
- 在3-5比特下，图像分类与分割任务精度比直接量化高至29.28%。
- 专为亚6比特深度量化设计，适合边缘端部署的轻量级模型应用。
深度神经网络（DNN）激活值的量化是降低推理阶段计算与内存开销的常用技术，尤其适用于资源受限设备。现有方法为保证高精度，依赖复杂的数学运算或大量超参数搜索，但这些操作在计算能力、内存和能耗受限的设备上不切实际。此外，多数方法未关注6比特以下（即深度）量化。为此，本文提出DQA（Deep Quantization of DNN Activations），专注于亚6比特激活量化，采用简单的移位操作与哈夫曼编码，兼具高效性与高精度。我们在三个不同模型上，针对图像分类与分割两个任务，在两个数据集上评估了3、4、5比特量化。DQA在亚6比特量化下相比直接量化和当前最优方法NoisyQuant，精度显著提升，最高达29.28%。
原文摘要 · Abstract (English)
Quantization of Deep Neural Network (DNN) activations is a commonly used technique to reduce compute and memory demands during DNN inference, which can be particularly beneficial on resource-constrained devices. To achieve high accuracy, existing methods for quantizing activations rely on complex mathematical computations or perform extensive searches for the best hyper-parameters. However, these expensive operations are impractical on devices with limited computation capabilities, memory capacities, and energy budgets. Furthermore, many existing methods do not focus on sub-6-bit (or deep) quantization. To fill these gaps, in this paper we propose DQA (Deep Quantization of DNN Activations), a new method that focuses on sub-6-bit quantization of activations and leverages simple shifting-based operations and Huffman coding to be efficient and achieve high accuracy. We evaluate DQA with 3, 4, and 5-bit quantization levels and three different DNN models for two different tasks, image classification and image segmentation, on two different datasets. DQA shows significantly better accuracy (up to 29.28%) compared to the direct quantization method and the state-of-the-art NoisyQuant for sub-6-bit quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。