通过剪枝、量化与排序，实现低比特点积累积，提升推理效率。
PQS (Prune, Quantize, and Sort): Low-Bitwidth Accumulation of Dot Products in Neural Network Computations
- 先对浮点权重迭代剪枝，再量化至8比特以下，减少冗余信息。
- 按从小到大顺序累加部分积，避免溢出，支持2.5倍压缩的累加器位宽。
- 适用于图像分类模型，精度接近浮点基线，适合边缘设备部署。
我们提出PQS方法，结合剪枝、量化和排序三种技术，实现神经网络计算中点积的低比特宽度累加。传统量化（如8比特）点积需使用宽累加器（如32比特）以避免中间部分和溢出，但会增加内存带宽开销并降低能效。本文表明，在浮点域进行迭代N:M剪枝后，将结果量化至8比特或更少，并以“小到大”顺序累加部分积，可在不使用宽累加器的情况下实现准确且紧凑的模型，同时支持短点积长度。我们设计、分析并实现了PQS算法，成功在多种神经网络上消除推理时的累加溢出问题。该方法使累加器位宽降低2.5倍，同时在多个图像分类任务上保持与浮点基线相当的模型精度。
原文摘要 · Abstract (English)
We present PQS, which uses three techniques together - Prune, Quantize, and Sort - to achieve low-bitwidth accumulation of dot products in neural network computations. In conventional quantized (e.g., 8-bit) dot products, partial results are accumulated into wide (e.g., 32-bit) accumulators to avoid overflows when accumulating intermediate partial sums. However, such wide accumulators increase memory bandwidth usage and reduce energy efficiency. We show that iterative N:M pruning in floating point followed by quantization to 8 (or fewer) bits, and accumulation of partial products in a sorted order ("small to large") allows for accurate, compressed models with short dot product lengths that do not require wide accumulators. We design, analyze, and implement the PQS algorithm to eliminate accumulation overflows at inference time for several neural networks. Our method offers a 2.5x reduction in accumulator bitwidth while achieving model accuracy on par with floating-point baselines for multiple image classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。