为嵌入式设备优化神经网络的浮点精度分配，提升效率与准确性。
BitQ: Tailoring Block Floating Point Precision for Improved DNN Efficiency on Resource-Constrained Devices
- 基于块浮点设计可适配位宽的分析模型，自动优化精度分配。
- 相比等位宽设置，在主流基准上保持精度的同时显著降低计算开销。
- 适合资源受限设备上的深度神经网络部署，尤其关注能效比的场景。
深度神经网络（DNN）在图像分类、目标检测和场景分割等认知任务中表现强大，但其高计算复杂度和内存消耗使其难以在嵌入式平台实现实时运行。块浮点（BFP）量化是减少内存与计算负担的代表性压缩方法，因其能有效捕捉DNN模型中广泛的数据分布。然而，以往BFP量化方法通常经验性地选择块大小与精度。本文提出一种基于位宽感知的分析建模框架（称为“BitQ”），用于优化嵌入式平台上DNN推理的BFP实现。通过建立并求解一个兼顾精度与性能损失权衡的优化问题，确定最优的BFP块大小与位宽分布。实验结果表明，相较于等位宽设置，经优化位宽分配的BFP DNN在知名基准测试中实现了高效计算且保持了精度。代码与数据已开源。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) are powerful for cognitive tasks such as image classification, object detection, and scene segmentation. One drawback however is the significant high computational complexity and memory consumption, which makes them unfeasible to run real-time on embedded platforms because of the limited hardware resources. Block floating point (BFP) quantization is one of the representative compression approaches for reducing the memory and computational burden owing to their capability to effectively capture the broad data distribution of DNN models. Unfortunately, prior works on BFP-based quantization empirically choose the block size and the precision that preserve accuracy. In this paper, we develop a BFP-based bitwidth-aware analytical modeling framework (called ``BitQ'') for the best BFP implementation of DNN inference on embedded platforms. We formulate and resolve an optimization problem to identify the optimal BFP block size and bitwidth distribution by the trade-off of both accuracy and performance loss. Experimental results show that compared with an equal bitwidth setting, the BFP DNNs with optimized bitwidth allocation provide efficient computation, preserving accuracy on famous benchmarks. The source code and data are available at https://github.com/Cheliosoops/BitQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。