提出端到端混合精度量化框架，提升边缘设备模型效率与精度
MiCo: End-to-End Mixed Precision Neural Network Co-Exploration Framework for Edge AI
- 设计新优化算法，自动搜索满足延迟约束的最优混合精度方案
- 构建硬件感知延迟模型，实现快速精准的量化方案探索
- 支持从PyTorch直接部署到裸金属C代码，降低精度损失并加速推理
极低比特位宽的量化神经网络（QNN）在边缘设备上展现出高效的存储与计算潜力。为在进一步提升速度的同时减少精度损失，层级混合精度量化（MPQ）成为主流方案。然而，现有方法在灵活性与效率方面存在局限，难以全面理解不同MPQ方案对后训练量化和量化感知训练的影响。此外，缺乏端到端的MPQ模型优化与部署框架。本文提出MiCo框架，一个面向边缘AI应用的完整MPQ探索与部署系统。该框架采用新型优化算法，在满足延迟约束的前提下搜索最高精度的量化方案；构建针对不同硬件的目标感知延迟模型，实现快速探索；探索完成后，可直接将PyTorch中的MPQ模型转换为裸金属C代码，实现端到端加速且精度损失极小。
原文摘要 · Abstract (English)
Quantized Neural Networks (QNN) with extremely low-bitwidth data have proven promising in efficient storage and computation on edge devices. To further reduce the accuracy drop while increasing speedup, layer-wise mixed-precision quantization (MPQ) becomes a popular solution. However, existing algorithms for exploring MPQ schemes are limited in flexibility and efficiency. Comprehending the complex impacts of different MPQ schemes on post-training quantization and quantization-aware training results is a challenge for conventional methods. Furthermore, an end-to-end framework for the optimization and deployment of MPQ models is missing in existing work. In this paper, we propose the MiCo framework, a holistic MPQ exploration and deployment framework for edge AI applications. The framework adopts a novel optimization algorithm to search for optimal quantization schemes with the highest accuracies while meeting latency constraints. Hardware-aware latency models are built for different hardware targets to enable fast explorations. After the exploration, the framework enables direct deployment from PyTorch MPQ models to bare-metal C codes, leading to end-to-end speedup with minimal accuracy drops.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。