HQP框架通过感知敏感度的剪枝与量化协同压缩,实现边缘设备超低延迟推理。
HQP: Sensitivity-Aware Hybrid Quantization and Pruning for Ultra-Low-Latency Edge AI Inference
- 基于FIM近似动态评估权重敏感度,迭代剪枝冗余卷积核
- 剪枝后模型精度下降控制在1.5%以内,实现3.12倍加速和55%模型缩小
- 适配NVIDIA Jetson等边缘平台,适合资源受限的实时AI部署
分布式边缘-云环境中对高保真、实时推理的需求日益增长,亟需激进的模型优化以应对严重的延迟与能耗限制。本文提出混合量化与剪枝(HQP)框架,一种新型集成方法,在严格质量约束下实现协同加速。我们设计了一种感知敏感度的结构化剪枝算法,利用高效近似Fisher信息矩阵(FIM)计算动态权重敏感度,指导冗余滤波器的迭代移除。该剪枝过程受严格条件约束,仅在精度下降不超过允许阈值Δax时才进入8位后训练量化阶段。此严格协调至关重要,确保稀疏模型结构对量化误差及硬件特定内核优化具有最大鲁棒性。在异构NVIDIA Jetson边缘平台上的全面评估显示,使用MobileNetV3和ResNet-18等轻量架构,HQP框架实现最高3.12倍的推理加速与55%的模型尺寸缩减,同时将精度损失严格控制在1.5%以内。与传统单目标压缩技术的对比分析验证了HQP作为硬件无关的优越解决方案,适用于资源受限边缘基础设施中的超低延迟AI部署。
原文摘要 · Abstract (English)
The escalating demand for high-fidelity, real-time inference in distributed edge-cloud environments necessitates aggressive model optimization to counteract severe latency and energy constraints. This paper introduces the Hybrid Quantization and Pruning (HQP) framework, a novel, integrated methodology designed to achieve synergistic model acceleration while adhering to strict quality guarantees. We detail a sensitivity-aware structural pruning algorithm that employs a dynamic weight sensitivity metric, derived from a highly efficient approximation of the Fisher Information Matrix (FIM), to guide the iterative removal of redundant filters. This pruning is strictly conditional, enforcing an adherence to a maximum permissible accuracy drop (Delta ax) before the model proceeds to 8-bit post-training quantization. This rigorous coordination is critical, as it ensures the resultant sparse model structure is maximally robust to quantization error and hardware-specific kernel optimization. Exhaustive evaluation across heterogeneous NVIDIA Jetson edge platforms, utilizing resource-efficient architectures like MobileNetV3 and ResNet-18, demonstrates that the HQP framework achieves a peak performance gain of 3.12 times inference speedup and a 55 percent model size reduction, while rigorously containing the accuracy drop below the 1.5 percent constraint. A comprehensive comparative analysis against conventional single-objective compression techniques validates the HQP framework as a superior, hardware-agnostic solution for deploying ultra-low-latency AI in resource-limited edge infrastructures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。