用多智能体强化学习动态剪枝,让深度网络更轻更快
Dynamic Sensitivity Filter Pruning using Multi-Agent Reinforcement Learning For DCNN's
- 融合梯度、泰勒展开和激活分布差异,计算滤波器敏感度
- 70%剪枝率下保持98.23%准确率,浮点运算量降低超80%
- 单次前向传播完成剪枝,适合移动端高效部署
深度卷积神经网络在各类计算机视觉任务中达到顶尖性能,但其实际部署受限于计算与内存开销。本文提出差分敏感度融合剪枝(Differential Sensitivity Fusion Pruning),一种新型单次剪枝框架,通过融合梯度敏感度、一阶泰勒展开与激活分布的KL散度差异,评估滤波器重要性在多标准下的稳定性与冗余性。对每个滤波器计算差分敏感度,并采用指数缩放机制突出跨指标不一致的滤波器,识别结构不稳定或对模型性能贡献较小的候选。与迭代或强化学习剪枝方法不同,该方法高效且确定,仅需一次前向-反向传播完成评分与剪枝。在50%至70%剪枝率范围内的大量实验表明,该方法显著降低模型复杂度,在70%剪枝率下仍可保持98.23%基线准确率,浮点运算量减少超过80%,优于传统启发式方法在压缩与泛化能力上的表现。该方法为可扩展、自适应的深度卷积网络压缩提供了有效解决方案,推动其在边缘与移动平台的高效部署。
原文摘要 · Abstract (English)
Deep Convolutional Neural Networks have achieved state of the art performance across various computer vision tasks, however their practical deployment is limited by computational and memory overhead. This paper introduces Differential Sensitivity Fusion Pruning, a novel single shot filter pruning framework that focuses on evaluating the stability and redundancy of filter importance scores across multiple criteria. Differential Sensitivity Fusion Pruning computes a differential sensitivity score for each filter by fusing the discrepancies among gradient based sensitivity, first order Taylor expansion, and KL divergence of activation distributions. An exponential scaling mechanism is applied to emphasize filters with inconsistent importance across metrics, identifying candidates that are structurally unstable or less critical to the model performance. Unlike iterative or reinforcement learning based pruning strategies, Differential Sensitivity Fusion Pruning is efficient and deterministic, requiring only a single forward-backward pass for scoring and pruning. Extensive experiments across varying pruning rates between 50 to 70 percent demonstrate that Differential Sensitivity Fusion Pruning significantly reduces model complexity, achieving over 80 percent Floating point Operations Per Seconds reduction while maintaining high accuracy. For instance, at 70 percent pruning, our approach retains up to 98.23 percent of baseline accuracy, surpassing traditional heuristics in both compression and generalization. The proposed method presents an effective solution for scalable and adaptive Deep Convolutional Neural Networks compression, paving the way for efficient deployment on edge and mobile platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。