针对控制器神经网络,提出可保留关键性能的结构化剪枝方法。
Application-Specific Component-Aware Structured Pruning of Deep Neural Networks in Control via Soft Coefficient Optimization
- 基于软系数优化计算剪枝重要性,兼顾压缩与性能。
- 在MNIST自编码器和TDMPC控制任务中实现显著瘦身且性能不降。
- 适合对可靠性要求高的工业控制类DNN部署场景。
深度神经网络(DNN)具有高度灵活性和鲁棒性,适用于构建系统模型及神经网络控制器(NNC)。然而其高复杂度和计算需求限制了实际应用。尽管已有多种模型压缩策略,但它们多针对通用DNN,难以直接应用于需要同时缩减规模并保留特定应用性能的NNC。传统结构化剪枝依赖通用重要性度量,常破坏关键性能特征。本文提出一种新型剪枝重要性计算框架,不仅减小模型尺寸,还考虑多种应用特定约束。通过网格搜索与梯度下降优化两种方式寻找最优剪枝系数。在两个用例中验证:一是MNIST自编码器,二是时序差分模型预测控制(TDMPC)代理。结果表明,该方法在显著压缩模型的同时,有效维持了应用相关性能。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) offer significant flexibility and robust performance. This makes them ideal for building not only system models but also advanced neural network controllers (NNCs). However, their high complexity and computational needs often limit their use. Various model compression strategies have been developed over the past few decades to address these issues. These strategies are effective for general DNNs but do not directly apply to NNCs. NNCs need both size reduction and the retention of key application-specific performance features. In structured pruning, which removes groups of related elements, standard importance metrics often fail to protect these critical characteristics. In this paper, we introduce a novel framework for calculating importance metrics in pruning groups. This framework not only shrinks the model size but also considers various application-specific constraints. To find the best pruning coefficient for each group, we evaluate two approaches. The first approach involves simple exploration through grid search. The second utilizes gradient descent optimization, aiming to balance compression and task performance. We test our method in two use cases: one on an MNIST autoencoder and the other on a Temporal Difference Model Predictive Control (TDMPC) agent. Results show that the method effectively maintains application-relevant performance while achieving a significant reduction in model size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。