用可解释AI识别重要参数,实现高效模型压缩且精度提升
Compressing Deep Neural Networks Using Explainable AI
- 基于LRP方法计算权重重要性,指导剪枝与量化
- 压缩后模型体积减64%,精度反而比当前最佳方法高42%
- 适合需要小模型且保持高性能的边缘设备应用
深度神经网络(DNN)在众多任务中表现优异,但伴随高计算成本和内存占用。为降低资源消耗,常采用剪枝和量化等压缩技术。近年来,可解释人工智能(XAI)被用于理解DNN内部机制,如各神经元和特征的重要性。本文提出一种基于XAI的新压缩方法:使用基于梯度的层间相关传播(LRP)计算权重重要性得分,随后进行剪枝与混合精度量化——重要性低(负或零)的参数被移除,重要性高的参数用更多比特表示,反之则减少比特。实验表明,该方法在保持性能的同时使模型大小减少64%,相比现有最优的XAI压缩方法,准确率反而提升了42%。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) have demonstrated remarkable performance in many tasks but it often comes at a high computational cost and memory usage. Compression techniques, such as pruning and quantization, are applied to reduce the memory footprint of DNNs and make it possible to accommodate them on resource-constrained edge devices. Recently, explainable artificial intelligence (XAI) methods have been introduced with the purpose of understanding and explaining AI methods. XAI can be utilized to get to know the inner functioning of DNNs, such as the importance of different neurons and features in the overall performance of DNNs. In this paper, a novel DNN compression approach using XAI is proposed to efficiently reduce the DNN model size with negligible accuracy loss. In the proposed approach, the importance score of DNN parameters (i.e. weights) are computed using a gradient-based XAI technique called Layer-wise Relevance Propagation (LRP). Then, the scores are used to compress the DNN as follows: 1) the parameters with the negative or zero importance scores are pruned and removed from the model, 2) mixed-precision quantization is applied to quantize the weights with higher/lower score with higher/lower number of bits. The experimental results show that, the proposed compression approach reduces the model size by 64% while the accuracy is improved by 42% compared to the state-of-the-art XAI-based compression method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。