通过梯度引导训练与量化结合,实现高效可解释的分类模型。
Quantized and Interpretable Learning Scheme for Deep Neural Networks in Classification Task
- 用梯度小的特征迭代屏蔽提升可解释性
- 在MNIST和CIFAR-10上保持准确率同时降低资源消耗
- 适合嵌入式设备等资源受限场景部署
深度学习在图像分类中表现优异,但在资源受限环境部署面临计算开销大、可解释性差的问题。本文提出将梯度引导训练与量化技术结合的方法,利用参数化截断激活(PACT)对权重和激活值进行量化感知训练,以优化精度并减少资源使用。同时,通过迭代屏蔽低梯度特征,生成更聚焦、有意义的显著性图,改善噪声梯度问题,提升模型决策过程的可解释性。在MNIST和CIFAR-10两个基准数据集上,使用经典卷积神经网络架构进行实验,对比标准模型与量化模型的显著性图,结果表明该方法在保持分类性能的同时,显著提升模型效率与可解释性,适用于资源受限场景的部署。
原文摘要 · Abstract (English)
Deep learning techniques have proven highly effective in image classification, but their deployment in resourceconstrained environments remains challenging due to high computational demands. Furthermore, their interpretability is of high importance which demands even more available resources. In this work, we introduce an approach that combines saliency-guided training with quantization techniques to create an interpretable and resource-efficient model without compromising accuracy. We utilize Parameterized Clipping Activation (PACT) to perform quantization-aware training, specifically targeting activations and weights to optimize precision while minimizing resource usage. Concurrently, saliency-guided training is employed to enhance interpretability by iteratively masking features with low gradient values, leading to more focused and meaningful saliency maps. This training procedure helps in mitigating noisy gradients and yields models that provide clearer, more interpretable insights into their decision-making processes. To evaluate the impact of our approach, we conduct experiments using famous Convolutional Neural Networks (CNN) architecture on the MNIST and CIFAR-10 benchmark datasets as two popular datasets. We compare the saliency maps generated by standard and quantized models to assess the influence of quantization on both interpretability and classification accuracy. Our results demonstrate that the combined use of saliency-guided training and PACT-based quantization not only maintains classification performance but also produces models that are significantly more efficient and interpretable, making them suitable for deployment in resource-limited settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。