arXiv:2411.05858cs.CVcs.LG2024-11被引 5

用显著性辅助量化,在压缩模型时保持可解释性。

Saliency Assisted Quantization for Neural Networks

  • 训练中引入显著性引导,让模型聚焦关键输入特征。
  • 2/4位量化使准确率下降但显著图更清晰,混合配置平衡效果最佳。
  • 适合对透明度要求高的边缘设备部署场景。

深度学习在图像分类中已取得显著成效,但其决策过程的黑箱特性仍令专家担忧,且在资源受限环境下部署困难。本文通过在训练阶段引入实时解释机制,迫使模型关注输入中最显著和关键的部分,以缓解黑箱问题。同时采用参数化截断激活(Parameterized Clipping Activation)方法实现量化,以应对资源限制。在MNIST与FashionMNIST数据集上,评估了2位、4位及混合4/2位三种比特配置下的性能。结果表明,量化虽有助于效率提升,但会降低准确率与可解释性;比特越低,二者下降越明显。因此,在需要模型透明的应用中,量化参数的选择需格外谨慎。研究强调了效率与可解释性之间的平衡重要性。

原文摘要 · Abstract (English)

Deep learning methods have established a significant place in image classification. While prior research has focused on enhancing final outcomes, the opaque nature of the decision-making process in these models remains a concern for experts. Additionally, the deployment of these methods can be problematic in resource-limited environments. This paper tackles the inherent black-box nature of these models by providing real-time explanations during the training phase, compelling the model to concentrate on the most distinctive and crucial aspects of the input. Furthermore, we employ established quantization techniques to address resource constraints. To assess the effectiveness of our approach, we explore how quantization influences the interpretability and accuracy of Convolutional Neural Networks through a comparative analysis of saliency maps from standard and quantized models. Quantization is implemented during the training phase using the Parameterized Clipping Activation method, with a focus on the MNIST and FashionMNIST benchmark datasets. We evaluated three bit-width configurations (2-bit, 4-bit, and mixed 4/2-bit) to explore the trade-off between efficiency and interpretability, with each configuration designed to highlight varying impacts on saliency map clarity and model accuracy. The results indicate that while quantization is crucial for implementing models on resource-limited devices, it necessitates a trade-off between accuracy and interpretability. Lower bit-widths result in more pronounced reductions in both metrics, highlighting the necessity of meticulous quantization parameter selection in applications where model transparency is paramount. The study underscores the importance of achieving a balance between efficiency and interpretability in the deployment of neural networks.

量化可解释性神经网络边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。