用可解释概念指导剪枝,让大模型变小变快
Engineering Resource-constrained Software Systems with DNN Components: a Concept-based Pruning Approach
- 基于人类可懂的概念识别关键神经元,引导剪枝决策
- 在26384张图像上验证,剪枝后模型更小且性能保持良好
- 适合需在资源受限场景中部署AI系统的工程师使用
深度神经网络(DNN)被广泛用于解决需要从数据中进行预测建模的难题,但这些模型通常规模巨大,拥有数百万甚至数十亿参数,需要大量计算能力、内存和存储空间。这在实际应用中成为限制因素,尤其是在必须严格遵守尺寸和资源约束的情况下。本文提出一种新的基于概念的剪枝技术,通过人类可理解的概念(如特征、颜色、类别)指导剪枝决策。该方法分析神经元激活情况,从系统需求角度识别重要神经元,并据此指导剪枝过程。我们在VGG-19网络和包含26,384张RGB图像的数据集上评估了该方案,重点关注其生成小型、高效剪枝后DNN的能力,以及剪枝后模型的计算复杂度与性能表现。同时分析了不同配置下的剪枝效率。结果表明,基于概念的剪枝能有效生成显著更小且仍具高效率的模型,大幅提升计算效率与性能,特别适用于对内存和计算时间有严苛要求的实际应用。此外,多种配置选项帮助工程师根据具体场景权衡取舍。
原文摘要 · Abstract (English)
Deep Neural Networks (DNNs) are widely used by engineers to solve difficult problems that require predictive modeling from data. However, these models are often massive, with millions or billions of parameters, and require substantial computational power, RAM, and storage. This becomes a limitation in practical scenarios where strict size and resource constraints must be respected. In this paper, we present a novel concept-based pruning technique for DNNs that guides pruning decisions using human-interpretable concepts, such as features, colors, and classes. This is particularly important in a software engineering context, as DNNs are integrated into systems and must be pruned according to specific system requirements. Our concept-based pruning solution analyzes neuron activations to identify important neurons from a system requirements viewpoint and uses this information to guide the DNN pruning. We assess our solution using the VGG-19 network and a dataset of 26'384 RGB images, focusing on its ability to produce small, effective pruned DNNs and on the computational complexity and performance of these pruned DNNs. We also analyzed the pruning efficiency of our solution and compared alternative configurations. Our results show that concept-based pruning efficiently generates much smaller, effective pruned DNNs. Pruning greatly improves the computational efficiency and performance of DNNs, properties that are particularly useful for practical applications with stringent memory and computational time constraints. Finally, alternative configuration options enable engineers to identify trade-offs adapted to different practical situations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。