用安全约束剪枝量化,模型缩小60%还更准
Optimizing Deep Neural Networks using Safety-Guided Self Compression
- 基于保留集设计安全驱动的压缩策略
- 压缩后模型保持原大小60%,准确率提升2.5%
- 适合部署在资源受限设备的高可靠模型优化
将深度神经网络部署在资源受限设备上,需要在减小模型规模的同时保持性能。本文提出一种新型安全驱动量化框架,利用保留集系统性地剪枝和量化网络权重,从而在不降低精度的前提下优化模型复杂度。该方法在卷积神经网络(CNN)和基于注意力的语言模型上进行了严格评估,验证了其在不同架构上的适用性。实验结果表明,该框架在保持初始模型60%大小的情况下,测试准确率相比原始未量化模型最高提升2.5%。与传统量化方法相比,该方法通过消除参数噪声并保留关键权重,不仅增强了泛化能力,还降低了方差,确保关键模型特征得以保留。这些发现证明了安全驱动量化作为高效、可靠的深度学习模型优化策略的有效性。框架实现及完整实验评估已公开于GitHub。
原文摘要 · Abstract (English)
The deployment of deep neural networks on resource-constrained devices necessitates effective model com- pression strategies that judiciously balance the reduction of model size with the preservation of performance. This study introduces a novel safety-driven quantization framework that leverages preservation sets to systematically prune and quantize neural network weights, thereby optimizing model complexity without compromising accuracy. The proposed methodology is rigorously evaluated on both a convolutional neural network (CNN) and an attention-based language model, demonstrating its applicability across diverse architectural paradigms. Experimental results reveal that our framework achieves up to a 2.5% enhancement in test accuracy relative to the original unquantized models while maintaining 60% of the initial model size. In comparison to conventional quantization techniques, our approach not only augments generalization by eliminating parameter noise and retaining essential weights but also reduces variance, thereby ensuring the retention of critical model features. These findings underscore the efficacy of safety-driven quantization as a robust and reliable strategy for the efficient optimization of deep learn- ing models. The implementation and comprehensive experimental evaluations of our framework are publicly accessible at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。