用非负矩阵分解一次剪枝，训练中精准保留稀疏结构。
ONG: One-Shot NMF-based Gradient Masking for Efficient Model Sparsification
- 基于NMF在训练初期一次性识别重要权重结构
- 通过梯度掩码确保仅保留权重更新，严格维持稀疏度
- 适合追求高效剪枝且需稳定稀疏性的模型部署场景
深度神经网络虽取得显著成功，但其庞大体量带来部署挑战。现有剪枝方法多依赖复杂迭代过程、专用标准，或难以在训练中有效保持稀疏性。本文提出ONG（One-shot NMF-based Gradient Masking），一种新型稀疏化策略：利用非负矩阵分解（NMF）在训练起始阶段一次性识别关键权重结构，并通过精确的梯度掩码机制，确保仅未剪枝权重参与更新，从而在整个训练过程中严格维持目标稀疏度。我们将ONG集成至BIMP对比框架，在CIFAR-10和CIFAR-100上使用ResNet56、ResNet34与ResNet18进行评估，结果表明，ONG在多种稀疏度下均能实现相当或更优性能，且剪枝后结构完整性良好，具备明确的目标稀疏度控制机制。
原文摘要 · Abstract (English)
Deep Neural Networks (DNNs) have achieved remarkable success but their large size poses deployment challenges. While various pruning techniques exist, many involve complex iterative processes, specialized criteria, or struggle to maintain sparsity effectively during training. We introduce ONG (One-shot NMF-based Gradient Masking), a novel sparsification strategy that identifies salient weight structures using Non-negative Matrix Factorization (NMF) for one-shot pruning at the outset of training. Subsequently, ONG employs a precise gradient masking mechanism to ensure that only unpruned weights are updated, strictly preserving the target sparsity throughout the training phase. We integrate ONG into the BIMP comparative framework and evaluate it on CIFAR-10 and CIFAR-100 with ResNet56, ResNet34, and ResNet18 against established stable sparsification methods. Our experiments demonstrate ONG's ability to achieve comparable or superior performance at various sparsity levels while maintaining structural integrity post-pruning and offering a clear mechanism for targeting desired sparsities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。