arXiv:2510.00062cs.CVcs.AI2025-10

通过特征图相似性实现高效卷积神经网络压缩。

Efficient CNN Compression via Multi-method Low Rank Factorization and Feature Map Similarity

  • 基于特征图相似性自动选秩,替代传统权重方法。
  • 单次微调完成压缩,训练时间大幅缩短。
  • 支持多种分解方式按层选择,适合工程部署。

低秩分解(LRF)是压缩深度神经网络的常用技术,但面临最优秩选择、设计空间庞大、微调时间长以及对不同层类型和分解方法兼容性差等挑战。本文提出一种端到端的设计空间探索(DSE)方法与框架,全面解决上述问题。我们引入基于特征图相似性的新型秩选择策略,更有效捕捉层输出间的非线性交互;相比以往方法,采用一次性的微调过程,显著降低整体微调时间。该框架完全兼容所有类型的卷积层(Conv)和全连接层(FC)。为提升压缩效果,框架在卷积层中集成三种不同的LRF技术,在全连接层中也集成三种,按层选择性应用。实验表明,将多种LRF方法结合使用比单一方法均匀应用更优。最后,我们对六种LRF技术进行了全面评估,提供了不同场景下的有效性洞察。该工作已集成至TensorFlow 2.x,确保与主流深度学习流程兼容。在8个数据集上的14个CNN模型实验显示,所提方法实现显著压缩且精度损失极小,优于多个现有先进方法。

原文摘要 · Abstract (English)

Low-Rank Factorization (LRF) is a widely adopted technique for compressing deep neural networks (DNNs). However, it faces several challenges, including optimal rank selection, a vast design space, long fine-tuning times, and limited compatibility with different layer types and decomposition methods. This paper presents an end-to-end Design Space Exploration (DSE) methodology and framework for compressing convolutional neural networks (CNNs) that addresses all these issues. We introduce a novel rank selection strategy based on feature map similarity, which captures non-linear interactions between layer outputs more effectively than traditional weight-based approaches. Unlike prior works, our method uses a one-shot fine-tuning process, significantly reducing the overall fine-tuning time. The proposed framework is fully compatible with all types of convolutional (Conv) and fully connected (FC) layers. To further improve compression, the framework integrates three different LRF techniques for Conv layers and three for FC layers, applying them selectively on a per-layer basis. We demonstrate that combining multiple LRF methods within a single model yields better compression results than using a single method uniformly across all layers. Finally, we provide a comprehensive evaluation and comparison of the six LRF techniques, offering practical insights into their effectiveness across different scenarios. The proposed work is integrated into TensorFlow 2.x, ensuring compatibility with widely used deep learning workflows. Experimental results on 14 CNN models across eight datasets demonstrate that the proposed methodology achieves substantial compression with minimal accuracy loss, outperforming several state-of-the-art techniques.

模型压缩低秩分解卷积网络高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。