arXiv:2509.10526cs.LGcs.AI2025-09被引 1

用图神经网络自动剪枝,精准识别冗余连接。

Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning

  • 构建网络拓扑图,全局捕捉层与通道关系。
  • 在CIFAR-10/100/ImageNet上优于传统方法,实现更优压缩率与精度平衡。
  • 适合需要高效模型部署的研究者与工程师。

本文提出一种新型神经网络剪枝方法,将图结构观测空间引入自动化机器学习框架,以克服现有方法依赖人工启发式和局部优化的局限性。通过构建捕捉层间与通道间完整拓扑关系的网络图表示,替代传统的逐层观测空间,实现对网络结构的全局视图。核心创新包括采用图注意力网络(GAT)编码器生成丰富嵌入,并将动作空间从连续剪枝比例改为细粒度二元动作,使智能体能直接从数据中学习通道重要性标准,摆脱预定义评分函数。上述设计在约束马尔可夫决策过程(CMDP)框架下建模,确保剪枝决策满足目标压缩率等资源约束。为此,我们设计自竞争奖励机制,激励智能体持续超越自身历史最佳性能。在CIFAR-10、CIFAR-100和ImageNet等基准数据集上的大量实验表明,该方法显著优于传统剪枝技术,在保持高精度的同时实现最先进的压缩效果,能发现超越权重大小考量的功能冗余连接。

原文摘要 · Abstract (English)

This paper presents a novel approach to neural network pruning by integrating a graph-based observation space into an AutoML framework to address the limitations of existing methods. Traditional pruning approaches often depend on hand-crafted heuristics and local optimization perspectives, which can lead to suboptimal performance and inefficient pruning strategies. Our framework transforms the pruning process by introducing a graph representation of the target neural network that captures complete topological relationships between layers and channels, replacing the limited layer-wise observation space with a global view of network structure. The core innovations include a Graph Attention Network (GAT) encoder that processes the network's graph representation and generates a rich embedding. Additionally, for the action space we transition from continuous pruning ratios to fine-grained binary action spaces which enables the agent to learn optimal channel importance criteria directly from data, moving away from predefined scoring functions. These contributions are modelled within a Constrained Markov Decision Process (CMDP) framework, allowing the agent to make informed pruning decisions while adhering to resource constraints such as target compression rates. For this, we design a self-competition reward system that encourages the agent to outperform its previous best performance while satisfying the defined constraints. We demonstrate the effectiveness of our approach through extensive experiments on benchmark datasets including CIFAR-10, CIFAR-100, and ImageNet. The experiments show that our method consistently outperforms traditional pruning techniques, showing state-of-the-art results while learning task-specific pruning strategies that identify functionally redundant connections beyond simple weight magnitude considerations.

神经网络剪枝图神经网络AutoML模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。