用梯度信息动态评估神经网络组件重要性，提升压缩精度
Component-Aware Pruning Framework for Neural Network Controllers via Gradient-Based Importance Estimation
- 训练中结合梯度累积、费舍尔信息与贝叶斯不确定性计算重要性
- 在自编码器和TD-MPC上验证，能发现静态方法忽略的结构依赖关系
- 适合需要精细压缩的复杂神经控制器设计者
先进神经网络控制器从单体架构向多组件架构演进，带来显著计算开销。传统基于范数的结构化剪枝方法常无法捕捉参数组的功能重要性。本文提出一种组件感知剪枝框架，利用训练过程中的梯度信息，计算三种重要性度量：梯度累积、费舍尔信息与贝叶斯不确定性。实验在自编码器与TD-MPC代理上验证，结果表明该框架可揭示静态启发式方法忽略的关键结构依赖关系及重要性动态变化，支持更精准的压缩决策。
原文摘要 · Abstract (English)
The transition from monolithic to multi-component neural architectures in advanced neural network controllers poses substantial challenges due to the high computational complexity of the latter. Conventional model compression techniques for complexity reduction, such as structured pruning based on norm-based metrics to estimate the relative importance of distinct parameter groups, often fail to capture functional significance. This paper introduces a component-aware pruning framework that utilizes gradient information to compute three distinct importance metrics during training: Gradient Accumulation, Fisher Information, and Bayesian Uncertainty. Experimental results with an autoencoder and a TD-MPC agent demonstrate that the proposed framework reveals critical structural dependencies and dynamic shifts in importance that static heuristics often miss, supporting more informed compression decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。