首次为神经控制网络的二阶剪枝提供可验证的稳定性保障。
Closed-Form Robustness Bounds for Second-Order Pruning of Neural Controller Policies
- 通过局部海森矩阵实现单次剪枝,提升压缩率
- 推导出剪枝后控制误差的闭式上界,可预估最大允许剪枝量
- 适用于对安全性要求高的嵌入式机器人系统
深度神经控制策略已实现四旋翼的敏捷飞行、机械臂的自适应抓取及地面机器人的可靠导航,但其数百万参数与嵌入式微控制器的内存和实时性约束相冲突。二阶剪枝方法(如最优大脑损伤OBD及其变体)利用局部海森矩阵,在单次遍历中实现远高于幅度阈值法的稀疏度。尽管在视觉与语言任务中表现优异,其对闭环稳定性、跟踪精度与安全性的潜在影响仍不明确。本文首次对非线性离散时间控制中的二阶剪枝进行严格数学分析。系统在连续转移映射下演化,控制器为具有ReLU激活函数的L层多层感知机,其全局1-利普希茨连续。剪枝第k层权重矩阵$W_k$,替换为$W_k+δW_k$,得到扰动参数向量$Θ̂=Θ+δΘ$与剪枝后的策略$π(·;Θ̂)$。对任意输入状态$s∈ X$,我们推导出闭式不等式:$\|π(s;Θ)-π(s;Θ̂)\|_2 \le C_k(s)\,\|δW_k\|_2$,其中常数$C_k(s)$仅依赖于未剪枝的谱范数与偏置,可通过一次前向传播闭式计算。该边界可在部署前确定与指定控制误差阈值兼容的最大可接受剪枝幅度。本工作将二阶网络压缩与闭环性能保证联系起来,弥合了现代深度学习工具与安全关键自主系统鲁棒性需求之间的关键鸿沟。
原文摘要 · Abstract (English)
Deep neural policies have unlocked agile flight for quadcopters, adaptive grasping for manipulators, and reliable navigation for ground robots, yet their millions of weights conflict with the tight memory and real-time constraints of embedded microcontrollers. Second-order pruning methods, such as Optimal Brain Damage (OBD) and its variants, including Optimal Brain Surgeon (OBS) and the recent SparseGPT, compress networks in a single pass by leveraging the local Hessian, achieving far higher sparsity than magnitude thresholding. Despite their success in vision and language, the consequences of such weight removal on closed-loop stability, tracking accuracy, and safety have remained unclear. We present the first mathematically rigorous robustness analysis of second-order pruning in nonlinear discrete-time control. The system evolves under a continuous transition map, while the controller is an $L$-layer multilayer perceptron with ReLU-type activations that are globally 1-Lipschitz. Pruning the weight matrix of layer $k$ replaces $W_k$ with $W_k+δW_k$, producing the perturbed parameter vector $\widehatΘ=Θ+δΘ$ and the pruned policy $π(\cdot;\widehatΘ)$. For every input state $s\in X$ we derive the closed-form inequality $ \|π(s;Θ)-π(s;\widehatΘ)\|_2 \le C_k(s)\,\|δW_k\|_2, $ where the constant $C_k(s)$ depends only on unpruned spectral norms and biases, and can be evaluated in closed form from a single forward pass. The derived bounds specify, prior to field deployment, the maximal admissible pruning magnitude compatible with a prescribed control-error threshold. By linking second-order network compression with closed-loop performance guarantees, our work narrows a crucial gap between modern deep-learning tooling and the robustness demands of safety-critical autonomous systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。