arXiv:2510.12939cs.LG2025-10被引 1

剪枝能提升强化学习的鲁棒性,且不会损害正常性能。

Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning

  • 理论证明剪枝可收紧对抗鲁棒性边界,永不降低鲁棒性。
  • 在中等稀疏度下,剪枝显著提升鲁棒性,且不损失甚至提升正常性能。
  • 适用于需要高鲁棒性的强化学习部署场景,如自动驾驶、机器人控制。

强化学习策略在真实环境部署时必须对对抗扰动保持可靠。然而现代深度强化学习智能体过度参数化,带来成本与脆弱性问题。尽管剪枝在监督学习中被证明可提升鲁棒性,其在对抗强化学习中的作用仍不明确。本文建立了首个针对状态对抗马尔可夫决策过程(SA-MDPs)中剪枝的认证鲁棒性理论框架。对于具有利普希茨网络的高斯与分类策略,我们证明逐元素剪枝只会收紧认证鲁棒性边界;剪枝绝不会使策略更脆弱。基于此,我们推导出一种新型三重损失分解,解耦了干净任务性能、剪枝导致的性能损失与鲁棒性增益,揭示了性能-鲁棒性的基本权衡前沿。在连续控制基准上,我们评估了幅度剪枝与微剪枝策略,面对强策略感知对抗者,剪枝在多个任务中稳定发现中等稀疏度的‘甜蜜点’,此时鲁棒性显著提升,且不损害——甚至有时增强——干净性能。结果表明,剪枝不仅是压缩工具,更是提升强化学习鲁棒性的结构性干预手段。

原文摘要 · Abstract (English)

Reinforcement learning (RL) policies deployed in real-world environments must remain reliable under adversarial perturbations. At the same time, modern deep RL agents are heavily over-parameterized, raising costs and fragility concerns. While pruning has been shown to improve robustness in supervised learning, its role in adversarial RL remains poorly understood. We develop the first theoretical framework for certified robustness under pruning in state-adversarial Markov decision processes (SA-MDPs). For Gaussian and categorical policies with Lipschitz networks, we prove that element-wise pruning can only tighten certified robustness bounds; pruning never makes the policy less robust. Building on this, we derive a novel three-term regret decomposition that disentangles clean-task performance, pruning-induced performance loss, and robustness gains, exposing a fundamental performance--robustness frontier. Empirically, we evaluate magnitude and micro-pruning schedules on continuous-control benchmarks with strong policy-aware adversaries. Across tasks, pruning consistently uncovers reproducible ``sweet spots'' at moderate sparsity levels, where robustness improves substantially without harming - and sometimes even enhancing - clean performance. These results position pruning not merely as a compression tool but as a structural intervention for robust RL.

强化学习剪枝鲁棒性结构优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。