通过滤波分析神经网络参数，发现强化学习中存在抗压增强的参数。
Parameter Stress Analysis in Reinforcement Learning: Applying Synaptic Filtering to Policy Networks
- 用高低通和脉冲滤波器模拟内部扰动，分析策略网络参数敏感性。
- 在Mujoco环境中验证，部分参数在压力下反而提升性能。
- 为设计更鲁棒的强化学习系统提供新思路，适合关注模型可靠性研究者。
本文通过系统分析强化学习（RL）策略网络在内外部压力下的参数表现，探索其鲁棒性。采用来自文献[pravin2024fragility]的高通、低通和脉冲波滤波方法作为内部应力，选择性扰动网络参数；同时，对抗攻击作为外部应力，通过修改智能体观测值施加影响。该双重策略使参数可被分类为脆弱、稳健或抗脆弱，依据其在干净与对抗环境中的表现。定义参数评分以量化上述特性，并在基于近端策略优化（PPO）训练的Mujoco连续控制环境中验证框架有效性。结果表明，存在抗脆弱参数，在压力下能提升策略性能，证明定向滤波技术可增强强化学习策略的适应能力。这些发现为未来构建鲁棒且抗脆弱的强化学习系统奠定基础。
原文摘要 · Abstract (English)
This paper explores reinforcement learning (RL) policy robustness by systematically analyzing network parameters under internal and external stresses. \textcolor{black}{We apply synaptic filtering methods using high-pass, low-pass, and pulse-wave filters from} \citep{pravin2024fragility}, as an internal stress by selectively perturbing parameters, while adversarial attacks apply external stress through modified agent observations. This dual approach enables the classification of parameters as \textit{fragile}, \textit{robust}, or \textit{antifragile}, based on their influence on policy performance in clean and adversarial settings. Parameter scores are defined to quantify these characteristics, and the framework is validated on proximal policy optimization (PPO)-trained agents in Mujoco continuous control environments. The results highlight the presence of antifragile parameters that enhance policy performance under stress, demonstrating the potential of targeted filtering techniques to improve RL policy adaptability. These insights provide a foundation for future advancements in the design of robust and antifragile RL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。