提出新防御框架，提升强化学习在环境状态扰动下的鲁棒性
Towards Robust Deep Reinforcement Learning against Environmental State Perturbation
- 先用监督学习预训练，再结合对抗训练增强抗扰能力
- 实验表明主流算法在状态扰动下易失效，新方法显著提升鲁棒性
- 适合需要稳定运行的物理模拟或机器人任务场景
深度强化学习中的对抗攻击与鲁棒性已得到广泛研究，但多数工作未考虑具身场景中常见的环境状态扰动。为提升DRL代理的鲁棒性，本文定义了环境状态扰动问题，提出一种初步的非目标攻击方法作为校准对抗器，并设计名为增强对抗训练（BAT）的防御框架：首先通过监督学习调优代理以避免灾难性失败，随后在强化学习中进行对抗训练。大量实验验证了主流代理在环境状态扰动下的脆弱性及所提攻击的有效性。防御结果表明，尽管现有鲁棒强化学习算法不适用，但本框架能显著提升代理在多种情境下对环境状态扰动的鲁棒性。
原文摘要 · Abstract (English)
Adversarial attacks and robustness in Deep Reinforcement Learning (DRL) have been widely studied in various threat models; however, few consider environmental state perturbations, which are natural in embodied scenarios. To improve the robustness of DRL agents, we formulate the problem of environmental state perturbation, introducing a preliminary non-targeted attack method as a calibration adversary, and then propose a defense framework, named Boosted Adversarial Training (BAT), which first tunes the agents via supervised learning to avoid catastrophic failure and subsequently adversarially trains the agent with reinforcement learning. Extensive experimental results substantiate the vulnerability of mainstream agents under environmental state perturbations and the effectiveness of our proposed attack. The defense results demonstrate that while existing robust reinforcement learning algorithms may not be suitable, our BAT framework can significantly enhance the robustness of agents against environmental state perturbations across various situations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。