从策略分布角度改进强化学习对抗攻击，提升对连续动作空间的破坏力。
Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective
- 基于策略分布相似性设计梯度扰动，不再依赖单个动作采样。
- 在三个机器人导航任务中平均导致22.03%更高奖励下降，优于现有方法。
- 适合关注DRL鲁棒性评估与对抗防御的研究者使用。
深度强化学习（DRL）在真实应用中常面临观测信号不确定性和不准确性的问题。对抗攻击是评估DRL智能体鲁棒性的有效手段。然而，现有针对单一采样动作的攻击方法对整体策略分布影响有限，尤其在连续动作空间中表现不足。为此，我们提出分布感知投影梯度下降攻击（DAPGD）。DAPGD利用策略分布相似性作为梯度扰动输入，通过整个策略分布而非单个样本进行攻击。我们采用巴塔查里亚距离（Bhattacharyya distance）衡量策略相似性，可敏感检测概率分布间的细微但关键差异。实验结果表明，DAPGD在三个机器人导航任务中达到最优性能，相比最佳基线平均奖励下降幅度提升22.03%。
原文摘要 · Abstract (English)
Deep Reinforcement Learning (DRL) suffers from uncertainties and inaccuracies in the observation signal in realworld applications. Adversarial attack is an effective method for evaluating the robustness of DRL agents. However, existing attack methods targeting individual sampled actions have limited impacts on the overall policy distribution, particularly in continuous action spaces. To address these limitations, we propose the Distribution-Aware Projected Gradient Descent attack (DAPGD). DAPGD uses distribution similarity as the gradient perturbation input to attack the policy network, which leverages the entire policy distribution rather than relying on individual samples. We utilize the Bhattacharyya distance in DAPGD to measure policy similarity, enabling sensitive detection of subtle but critical differences between probability distributions. Our experiment results demonstrate that DAPGD achieves SOTA results compared to the baselines in three robot navigation tasks, achieving an average 22.03% higher reward drop compared to the best baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。