arXiv:2503.20844cs.LGcs.AI2025-03被引 16

针对机器人强化学习的脆弱性,提出动态扰动关键状态的攻击方法。

Robust Deep Reinforcement Learning in Robotics via Adaptive Gradient-Masked Adversarial Attacks

  • 基于梯度软掩码,动态识别影响最大的状态维度进行针对性扰动。
  • 在多个机器人环境中使目标智能体性能下降超40%,优于现有方法。
  • 适合研究强化学习鲁棒性或防御机制的研究者参考。

深度强化学习(DRL)在机器人控制中展现出巨大潜力,但其在真实场景部署时仍面临环境扰动带来的脆弱性挑战。现有白盒对抗攻击方法源自监督学习,忽视了时序动态性,且对所有状态维度进行无差别扰动,难以有效影响长期奖励。为此,本文提出自适应梯度掩码强化学习(AGMR)攻击方法,将DRL与基于梯度的软掩码机制结合,动态识别关键状态维度,并优化对抗策略。AGMR选择性地将扰动集中在最具影响力的特征上,同时引入动态调节机制平衡训练中的探索与利用。大量实验表明,AGMR在降低目标智能体性能方面显著优于现有先进攻击方法,并通过对抗防御机制提升其鲁棒性。

原文摘要 · Abstract (English)

Deep reinforcement learning (DRL) has emerged as a promising approach for robotic control, but its realworld deployment remains challenging due to its vulnerability to environmental perturbations. Existing white-box adversarial attack methods, adapted from supervised learning, fail to effectively target DRL agents as they overlook temporal dynamics and indiscriminately perturb all state dimensions, limiting their impact on long-term rewards. To address these challenges, we propose the Adaptive Gradient-Masked Reinforcement (AGMR) Attack, a white-box attack method that combines DRL with a gradient-based soft masking mechanism to dynamically identify critical state dimensions and optimize adversarial policies. AGMR selectively allocates perturbations to the most impactful state features and incorporates a dynamic adjustment mechanism to balance exploration and exploitation during training. Extensive experiments demonstrate that AGMR outperforms state-of-the-art adversarial attack methods in degrading the performance of the victim agent and enhances the victim agent's robustness through adversarial defense mechanisms.

强化学习对抗攻击机器人鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。