arXiv:2503.20613cs.LGcs.AI2025-03被引 12

针对强化学习对扰动敏感问题,提出状态感知的自适应攻击方法提升鲁棒性评估效果。

State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning

  • 基于状态目标掩码减少冗余扰动,增强攻击隐蔽性
  • 通过最大化扰动与状态动作间互信息,使智能体访问脆弱状态
  • 适用于评估机器人控制中DRL模型的抗干扰能力

深度强化学习(DRL)在机器人控制中展现出巨大潜力,但其在真实环境部署中受限于对环境扰动的敏感性。现有白盒对抗攻击依赖局部梯度信息,在所有状态上施加统一扰动,忽视了时序动态和状态特异性脆弱性。本文首次建立对抗受害者动态马尔可夫决策过程(AVD-MDP),推导出成功攻击的充要条件。据此提出选择性状态感知强化对抗攻击方法(STAR),优化扰动隐蔽性和状态访问分布。STAR首先采用基于软掩码的状态目标机制,最小化冗余扰动,提升隐蔽性与攻击效果;随后引入信息论优化目标,最大化扰动、环境状态与受攻击动作间的互信息,确保状态访问分布分散,引导智能体进入脆弱状态以实现最大回报降低。大量实验表明,STAR显著优于现有基准方法。

原文摘要 · Abstract (English)

Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing whitebox adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the adversarial victim-dynamics Markov decision process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose a selective state-aware reinforcement adversarial attack method, named STAR, to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks.

强化学习对抗攻击鲁棒性评估状态感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。