arXiv:2501.15529cs.LGcs.AI2025-01被引 8

提出首个通用动作级后门攻击框架,自适应调整奖励函数提升攻击成功率。

UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning

  • 通过性能监控动态优化后门奖励函数,无需人工调参。
  • 在多种环境和动作空间中攻击成功率超90%且稳定生效。
  • 适合研究后门安全的开发者,尤其关注连续动作场景的攻防设计。

深度强化学习(DRL)广泛应用于安全关键决策场景,但易受后门攻击威胁,尤其是动作级后门攻击。这类攻击通过特定奖励函数将触发器与目标动作关联,实现精准操控,可能导致车辆碰撞或无人机坠毁。现有方法依赖固定值或条件翻转的奖励函数,在不同任务中缺乏通用性,常导致攻击失败。本文提出首个通用动作级后门攻击框架UNIDOOR,通过性能监控自适应探索后门奖励函数,消除对专家知识和网格搜索的依赖。研究发现,在连续动作场景中,动作篡改是攻击成功的关键,可缓解低频目标动作导致的失效问题。大量实验表明,UNIDOOR在单/多智能体、单/多后门、离散/连续动作空间及稀疏/密集奖励信号下均表现优异,攻击成功率普遍超过90%。可视化结果(状态分布、神经元激活、动画)证实其隐蔽性。代码已开源:https://github.com/maoubo/UNIDOOR。

原文摘要 · Abstract (English)

Deep reinforcement learning (DRL) is widely applied to safety-critical decision-making scenarios. However, DRL is vulnerable to backdoor attacks, especially action-level backdoors, which pose significant threats through precise manipulation and flexible activation, risking outcomes like vehicle collisions or drone crashes. The key distinction of action-level backdoors lies in the utilization of the backdoor reward function to associate triggers with target actions. Nevertheless, existing studies typically rely on backdoor reward functions with fixed values or conditional flipping, which lack universality across diverse DRL tasks and backdoor designs, resulting in fluctuations or even failure in practice. This paper proposes the first universal action-level backdoor attack framework, called UNIDOOR, which enables adaptive exploration of backdoor reward functions through performance monitoring, eliminating the reliance on expert knowledge and grid search. We highlight that action tampering serves as a crucial component of action-level backdoor attacks in continuous action scenarios, as it addresses attack failures caused by low-frequency target actions. Extensive evaluations demonstrate that UNIDOOR significantly enhances the attack performance of action-level backdoors, showcasing its universality across diverse attack scenarios, including single/multiple agents, single/multiple backdoors, discrete/continuous action spaces, and sparse/dense reward signals. Furthermore, visualization results encompassing state distribution, neuron activation, and animations demonstrate the stealthiness of UNIDOOR. The source code of UNIDOOR can be found at https://github.com/maoubo/UNIDOOR.

强化学习后门攻击安全防御通用框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。