研究发现13种神经可塑性干预能降低后门攻击风险，仅1种会加剧威胁。
Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning
- 通过实验分析1.4万种组合，揭示干预机制对后门的影响
- 多数干预通过破坏激活路径和压缩表征空间来抑制后门
- 提出新框架与损失曲面锐度指标，助力后门检测
大量研究指出后门攻击对深度强化学习（DRL）构成严重威胁。然而，以往工作主要聚焦于基础场景，而现代DRL智能体中不可或缺的可塑性干预措施，其对后门漏洞的影响尚未系统研究，这给实际部署带来风险。为填补这一空白，我们实证分析了14,664个集成代表性干预与攻击场景的案例。结果表明，仅一种干预（即SAM）会加剧后门威胁，其余均具有缓解作用。病理分析发现，威胁加剧源于后门梯度放大，而缓解则来自激活路径破坏与表征空间压缩。基于此，我们提出两个新见解：(1) 构建了概念框架SCC，解析干预与后门在DRL中的机理交互；(2) 异常损失曲面锐度可作为DRL后门检测的关键指标。
原文摘要 · Abstract (English)
Extensive research has highlighted the severe threats posed by backdoor attacks to deep reinforcement learning (DRL). However, prior studies primarily focus on vanilla scenarios, while plasticity interventions have emerged as indispensable built-in components of modern DRL agents. Despite their effectiveness in mitigating plasticity loss, the impact of these interventions on DRL backdoor vulnerabilities remains underexplored, and this lack of systematic investigation poses risks in practical DRL deployments. To bridge this gap, we empirically study 14,664 cases integrating representative interventions and attack scenarios. We find that only one intervention (i.e., SAM) exacerbates backdoor threats, while other interventions mitigate them. Pathological analysis identifies that the exacerbation is attributed to backdoor gradient amplification, while the mitigation stems from activation pathway disruption and representation space compression. From these findings, we derive two novel insights: (1) a conceptual framework SCC for robust backdoor injection that deconstructs the mechanistic interplay between interventions and backdoors in DRL, and (2) abnormal loss landscape sharpness as a key indicator for DRL backdoor detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。