提出在线检测并抑制深度强化学习后门攻击的新方法
BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning
- 通过分析动作分布尾部变化，无须识别触发器即可发现后门行为
- 在多类基准测试中，检测准确率超已有方法，且无需模型微调
- 适合实时系统防御，尤其适用于复杂触发模式的攻击场景
后门攻击对深度强化学习（DRL）构成严重威胁。现有防御方法通常依赖奖励异常来逆向推导触发器，并通过模型微调消除后门，但复杂触发模式会削弱其鲁棒性，且微调成本高，实用性受限。为此，我们转向与触发器无关的后门输出行为，提出 BehaviorGuard——一种面向 DRL 的在线行为基后门检测与缓解框架。具体而言，我们发现无论是否出现触发器，被植入后门的策略都会导致动作分布产生一致偏移，以确保后门可靠激活，从而在高分位区域和分布尾部留下可检测痕迹。基于此，我们设计了一种新度量，捕捉动作分布中的行为漂移，实现在运行时识别并抑制后门动作。据我们所知，这是首个能同时应对单智能体与多智能体 DRL 后门攻击的在线防御方法。在多种基准和不同攻击类型下评估，BehaviorGuard 在有效性与效率上均持续优于先前方法。
原文摘要 · Abstract (English)
Backdoor attacks pose a serious threat to deep reinforcement learning (DRL). Current defenses typically rely on reward anomalies to reverse-engineer triggers and model finetuning to remove backdoors. However, complex trigger patterns undermine their robustness, and fine-tuning entails high costs, limiting practical utility. Therefore, we shift defense concerns to trigger-agnostic backdoor output behaviors and propose BehaviorGuard, an online behavior-based backdoor detection and mitigation framework for DRL. Specifically, we find that regardless of attacks, backdoored policies induce consistent shifts in action distributions to ensure reliable activation, leaving detectable traces in high-quantile regions and distribution tails, even in the absence of triggers. Based on this, we design a novel metric that captures behavioral drift in action distributions to identify and suppress backdoor actions at runtime. To our knowledge, this is the first online backdoor defense that counters attacks both in single- and multi-agent DRL. Evaluated across diverse benchmarks with different backdoor attacks, BehaviorGuard consistently surpasses prior methods in both efficacy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。