用向量量化降低强化学习对抗攻击影响，提升部署稳定性。
Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantization
- 用向量量化对输入观测做变换，压缩对抗扰动空间。
- 在多个环境实验中有效抵御观测层面的对抗攻击。
- 计算高效，可与对抗训练无缝结合，适合实际部署。
近期研究表明，训练中表现良好的强化学习（RL）智能体在部署阶段往往对对抗扰动缺乏鲁棒性，凸显了部署前构建鲁棒智能体的重要性。现有方法多聚焦于基于训练的增强策略，如提升神经网络本身的鲁棒性或在强攻击下对抗训练智能体。本文提出一种基于输入变换的防御机制：采用向量量化（VQ）对输入观测进行变换，从而在测试阶段缩小对抗攻击的空间，使变换后的观测更不易受攻击影响。该方法计算高效，可与对抗训练无缝集成，进一步提升智能体对对抗攻击的鲁棒性。在多个环境中的大量实验表明，使用VQ作为输入变换能有效防御针对智能体观测的对抗攻击。
原文摘要 · Abstract (English)
Recent studies reveal that well-performing reinforcement learning (RL) agents in training often lack resilience against adversarial perturbations during deployment. This highlights the importance of building a robust agent before deploying it in the real world. Most prior works focus on developing robust training-based procedures to tackle this problem, including enhancing the robustness of the deep neural network component itself or adversarially training the agent on strong attacks. In this work, we instead study an input transformation-based defense for RL. Specifically, we propose using a variant of vector quantization (VQ) as a transformation for input observations, which is then used to reduce the space of adversarial attacks during testing, resulting in the transformed observations being less affected by attacks. Our method is computationally efficient and seamlessly integrates with adversarial training, further enhancing the robustness of RL agents against adversarial attacks. Through extensive experiments in multiple environments, we demonstrate that using VQ as the input transformation effectively defends against adversarial attacks on the agent's observations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。