提出自适应扰动攻击，让强化学习模型在对抗训练中既鲁棒又不丢性能。
Reward-Preserving Attacks For Robust Reinforcement Learning
- 根据目标保留收益比例动态调整扰动强度
- 在多种扰动下保持性能，优于固定强度攻击
- 适合需要高鲁棒性且不牺牲正常表现的RL应用
强化学习中的对抗训练面临挑战：扰动随轨迹传播并随时间累积，固定强度的攻击要么破坏过度，要么过于保守。本文提出奖励保持型攻击，使每个状态下的最优到最差回报差距中仍有α比例可实现。在深度强化学习中，扰动幅度η通过学习的评判函数Q((s,a),η)动态选择，该函数估计α-奖励保持回溯的期望回报。当α取中间值时，这种自适应训练使策略在广泛扰动范围内保持鲁棒性，同时维持原始性能，优于固定半径和均匀采样半径的对抗训练。
原文摘要 · Abstract (English)
Adversarial training in reinforcement learning (RL) is challenging because perturbations cascade through trajectories and compound over time, making fixed-strength attacks either overly destructive or too conservative. We propose reward-preserving attacks, which adapt adversarial strength so that an $α$ fraction of the nominal-to-worst-case return gap remains achievable at each state. In deep RL, perturbation magnitudes $η$ are selected dynamically, using a learned critic $Q((s,a),η)$ that estimates the expected return of $α$-reward-preserving rollouts. For intermediate values of $α$, this adaptive training yields policies that are robust across a wide range of perturbation magnitudes while preserving nominal performance, outperforming fixed-radius and uniformly sampled-radius adversarial training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。