让强化学习在对抗扰动下更可靠，通过优化最坏情况下的表现。
DR-PETS: Learning-Based Control With Planning in Adversarial Environments
- 基于PETS框架，引入对抗性扰动的分布鲁棒优化
- 在摆杆和小车平衡任务中实现最坏情况下的稳定性能
- 适合需要高可靠性决策的工业控制场景
确保对认知性、可能具有对抗性的扰动具备鲁棒性，是实现可靠现实决策的关键。尽管基于概率集成的轨迹采样算法（PETS）通过集成式概率模型天然处理不确定性，但其缺乏对结构化对抗或最坏情况不确定性分布的保障。为此，我们提出DR-PETS，一种分布鲁棒的PETS扩展，可认证对抗扰动下的鲁棒性。通过p-Wasserstein模糊集形式化不确定性，利用最小-最大优化框架实现对抗感知规划。与被动应对随机性的PETS不同，DR-PETS通过可计算的凸近似主动优化鲁棒性，并融入PETS的规划循环。在摆杆稳定和小车平衡任务上的实验表明，DR-PETS能有效认证对抗参数扰动下的鲁棒性，在最坏情况下保持一致性能，而传统PETS则显著退化。
原文摘要 · Abstract (English)
Ensuring robustness against epistemic, possibly adversarial, perturbations is essential for reliable real-world decision-making. While the Probabilistic Ensembles with Trajectory Sampling (PETS) algorithm inherently handles uncertainty via ensemble-based probabilistic models, it lacks guarantees against structured adversarial or worst-case uncertainty distributions. To address this, we propose DR-PETS, a distributionally robust extension of PETS that certifies robustness against adversarial perturbations. We formalize uncertainty via a p-Wasserstein ambiguity set, enabling worst-case-aware planning through a min-max optimization framework. While PETS passively accounts for stochasticity, DR-PETS actively optimizes robustness via a tractable convex approximation integrated into PETS planning loop. Experiments on pendulum stabilization and cart-pole balancing show that DR-PETS certifies robustness against adversarial parameter perturbations, achieving consistent performance in worst-case scenarios where PETS deteriorates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。