用自抗扰控制提升强化学习安全性，减少违规与震荡。
Enhance the Safety in Reinforcement Learning by ADRC Lagrangian Methods
- 引入自抗扰控制优化拉格朗日方法，增强鲁棒性
- 安全违规减少74%，约束违反幅度降低89%
- 适合高安全要求的复杂强化学习场景
安全强化学习旨在最大化奖励的同时满足安全约束，通常采用基于拉格朗日的方法。然而，现有方法如PID和经典拉格朗日方法因参数敏感性和固有相位滞后，易产生振荡和频繁安全违规。为此，我们提出基于自抗扰控制（ADRC）的拉格朗日方法，显著提升鲁棒性并减少振荡。该统一框架包含经典与PID拉格朗日方法作为特例，实验表明，本方法可将安全违规减少74%，约束违反幅度降低89%，平均代价下降67%，在复杂环境中的安全强化学习中表现卓越。
原文摘要 · Abstract (English)
Safe reinforcement learning (Safe RL) seeks to maximize rewards while satisfying safety constraints, typically addressed through Lagrangian-based methods. However, existing approaches, including PID and classical Lagrangian methods, suffer from oscillations and frequent safety violations due to parameter sensitivity and inherent phase lag. To address these limitations, we propose ADRC-Lagrangian methods that leverage Active Disturbance Rejection Control (ADRC) for enhanced robustness and reduced oscillations. Our unified framework encompasses classical and PID Lagrangian methods as special cases while significantly improving safety performance. Extensive experiments demonstrate that our approach reduces safety violations by up to 74%, constraint violation magnitudes by 89%, and average costs by 67\%, establishing superior effectiveness for Safe RL in complex environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。