通过自适应切换规划与执行，实现安全与性能的协同优化。
Safe Planning and Policy Optimization via World Model Learning
- 动态切换模型规划与直接策略执行，应对模型误差
- 安全阈值随能力进化，确保动作持续优于安全基线
- 适用于连续控制中的高风险场景，兼顾可靠与高效
强化学习在真实场景应用中需优先保障安全与可靠性，对智能体行为施加严格约束。基于模型的强化学习利用预测世界模型进行动作规划与策略优化，但模型固有的不准确性可能导致安全关键环境下的灾难性失败。本文提出一种新型基于模型的强化学习框架，联合优化任务性能与安全性。为缓解世界模型误差问题，方法引入自适应机制，动态切换模型规划与直接策略执行；通过隐式世界模型解决传统方法的目标不匹配问题；同时采用随智能体能力演进而动态调整的安全阈值，始终选择在性能与安全性上均优于安全策略建议的动作。实验表明，该方法显著优于非自适应方法，在多种安全关键的连续控制任务中实现安全与性能的同步优化，表现出更强鲁棒性。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) applications in real-world scenarios must prioritize safety and reliability, which impose strict constraints on agent behavior. Model-based RL leverages predictive world models for action planning and policy optimization, but inherent model inaccuracies can lead to catastrophic failures in safety-critical settings. We propose a novel model-based RL framework that jointly optimizes task performance and safety. To address world model errors, our method incorporates an adaptive mechanism that dynamically switches between model-based planning and direct policy execution. We resolve the objective mismatch problem of traditional model-based approaches using an implicit world model. Furthermore, our framework employs dynamic safety thresholds that adapt to the agent's evolving capabilities, consistently selecting actions that surpass safe policy suggestions in both performance and safety. Experiments demonstrate significant improvements over non-adaptive methods, showing that our approach optimizes safety and performance simultaneously rather than merely meeting minimum safety requirements. The proposed framework achieves robust performance on diverse safety-critical continuous control tasks, outperforming existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。