用深度高斯过程提升强化学习的不确定性估计，让智能体更安全高效地探索。
Deep Gaussian Process Proximal Policy Optimization
- 用深度高斯过程同时建模策略和价值函数
- 在连续控制任务上性能媲美PPO，且不确定性校准良好
- 适合对安全性要求高的强化学习应用
强化学习中的不确定性估计在需要平衡安全探索与高效学习的控制任务中至关重要。尽管深度神经网络推动了强化学习的发展，但其往往缺乏校准良好的不确定性估计。本文提出深度高斯过程近端策略优化（GPPO），一种可扩展的、无模型的演员-评论家算法，利用深度高斯过程（DGPs）同时逼近策略和价值函数。GPPO在标准高维连续控制基准测试上表现不逊于近端策略优化（PPO），同时提供了校准良好的不确定性估计，可指导更安全、更高效的探索。
原文摘要 · Abstract (English)
Uncertainty estimation for Reinforcement Learning (RL) is a critical component in control tasks where agents must balance safe exploration and efficient learning. While deep neural networks have enabled breakthroughs in RL, they often lack calibrated uncertainty estimates. We introduce Deep Gaussian Process Proximal Policy Optimization (GPPO), a scalable, model-free actor-critic algorithm that leverages Deep Gaussian Processes (DGPs) to approximate both the policy and value function. GPPO maintains competitive performance with respect to Proximal Policy Optimization on standard high-dimensional continuous control benchmarks while providing well-calibrated uncertainty estimates that can inform safer and more effective exploration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。