用神经微分方程建模随机环境，提升强化学习的适应性与效率。
Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning
- 用神经SDE捕捉动态随机性，优于传统模型。
- 仅需少量交互即可适应环境变化，样本效率高。
- 适合处理部分可观测的复杂连续控制任务。
我们研究在模型化强化学习框架中，使用神经常微分方程（neural ODE）和随机微分方程（neural SDE）来建模全观测和部分观测环境中固有的随机动态。通过一系列仿真，我们发现神经SDE能更有效地捕捉转移动态中的内在随机性，在复杂场景中实现高性能策略并显著提升样本效率。我们利用神经ODE和SDE通过逆模型实现对环境动态变化的高效策略适应，仅需少量与新环境的交互。为应对部分可观测性，我们提出一种结合隐空间中ODE与GAN训练随机组件的隐式SDE模型。基于该模型生成的策略在多个具有随机性的连续控制基准测试中表现优异，优于或匹配通用的模型基与模型无关方法。本工作验证了动作条件隐式SDE在存在随机转移环境中的强化学习规划有效性。代码已开源：https://github.com/ChaoHan-UoS/NeuralRL。
原文摘要 · Abstract (English)
We investigate neural ordinary and stochastic differential equations (neural ODEs and SDEs) to model stochastic dynamics in fully and partially observed environments within a model-based reinforcement learning (RL) framework. Through a sequence of simulations, we show that neural SDEs more effectively capture the inherent stochasticity of transition dynamics, enabling high-performing policies with improved sample efficiency in challenging scenarios. We leverage neural ODEs and SDEs for efficient policy adaptation to changes in environment dynamics via inverse models, requiring only limited interactions with the new environment. To address partial observability, we introduce a latent SDE model that combines an ODE with a GAN-trained stochastic component in latent space. Policies derived from this model provide a strong baseline, outperforming or matching general model-based and model-free approaches across stochastic continuous-control benchmarks. This work demonstrates the applicability of action-conditional latent SDEs for RL planning in environments with stochastic transitions. Our code is available at: https://github.com/ChaoHan-UoS/NeuralRL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。