用数据融合方法让强化学习在观测不全时也能控制混沌流。
Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flows
- 用环状网络预测流体动态，结合卡尔曼滤波实时估计状态。
- 在噪声和部分观测下稳定了时空混沌的库兰托-希瓦辛斯基方程。
- 适合做能源与交通中复杂流体控制的研究者参考。
能源与交通领域许多应用的目标是控制湍流。然而，由于混沌动力学和高维性，湍流控制极为困难。无模型强化学习（RL）可通过与环境交互发现最优控制策略，但通常需要完整状态信息，这在实验中往往不可得。我们提出一种数据融合的模型基强化学习（DA-MBRL）框架，适用于部分可观测和噪声测量系统。该框架采用控制感知的回声状态网络进行数据驱动的动力学预测，并结合集合卡尔曼滤波实现实时状态估计。使用离策略演员-评论家算法从状态估计中学习最优控制策略。该框架在库兰托-希瓦辛斯基方程上进行了测试，证明其能从噪声和部分观测中有效稳定时空混沌流。
原文摘要 · Abstract (English)
The goal of many applications in energy and transport sectors is to control turbulent flows. However, because of chaotic dynamics and high dimensionality, the control of turbulent flows is exceedingly difficult. Model-free reinforcement learning (RL) methods can discover optimal control policies by interacting with the environment, but they require full state information, which is often unavailable in experimental settings. We propose a data-assimilated model-based RL (DA-MBRL) framework for systems with partial observability and noisy measurements. Our framework employs a control-aware Echo State Network for data-driven prediction of the dynamics, and integrates data assimilation with an Ensemble Kalman Filter for real-time state estimation. An off-policy actor-critic algorithm is employed to learn optimal control strategies from state estimates. The framework is tested on the Kuramoto-Sivashinsky equation, demonstrating its effectiveness in stabilizing a spatiotemporally chaotic flow from noisy and partial measurements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。