arXiv:2511.00272cs.LGphysics.flu-dyn2025-11

用领域知识增强强化学习,让混沌对流更稳定可控。

Improving the Robustness of Control of Chaotic Convective Flows with Domain-Informed Reinforcement Learning

  • 在奖励函数中加入贝纳德胞合并项,引导宏观结构优化。
  • 混沌状态下仍实现10%热对流抑制,优于传统控制器。
  • 无需重训练即可跨流态泛化,适合真实系统部署。

混沌对流广泛存在于微流控器件和化学反应器等实际系统中,其稳定控制极具挑战性,尤其在混沌区域常规方法失效。本文聚焦瑞利-贝纳德对流(RBC)这一典型对流传热模型,提出基于领域知识的强化学习(RL)控制方法。采用近端策略优化(PPO)在多种初始条件与流态下训练智能体,并在奖励函数中引入促进贝纳德胞合并的项以融入领域先验。在层流状态下,该方法可将对流传热降低最高达33%;在混沌状态下仍实现10%的抑制效果,显著优于实际应用中的传统控制器。对比无领域先验的智能体,本方法展现出更稳定的流态、更快的训练收敛速度及跨流态泛化能力,无需重新训练。结果表明,精巧的领域先验能极大提升强化学习对混沌流动控制的鲁棒性,推动其实用化部署。

原文摘要 · Abstract (English)

Chaotic convective flows arise in many real-world systems, such as microfluidic devices and chemical reactors. Stabilizing these flows is highly desirable but remains challenging, particularly in chaotic regimes where conventional control methods often fail. Reinforcement Learning (RL) has shown promise for control in laminar flow settings, but its ability to generalize and remain robust under chaotic and turbulent dynamics is not well explored, despite being critical for real-world deployment. In this work, we improve the practical feasibility of RL-based control of such flows focusing on Rayleigh-Bénard Convection (RBC), a canonical model for convective heat transport. To enhance generalization and sample efficiency, we introduce domain-informed RL agents that are trained using Proximal Policy Optimization across diverse initial conditions and flow regimes. We incorporate domain knowledge in the reward function via a term that encourages Bénard cell merging, as an example of a desirable macroscopic property. In laminar flow regimes, the domain-informed RL agents reduce convective heat transport by up to 33%, and in chaotic flow regimes, they still achieve a 10% reduction, which is significantly better than the conventional controllers used in practice. We compare the domain-informed to uninformed agents: Our results show that the domain-informed reward design results in steady flows, faster convergence during training, and generalization across flow regimes without retraining. Our work demonstrates that elegant domain-informed priors can greatly enhance the robustness of RL-based control of chaotic flows, bringing real-world deployment closer.

强化学习混沌控制流体模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。