arXiv:2504.05588cs.LGcs.AI2025-04被引 3

用多精度模型+谱奖励函数,高效控制复杂混沌系统。

Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems

  • 结合物理模型与少量高精度数据,构建可微分混合模型。
  • 在两个物理系统上实现媲美高成本仿真评估的控制效果。
  • 适合需要快速稳定控制的复杂系统研究者。

复杂动力系统的不稳定控制在科学与工程中极具挑战。深度强化学习(DRL)在多个科学应用中展现潜力,但控制任务通常需多次与真实物理环境交互,而实验数据稀疏或复杂动力学仿真成本高昂。替代方案是使用代理模型降低计算开销,但离线训练的基于学习的模型在混沌动力学下难以精确捕捉逐点动态。为此,本文提出多精度强化学习(MFRL)框架,利用可微分混合模型进行控制,其中基于物理的混合模型通过有限高精度数据修正。同时提出基于谱的奖励函数以优化RL学习。该框架在两个物理复杂动力系统上验证有效,其控制结果统计特性与高精度环境的多次查询评估一致,优于其他先进基线方法。

原文摘要 · Abstract (English)

Controlling instabilities in complex dynamical systems is challenging in scientific and engineering applications. Deep reinforcement learning (DRL) has seen promising results for applications in different scientific applications. The many-query nature of control tasks requires multiple interactions with real environments of the underlying physics. However, it is usually sparse to collect from the experiments or expensive to simulate for complex dynamics. Alternatively, controlling surrogate modeling could mitigate the computational cost issue. However, a fast and accurate learning-based model by offline training makes it very hard to get accurate pointwise dynamics when the dynamics are chaotic. To bridge this gap, the current work proposes a multi-fidelity reinforcement learning (MFRL) framework that leverages differentiable hybrid models for control tasks, where a physics-based hybrid model is corrected by limited high-fidelity data. We also proposed a spectrum-based reward function for RL learning. The effect of the proposed framework is demonstrated on two complex dynamics in physics. The statistics of the MFRL control result match that computed from many-query evaluations of the high-fidelity environments and outperform other SOTA baselines.

强化学习多精度建模混沌控制物理信息

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。