arXiv:2504.16875cs.LG2025-04

用强化学习动态调参,让氢柴油双燃料发动机更稳更准。

Hybrid Reinforcement Learning and Model Predictive Control for Adaptive Control of Hydrogen-Diesel Dual-Fuel Combustion

  • 混合方法:用模型预测控制保安全,强化学习自适应调参考值。
  • 实验误差从0.57巴降到0.44巴,负载跟踪更精准。
  • 适合做发动机智能控制、工业系统自适应优化的研究者。

强化学习(RL)与机器学习集成的模型预测控制(ML-MPC)在优化氢柴油双燃料发动机控制方面具有潜力,能有效处理多输入多输出系统和非线性过程。ML-MPC可提供安全且最优的控制,确保发动机运行在预设安全限值内;而RL则凭借其学习能力,在环境变化时具备强适应性。然而单独使用任一方法均有局限:RL早期学习阶段控制输入方差高,可能执行不安全动作,导致机械损伤;而ML-MPC依赖精确系统模型,对喷油器老化等系统漂移适应能力弱。为此,本文提出一种混合方法:基于ML-MPC框架,引入RL代理动态调整负载跟踪参考值,同时由ML-MPC保障探索过程中的控制安全性。为验证效果,故意改变燃油压力以引入模型-实物失配。结果显示,仅用ML-MPC时指示平均有效压力的均方根误差(RMSE)达0.57巴;引入RL后,误差降至0.44巴,量化提升了负载跟踪性能。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) and Machine Learning Integrated Model Predictive Control (ML-MPC) are promising approaches for optimizing hydrogen-diesel dual-fuel engine control, as they can effectively control multiple-input multiple-output systems and nonlinear processes. ML-MPC is advantageous for providing safe and optimal controls, ensuring the engine operates within predefined safety limits. In contrast, RL is distinguished by its adaptability to changing conditions through its learning-based approach. However, the practical implementation of either method alone poses challenges. RL requires high variance in control inputs during early learning phases, which can pose risks to the system by potentially executing unsafe actions, leading to mechanical damage. Conversely, ML-MPC relies on an accurate system model to generate optimal control inputs and has limited adaptability to system drifts, such as injector aging, which naturally occur in engine applications. To address these limitations, this study proposes a hybrid RL and ML-MPC approach that uses an ML-MPC framework while incorporating an RL agent to dynamically adjust the ML-MPC load tracking reference in response to changes in the environment. At the same time, the ML-MPC ensures that actions stay safe throughout the RL agent's exploration. To evaluate the effectiveness of this approach, fuel pressure is deliberately varied to introduce a model-plant mismatch between the ML-MPC and the engine test bench. The result of this mismatch is a root mean square error (RMSE) in indicated mean effective pressure of 0.57 bar when running the ML-MPC. The experimental results demonstrate that RL successfully adapts to changing boundary conditions by altering the tracking reference while ML-MPC ensures safe control inputs. The quantitative improvement in load tracking by implementing RL is an RSME of 0.44 bar.

发动机控制强化学习模型预测双燃料

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。