用移动参考系让深度强化学习控制旋转爆震发动机模式切换更稳定
Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions
- 将强化学习置于随爆震波移动的参考系，实现快慢动态分离
- 在不同起始状态和目标模式下，新方法控制成功率显著提升
- 适合研究多时标非线性系统控制的工程师与学者
旋转爆震发动机（RDE）是一种具有更高热效率和比冲潜力的推进概念，但其非线性特性，包括向振荡或混沌传播模式的转变，可能影响实际运行。深度强化学习（DRL）是控制此类复杂非线性动力学的有前景方法，但RDE系统的多时标特性使直接应用DRL面临挑战。本文通过将DRL问题重构到随爆震波形移动的参考系中，使波结构对智能体呈现准稳态，从而实现快速爆震传播与缓慢运行模式动态之间的尺度分离。在简化的一维RDE模型中，训练DRL控制器调节分段注入压力，实现不同锁相模式间的快速转换。在多种执行周期、初始状态和目标模式下，移动参考系训练的控制器比静止参考系更可靠，且在更宽的执行周期范围内有效。结果表明，具备对称性的移动参考系建模对相关多尺度流动控制问题具有价值，应尽可能利用尺度分离以实现多时标系统的DRL控制。
原文摘要 · Abstract (English)
Rotating detonation engines (RDEs) are a promising propulsion concept that may offer higher thermodynamic efficiency and specific impulse than conventional systems, but nonlinear phenomena, including transitions to oscillatory or chaotic propagation modes, can hinder practical operation. Deep Reinforcement Learning (DRL) has emerged as a promising method for controlling complex nonlinear dynamics such as those observed in RDEs. However, the multi-timescale nature of the RDE system makes direct application of DRL challenging. We address this challenge by reformulating the DRL problem in a moving reference frame that follows the detonation-wave pattern, making the wave structure appear quasi-steady to the agent. This reformulation enables scale separation between fast detonation propagation and slower operating-mode dynamics. We train DRL controllers to modulate spatially segmented injection pressure in a one-dimensional reduced-order RDE model and induce rapid transitions between different mode-locked states. Across a range of actuation periods, initial states, and target modes, controllers trained in the moving frame learn more reliably than those trained in a stationary frame and remain effective over a broader range of actuation periods. These results suggest that symmetry-aware moving reference frame formulations may be useful for related multiscale flow-control problems and that scale separation should be exploited whenever possible to enable DRL control of multi-timescale systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。