用Transformer强化学习,让飞机在强风中自动调节升力。
Attention on flow control: transformer-based reinforcement learning for lift regulation in highly disturbed flows
- 用Transformer处理有限传感器数据,应对复杂风扰的观测难题。
- 训练加速后,控制效果超越最优比例控制,且风浪越多优势越明显。
- 机翼前四分之一处旋转比中间旋转更省力,升力调节更高效。
针对强风扰动下线性控制失效的问题,本文提出一种基于Transformer的强化学习框架,通过舵面俯仰控制来调节任意长阵风序列中的气动升力。随机阵风导致间歇性高方差流动,仅能通过有限表面压力传感器观测,使控制问题远超静态流情形。Transformer有效缓解了部分可观测性挑战。我们采用两种技术加速训练:利用线性控制作为专家策略进行预训练,以及任务级迁移学习(将孤立阵风训练的策略扩展至多阵风场景)。结果表明,所学策略优于最优比例控制,且阵风数量越多,性能差距越大。在少量连续阵风环境中训练的策略可有效泛化至任意长阵风序列。通过分解升力发现,前四分之一弦长俯仰控制相比中弦长控制显著降低控制努力,其优势源于前四分之一俯仰能更好利用附加质量效应。
原文摘要 · Abstract (English)
A linear flow control strategy designed for weak disturbances may not remain effective in sequences of strong disturbances due to nonlinear interactions, but it is sensible to leverage it for developing a better strategy. In the present study, we propose a transformer-based reinforcement learning (RL) framework to learn an effective control strategy for regulating aerodynamic lift in arbitrarily long gust sequences via pitch control. The random gusts produce intermittent, high-variance flows observed only through limited surface pressure sensors, making this control problem inherently challenging compared to stationary flows. The transformer addresses the challenge of partial observability from the limited surface pressures. We demonstrate that the training can be accelerated with two techniques -- pretraining with an expert policy (here, linear control) and task-level transfer learning (here, extending a policy trained on isolated gusts to multiple gusts). We show that the learned strategy outperforms the best proportional control, with the performance gap widening as the number of gusts increases. The control strategy learned in an environment with a small number of successive gusts is shown to effectively generalize to an environment with an arbitrarily long sequence of gusts. We investigate the pivot configuration and show that quarter-chord pitching control can achieve superior lift regulation with substantially less control effort compared to mid-chord pitching control. Through a decomposition of the lift, we attribute this advantage to the dominant added-mass contribution accessible via quarter-chord pitching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。