用Transformer模型提升飞行器间距保障的适应性,应对复杂空域挑战。
Transformer-based Multi-agent Reinforcement Learning for Separation Assurance in Structured and Unstructured Airspaces
- 将状态空间转为相对极坐标,结合Transformer编码器训练
- 单层编码器在两类空域中碰撞率接近零,优于更深结构
- 适合需动态调整的先进空中交通管理系统
传统基于优化的流量管理依赖严格预设时刻表,难以适应先进空中交通(AAM)的随机运行需求。相比之下,多智能体强化学习(MARL)提供去中心化、自适应框架,更利于应对不确定性,保障飞行器安全间距。然而现有MARL方法常过度拟合特定空域结构,泛化能力受限。为此,本文将MARL问题重构为相对极坐标状态空间,采用Transformer编码器在多样化交通模式与交叉角度下进行训练。所学模型可生成速度建议以化解冲突,同时保持飞机接近期望巡航速度。实验评估了1、2、3层编码器在结构化与非结构化空域的表现,发现单层配置优于深层结构,在近地空中碰撞率近乎为零且间距违规时间更短。此外,该配置也优于纯注意力机制设计的基线模型。结果表明,新的状态表示、神经网络架构设计及训练策略共同构建了一个可适应、可扩展的去中心化飞行器间距保障方案。
原文摘要 · Abstract (English)
Conventional optimization-based metering depends on strict adherence to precomputed schedules, which limits the flexibility required for the stochastic operations of Advanced Air Mobility (AAM). In contrast, multi-agent reinforcement learning (MARL) offers a decentralized, adaptive framework that can better handle uncertainty, required for safe aircraft separation assurance. Despite this advantage, current MARL approaches often overfit to specific airspace structures, limiting their adaptability to new configurations. To improve generalization, we recast the MARL problem in a relative polar state space and train a transformer encoder model across diverse traffic patterns and intersection angles. The learned model provides speed advisories to resolve conflicts while maintaining aircraft near their desired cruising speeds. In our experiments, we evaluated encoder depths of 1, 2, and 3 layers in both structured and unstructured airspaces, and found that a single encoder configuration outperformed deeper variants, yielding near-zero near mid-air collision rates and shorter loss-of-separation infringements than the deeper configurations. Additionally, we showed that the same configuration outperforms a baseline model designed purely with attention. Together, our results suggest that the newly formulated state representation, novel design of neural network architecture, and proposed training strategy provide an adaptable and scalable decentralized solution for aircraft separation assurance in both structured and unstructured airspaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。