用Transformer统一多阶段航天器轨迹优化,实现跨阶段自适应控制。
Multi-Phase Spacecraft Trajectory Optimization via Transformer-Based Reinforcement Learning
- 采用Transformer替代传统循环网络,统一建模多阶段任务
- 在火箭发射、分离、入轨等复杂场景中达到近最优控制性能
- 适合需要跨阶段自主决策的航天任务,减少对分段控制器依赖
自主航天器控制在发射、上升、级间分离和轨道插入等任务阶段仍面临挑战,需具备跨动态差异阶段的自适应策略。尽管强化学习在单个航天动力学任务中表现良好,但现有方法通常为不同阶段分别设计策略,限制了适应性并增加操作复杂度。本文提出一种基于Transformer的强化学习框架,通过单一策略架构统一多阶段轨迹优化,利用Transformer对长时序上下文的建模能力。在近端策略优化(PPO)基础上,以Transformer编码器-解码器结构取代传统循环网络,使智能体在关键操作持续数秒至数分钟的跨阶段过程中保持连贯记忆。通过引入门控Transformer-XL(GTrXL)结构,消除人工阶段切换,同时保障控制决策稳定性。验证过程逐步推进:先在单阶段基准任务(双积分器与Van der Pol振子)上展示近最优性能,再扩展至多阶段路径点导航,最终解决包含大气飞行、级间分离和真空操作的复杂火箭上升问题。结果表明,该框架不仅在简单情形下匹配解析解,还能在动态差异显著的阶段间学习出一致的控制策略,为可扩展的自主任务规划奠定基础,降低对阶段专用控制器的依赖,同时兼容安全关键验证协议。
原文摘要 · Abstract (English)
Autonomous spacecraft control for mission phases such as launch, ascent, stage separation, and orbit insertion remains a critical challenge due to the need for adaptive policies that generalize across dynamically distinct regimes. While reinforcement learning (RL) has shown promise in individual astrodynamics tasks, existing approaches often require separate policies for distinct mission phases, limiting adaptability and increasing operational complexity. This work introduces a transformer-based RL framework that unifies multi-phase trajectory optimization through a single policy architecture, leveraging the transformer's inherent capacity to model extended temporal contexts. Building on proximal policy optimization (PPO), our framework replaces conventional recurrent networks with a transformer encoder-decoder structure, enabling the agent to maintain coherent memory across mission phases spanning seconds to minutes during critical operations. By integrating a Gated Transformer-XL (GTrXL) architecture, the framework eliminates manual phase transitions while maintaining stability in control decisions. We validate our approach progressively: first demonstrating near-optimal performance on single-phase benchmarks (double integrator and Van der Pol oscillator), then extending to multiphase waypoint navigation variants, and finally tackling a complex multiphase rocket ascent problem that includes atmospheric flight, stage separation, and vacuum operations. Results demonstrate that the transformer-based framework not only matches analytical solutions in simple cases but also effectively learns coherent control policies across dynamically distinct regimes, establishing a foundation for scalable autonomous mission planning that reduces reliance on phase-specific controllers while maintaining compatibility with safety-critical verification protocols.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。