用强化学习优化卫星轨道,实现精准地面覆盖并提升规划效率。
TLE-Based A2C Agent for Terrestrial Coverage Orbital Path Planning
- 基于开普勒参数构建马尔可夫决策过程,用A2C算法动态调整五项轨道参数。
- 相较PPO算法奖励提升5.8倍,收敛速度加快31.5倍,仅需2000步完成训练。
- 适合需要实时响应的低轨卫星任务规划,兼顾安全性与计算效率。
低地球轨道(LEO)日益拥挤,给地球观测卫星的高效部署与安全运行带来持续挑战。任务规划需兼顾任务需求与与活跃卫星及空间碎片的碰撞风险。本文提出一种基于优势演员-评论家(A2C)算法的强化学习框架,通过自定义OpenAI Gymnasium环境模拟轨道动力学,利用经典开普勒元素优化卫星轨道参数,实现对预设地表半径内的精准覆盖。智能体逐步学习调节五个轨道参数——半长轴、偏心率、倾角、升交点赤经和近地点幅角,以达成目标覆盖。对比近端策略优化(PPO)算法,A2C在累积奖励上高出5.8倍(10.0 vs 9.263025),且收敛时间缩短31.5倍(2,000步 vs 63,000步)。该方法在多种目标坐标下均稳定达成任务目标,具备适用于实时任务规划的计算效率。主要贡献包括:(1)基于两行根轨迹(TLE)的包含物理约束的轨道仿真环境;(2)验证了演员-评论家方法在连续轨道控制中优于信任域方法;(3)展示了快速收敛能力,支持自适应卫星部署。本方法为可扩展、智能化的低轨任务规划提供了高效的强化学习替代方案。
原文摘要 · Abstract (English)
The increasing congestion of Low Earth Orbit (LEO) poses persistent challenges to the efficient deployment and safe operation of Earth observation satellites. Mission planners must now account not only for mission-specific requirements but also for the increasing collision risk with active satellites and space debris. This work presents a reinforcement learning framework using the Advantage Actor-Critic (A2C) algorithm to optimize satellite orbital parameters for precise terrestrial coverage within predefined surface radii. By formulating the problem as a Markov Decision Process (MDP) within a custom OpenAI Gymnasium environment, our method simulates orbital dynamics using classical Keplerian elements. The agent progressively learns to adjust five of the orbital parameters - semi-major axis, eccentricity, inclination, right ascension of ascending node, and the argument of perigee-to achieve targeted terrestrial coverage. Comparative evaluation against Proximal Policy Optimization (PPO) demonstrates A2C's superior performance, achieving 5.8x higher cumulative rewards (10.0 vs 9.263025) while converging in 31.5x fewer timesteps (2,000 vs 63,000). The A2C agent consistently meets mission objectives across diverse target coordinates while maintaining computational efficiency suitable for real-time mission planning applications. Key contributions include: (1) a TLE-based orbital simulation environment incorporating physics constraints, (2) validation of actor-critic methods' superiority over trust region approaches in continuous orbital control, and (3) demonstration of rapid convergence enabling adaptive satellite deployment. This approach establishes reinforcement learning as a computationally efficient alternative for scalable and intelligent LEO mission planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。