用强化学习让多颗卫星自主协作拍地球,提升资源利用率。
Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study
- 用多智能体强化学习建模卫星协作拍摄任务
- 在仿真环境中验证算法稳定性,提升资源管理效率
- 适合研究卫星自主决策与分布式系统的人参考
低地球轨道(LEO)卫星数量激增,推动了地球观测(EO)任务的发展,在气候监测、灾害管理等方面发挥重要作用。然而,多卫星系统的自主协同仍面临挑战。传统优化方法难以应对动态任务中的实时决策需求,亟需强化学习(RL)与多智能体强化学习(MARL)技术。本文通过建模单星操作并扩展至多星星座,采用PPO、IPPO、MAPPO和HAPPO等先进MARL算法,在近真实卫星仿真环境中评估其训练稳定性和性能。结果表明,MARL能有效平衡成像任务与能源、存储资源管理,缓解多星协同中的非平稳性与奖励依赖问题。研究为自主卫星运行提供了实践指导,助力分布式地球观测任务中的策略学习优化。
原文摘要 · Abstract (English)
The exponential growth of Low Earth Orbit (LEO) satellites has revolutionised Earth Observation (EO) missions, addressing challenges in climate monitoring, disaster management, and more. However, autonomous coordination in multi-satellite systems remains a fundamental challenge. Traditional optimisation approaches struggle to handle the real-time decision-making demands of dynamic EO missions, necessitating the use of Reinforcement Learning (RL) and Multi-Agent Reinforcement Learning (MARL). In this paper, we investigate RL-based autonomous EO mission planning by modelling single-satellite operations and extending to multi-satellite constellations using MARL frameworks. We address key challenges, including energy and data storage limitations, uncertainties in satellite observations, and the complexities of decentralised coordination under partial observability. By leveraging a near-realistic satellite simulation environment, we evaluate the training stability and performance of state-of-the-art MARL algorithms, including PPO, IPPO, MAPPO, and HAPPO. Our results demonstrate that MARL can effectively balance imaging and resource management while addressing non-stationarity and reward interdependency in multi-satellite coordination. The insights gained from this study provide a foundation for autonomous satellite operations, offering practical guidelines for improving policy learning in decentralised EO missions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。