arXiv:2508.16807cs.ROcs.AI2025-08被引 2

对比PPO与SAC在狭窄空间无人机导航中的表现,发现稳定收敛比样本效率更重要。

Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach

  • 用PPO和SAC分别训练无人机在模拟管道中自主飞行
  • PPO成功完成全程无碰撞飞行,SAC仅能通过前半段即失败
  • 适用于对安全性要求极高的工业巡检场景

自主无人机对通风管道等封闭工业设施的巡检,要求导航策略具备极高鲁棒性,碰撞不可接受。尽管深度强化学习(DRL)提供了强大方法,但其在基于策略(on-policy)与非基于策略(off-policy)算法间存在关键权衡:后者虽具高样本效率,利于减少昂贵且危险的真实世界调优;前者则通常更具训练稳定性,对危险密集环境中的可靠收敛至关重要。本文通过对比领先基于策略算法近端策略优化(PPO)与非基于策略算法软演员-批评机(SAC),在高保真模拟器中对程序生成的管道进行精确飞行任务测试。结果表明,PPO始终学习到稳定、无碰撞的策略并完成全程;而SAC未能找到完整解决方案,收敛至次优策略,仅能通过初始段后即失败。该研究为高精度、安全关键型导航任务提供证据:在特定场景下,成熟基于策略方法的可靠收敛性可能比非基于策略算法的名义样本效率更具决定性。

原文摘要 · Abstract (English)

Autonomous UAV inspection of confined industrial infrastructure, such as ventilation ducts, demands robust navigation policies where collisions are unacceptable. While Deep Reinforcement Learning (DRL) offers a powerful paradigm for developing such policies, it presents a critical trade-off between on-policy and off-policy algorithms. Off-policy methods promise high sample efficiency, a vital trait for minimizing costly and unsafe real-world fine-tuning. In contrast, on-policy methods often exhibit greater training stability, which is essential for reliable convergence in hazard-dense environments. This paper directly investigates this trade-off by comparing a leading on-policy algorithm, Proximal Policy Optimization (PPO), against an off-policy counterpart, Soft Actor-Critic (SAC), for precision flight in procedurally generated ducts within a high-fidelity simulator. Our results show that PPO consistently learned a stable, collision-free policy that completed the entire course. In contrast, SAC failed to find a complete solution, converging to a suboptimal policy that navigated only the initial segments before failure. This work provides evidence that for high-precision, safety-critical navigation tasks, the reliable convergence of a well-established on-policy method can be more decisive than the nominal sample efficiency of an off-policy algorithm.

无人机导航强化学习安全关键

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。