让无人机导航自动优化奖励函数,闭环提升成功率71%。
AgenticRL: Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation

- 用大模型自动生成任务奖励并迭代优化策略
- 闭环改进使导航成功率提升71%,真实场景成功率达91%
- 支持从仿真到现实的迁移,适合无人系统自主决策研究
深度强化学习在自主机器人复杂导航任务中展现出巨大潜力,但其实际应用仍严重依赖人工设计奖励函数和反复手动调参,耗时且难以保证任务效果。本文提出AgenticRL框架,通过多模态生成式预训练大模型(GPT)代理实现无人机导航任务中的自动化奖励设计、策略优化与真实部署。该代理能解析任务信息与视觉观测,生成特定任务奖励,使用近端策略优化(PPO)训练策略,并作为评判者通过诊断包评估训练后的策略以生成反馈。基于反馈,代理识别失败模式并闭环重构奖励函数,持续改进策略。为增强推理阶段能力,框架利用真实世界图像与自然语言任务信息自动识别当前场景并选择适配策略执行。在多个导航任务上进行评估,包括穿越门框、避障、墙障穿越与着陆、轨迹跟踪及运动行为学习。实验表明,闭环优化相比初始奖励显著提升策略表现71%;同时验证了仿真到现实的迁移能力,实测成功率达91%,仿真到现实精度达94%。
原文摘要 · Abstract (English)
Deep reinforcement learning has shown strong potential for enabling autonomous robots to learn complex navigational tasks. However, its practical use still depends heavily on human designed reward functions and repeated manual fine tuning, which is time consuming and does not guarantee high success in the desired task. This paper presents AgenticRL, agent guided reinforcement learning framework that increases autonomy in reward design, policy refinement, and real world deployment for unmanned aerial vehicles (UAV) navigation tasks. AgenticRL uses a multimodal generative pre-trained transformer (GPT) agent to interpret task information and visual scene observations, generate task specific reward functions, train policies using Proximal Policy Optimization (PPO) algorithm, and then act as a critic by evaluating the trained policy through diagnosis packets to generate feedback. Based on this feedback, the agent identifies failure modes and refines the reward function in a closed loop self improvement process. To further leverage the multimodal GPT agent during inference, AgenticRL uses real world images and natural language task information to automatically identify the active scenario and select the appropriate trained policy for execution. The framework is evaluated on multiple navigational tasks, including gate traversal, obstacle avoidance, wall barrier crossing with landing, trajectory following, and motion behavior learning. Experimental results show that the closed loop refinement process improves policy behavior compared with initial rewards by 71%. We also demonstrate sim-to-real transfer of the proposed framework, achieving a real world success rate of 91% and a sim-to-real accuracy of 94%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。