用大模型动态调整奖励函数,让无人机巡检更高效全覆盖。
Prompt Informed Reinforcement Learning for Visual Coverage Path Planning
- 用大语言模型提供语义反馈,实时优化强化学习的奖励函数。
- 在仿真中提升14%~27%视觉覆盖率,降低18%冗余,提高25%续航效率。
- 无需额外训练即可适应新环境,适合复杂空间探索任务的机器人系统。
无人飞行器(UAV)的视觉覆盖路径规划需协调飞行与摄像头控制以最大化覆盖范围、最小化冗余并保持电池效率。传统强化学习方法依赖特定环境的奖励设计,缺乏语义适应性。本文提出提示感知强化学习(PIRL),将大语言模型(GPT-3.5)的零样本推理与上下文学习能力融入好奇心驱动的强化学习中。PIRL利用LLM提供的语义反馈,动态调整基于PPO算法的策略奖励函数,指导无人机在位置与相机角度上的优化。模型在OpenAI Gym和Webots仿真器中进行训练与评估,后者引入真实物理动力学。结果表明,PIRL在不同环境中均优于多个基线方法:在OpenAI Gym中视觉覆盖率提升最高达14%,在Webots中提升达27%;电池效率最高提升25%,冗余降低最高达18%。实验验证了大模型引导奖励塑造在复杂空间探索中的有效性,为将自然语言先验融入机器人强化学习提供了新方向。
原文摘要 · Abstract (English)
Visual coverage path planning with unmanned aerial vehicles (UAVs) requires agents to strategically coordinate UAV motion and camera control to maximize coverage, minimize redundancy, and maintain battery efficiency. Traditional reinforcement learning (RL) methods rely on environment-specific reward formulations that lack semantic adaptability. This study proposes Prompt-Informed Reinforcement Learning (PIRL), a novel approach that integrates the zero-shot reasoning ability and in-context learning capability of large language models with curiosity-driven RL. PIRL leverages semantic feedback from an LLM, GPT-3.5, to dynamically shape the reward function of the Proximal Policy Optimization (PPO) RL policy guiding the agent in position and camera adjustments for optimal visual coverage. The PIRL agent is trained using OpenAI Gym and evaluated in various environments. Furthermore, the sim-to-real-like ability and zero-shot generalization of the agent are tested by operating the agent in Webots simulator which introduces realistic physical dynamics. Results show that PIRL outperforms multiple learning-based baselines such as PPO with static rewards, PPO with exploratory weight initialization, imitation learning, and an LLM-only controller. Across different environments, PIRL outperforms the best-performing baseline by achieving up to 14% higher visual coverage in OpenAI Gym and 27% higher in Webots, up to 25% higher battery efficiency, and up to 18\% lower redundancy, depending on the environment. The results highlight the effectiveness of LLM-guided reward shaping in complex spatial exploration tasks and suggest a promising direction for integrating natural language priors into RL for robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。