用多智能体和视觉语言模型自动规划无人机任务,精准定位目标且快速响应环境变化。
UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoning
- 采用多智能体ReAct框架,结合视觉语言模型理解指令与图像,协同生成飞行路径。
- 在大规模火灾检测任务中平均耗时96.96秒,成功率93%,温度设0.5时最稳定可靠。
- 支持实时动态调整,适合应急救援、环境监测等需快速响应的无人机应用。
我们提出UAV-CodeAgents,一个基于大语言模型和视觉语言模型(LLMs/VLMs)的可扩展多智能体框架,用于自主无人机任务生成。系统利用ReAct(思考+行动)范式解析卫星图像、理解高层自然语言指令,并在极少人工干预下协作生成无人机航迹。核心是基于视觉的像素级定位机制,实现对空中地图中语义目标的精确识别。为支持实时适应性,引入反应式思维循环,使智能体能迭代反思观测结果、修正任务目标并动态协调应对变化环境。在工业与环境火灾探测的大规模任务场景中评估表明,较低解码温度(0.5)可提升规划可靠性并缩短执行时间,平均任务生成时间为96.96秒,成功率93%。我们还在9,000张标注卫星图像上微调Qwen2.5VL-7B模型,实现了跨多样视觉类别的强空间定位能力。为促进复现与后续研究,将公开完整代码库及首个面向视觉语言的无人机规划基准数据集。
原文摘要 · Abstract (English)
We present UAV-CodeAgents, a scalable multi-agent framework for autonomous UAV mission generation, built on large language and vision-language models (LLMs/VLMs). The system leverages the ReAct (Reason + Act) paradigm to interpret satellite imagery, ground high-level natural language instructions, and collaboratively generate UAV trajectories with minimal human supervision. A core component is a vision-grounded, pixel-pointing mechanism that enables precise localization of semantic targets on aerial maps. To support real-time adaptability, we introduce a reactive thinking loop, allowing agents to iteratively reflect on observations, revise mission goals, and coordinate dynamically in evolving environments. UAV-CodeAgents is evaluated on large-scale mission scenarios involving industrial and environmental fire detection. Our results show that a lower decoding temperature (0.5) yields higher planning reliability and reduced execution time, with an average mission creation time of 96.96 seconds and a success rate of 93%. We further fine-tune Qwen2.5VL-7B on 9,000 annotated satellite images, achieving strong spatial grounding across diverse visual categories. To foster reproducibility and future research, we will release the full codebase and a novel benchmark dataset for vision-language-based UAV planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。