arXiv:2505.15725cs.ROcs.CV2025-05NeurIPS被引 37

首个真实世界无人机语言操控基准,让无人机听懂指令精准飞行。

UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning

  • 用真人飞手轨迹+语言指令训练无人机模仿精细飞行行为
  • 在多样真实环境收集数据,实现无仿真到现实的直接部署
  • 验证视觉语言模型在细微动作控制中的优势,适合智能飞行研究者

无人飞行器正向语言交互平台演进,实现更直观的人机互动。现有工作多关注高层规划与长程导航,本文聚焦语言引导的细粒度轨迹控制——即无人机对语言指令做出短距离、反应式飞行响应。我们提出「随词而飞(Flying-on-a-Word, Flow)」任务,并引入模仿学习作为有效解决方案:通过匹配专家飞手轨迹与原子级语言指令,训练无人机学习精细控制策略。为此,我们构建了首个真实世界下的细粒度语言控制基准 UAV-Flow,包含任务定义、大规模真实场景数据集、可部署控制框架及仿真评估套件。该设计使无人机能精准复现人类专家飞行轨迹,支持直接部署且无仿真到现实的差距。我们在 UAV-Flow 上进行了广泛实验,对比了视觉语言导航(VLN)与视觉语言动作(VLA)范式。结果表明,VLA 模型优于 VLN 基线,并凸显空间定位在细粒度飞行设置中的关键作用。

原文摘要 · Abstract (English)

Unmanned Aerial Vehicles (UAVs) are evolving into language-interactive platforms, enabling more intuitive forms of human-drone interaction. While prior works have primarily focused on high-level planning and long-horizon navigation, we shift attention to language-guided fine-grained trajectory control, where UAVs execute short-range, reactive flight behaviors in response to language instructions. We formalize this problem as the Flying-on-a-Word (Flow) task and introduce UAV imitation learning as an effective approach. In this framework, UAVs learn fine-grained control policies by mimicking expert pilot trajectories paired with atomic language instructions. To support this paradigm, we present UAV-Flow, the first real-world benchmark for language-conditioned, fine-grained UAV control. It includes a task formulation, a large-scale dataset collected in diverse environments, a deployable control framework, and a simulation suite for systematic evaluation. Our design enables UAVs to closely imitate the precise, expert-level flight trajectories of human pilots and supports direct deployment without sim-to-real gap. We conduct extensive experiments on UAV-Flow, benchmarking VLN and VLA paradigms. Results show that VLA models are superior to VLN baselines and highlight the critical role of spatial grounding in the fine-grained Flow setting.

无人机控制语言交互模仿学习真实世界

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。