arXiv:2512.08145cs.ROcs.AI2025-12被引 1

用大模型让无人机听懂人话,实现自然对话式操控。

Chat with UAV -- Human-UAV Interaction Based on Large Language Models

  • 设计双智能体架构,分别负责任务规划与执行
  • 在四类典型场景中提升交互流畅度与任务灵活性
  • 支持用户自定义指令,适合非专业人员使用

未来无人机交互系统正从工程师主导转向用户驱动,旨在替代传统预设的人机交互模式。该转变强调个性化任务规划与设计,以提升交互质量与灵活性,适用于农业、航拍、物流及环境监测等领域。然而,由于用户与无人机之间缺乏通用语言,此类交互难以实现。大型语言模型具备理解自然语言与机器人行为的能力,为个性化人机交互提供了可能。尽管已有基于大模型的交互框架提出,但普遍存在多任务规划与执行困难的问题,复杂场景适应性差。本文提出一种新型双智能体人机交互框架,构建两个独立的大型语言模型代理(任务规划代理与执行代理),通过不同的提示工程分别处理任务的理解、规划与执行。为验证框架有效性与性能,我们建立了涵盖四类典型无人机应用场景的任务数据库,并采用三个独立指标量化评估。同时选用不同大模型控制无人机进行对比实验。用户研究结果表明,该框架显著提升了交互的平滑性与任务执行的灵活性,有效满足了用户的个性化需求。

原文摘要 · Abstract (English)

The future of UAV interaction systems is evolving from engineer-driven to user-driven, aiming to replace traditional predefined Human-UAV Interaction designs. This shift focuses on enabling more personalized task planning and design, thereby achieving a higher quality of interaction experience and greater flexibility, which can be used in many fileds, such as agriculture, aerial photography, logistics, and environmental monitoring. However, due to the lack of a common language between users and the UAVs, such interactions are often difficult to be achieved. The developments of Large Language Models possess the ability to understand nature languages and Robots' (UAVs') behaviors, marking the possibility of personalized Human-UAV Interaction. Recently, some HUI frameworks based on LLMs have been proposed, but they commonly suffer from difficulties in mixed task planning and execution, leading to low adaptability in complex scenarios. In this paper, we propose a novel dual-agent HUI framework. This framework constructs two independent LLM agents (a task planning agent, and an execution agent) and applies different Prompt Engineering to separately handle the understanding, planning, and execution of tasks. To verify the effectiveness and performance of the framework, we have built a task database covering four typical application scenarios of UAVs and quantified the performance of the HUI framework using three independent metrics. Meanwhile different LLM models are selected to control the UAVs with compared performance. Our user study experimental results demonstrate that the framework improves the smoothness of HUI and the flexibility of task execution in the tasks scenario we set up, effectively meeting users' personalized needs.

人机交互大模型无人机

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。