用通用接口让大模型直接操控无人机,自然语言变飞行指令。
A Universal Large Language Model -- Drone Command and Control Interface
- 设计通用接口协议(MCP),兼容任意大模型和无人机
- 实测可控制真实无人机,模拟中实现实时导航规划
- 适合想快速接入大模型的无人机开发者与研究者
将人工智能用于无人机控制具有变革性潜力,尤其当能融合现实世界信息与无人机感知、指挥和控制时。大型语言模型(LLMs)若在广泛知识上训练,尤其是包含全球详细地理拓扑和实时天气等数据,则更具优势。但现有方法在大模型与无人机之间存在接口壁垒,每个应用都需大量人工开发连接。本文提出一种无需依赖特定大模型或无人机的通用接口策略,首次实现通用、灵活、全面且易用的无人机控制界面。通过新提出的模型上下文协议(MCP)标准,实现AI系统对外部数据、工具和服务的统一访问。我们部署基于Linux的云服务器,运行支持Mavlink协议的MCP服务端,该协议被数百万无人机广泛使用,包括Ardupilot和PX4框架。实验验证了真实无人机的飞行控制,并在模拟环境中展示了与谷歌地图MCP服务集成的飞行规划与控制能力,可获取实时更新的导航信息。这为大模型与无人机控制的融合提供了通用范式,使现代人工智能技术能以直观自然语言方式赋能无人机系统。
原文摘要 · Abstract (English)
The use of artificial intelligence (AI) for drone control can have a transformative impact on drone capabilities, especially when real world information can be integrated with drone sensing, command, and control, part of a growing field of physical AI. Large language models (LLMs) can be advantageous if trained at scale on general knowledge, but especially and in particular when the training data includes information such as detailed map geography topology of the entire planet, as well as the ability to access real time situational data such as weather. However, challenges remain in the interface between drones and LLMs in general, with each application requiring a tedious, labor intensive effort to connect the LLM trained knowledge to drone command and control. Here, we solve that problem, using an interface strategy that is LLM agnostic and drone agnostic, providing the first universal, versatile, comprehensive and easy to use drone control interface. We do this using the new model context protocol (MCP) standard, an open standard that provides a universal way for AI systems to access external data, tools, and services. We develop and deploy a cloud based Linux machine hosting an MCP server that supports the Mavlink protocol, an ubiquitous drone control language used almost universally by millions of drones including Ardupilot and PX4 framework.We demonstrate flight control of a real unmanned aerial vehicle. In further testing, we demonstrate extensive flight planning and control capability in a simulated drone, integrated with a Google Maps MCP server for up to date, real time navigation information. This demonstrates a universal approach to integration of LLMs with drone command and control, a paradigm that leverages and exploits virtually all of modern AI industry with drone technology in an easy to use interface that translates natural language to drone control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。