用自然语言控制模拟无人机完成多任务，降低操作门槛。
Large Language Models to Enhance Multi-task Drone Operations in Simulated Environments
- 用微调后的CodeT5将自然语言指令自动转为无人机可执行代码。
- 在AirSim模拟环境中实现高效多任务执行，理解能力更强。
- 适合想快速测试无人机应用的开发者和研究人员。
得益于大语言模型的快速发展，人机协同飞行迎来新机遇。本文提出一种方法，将微调后的CodeT5模型与基于Unreal Engine的AirSim无人机模拟器结合，通过自然语言命令高效执行多任务操作。用户可通过提示词或命令描述与模拟无人机交互，轻松获取并控制无人机状态，显著降低操作门槛。在AirSim中可灵活构建视觉逼真的动态环境，模拟复杂场景下的无人机应用。利用由ChatGPT生成的大规模（自然语言，程序代码）指令-执行对数据集，结合开发者编写的无人机代码作为训练数据，对CodeT5进行微调，实现自然语言到可执行代码的自动化转换。实验表明，该方法在模拟环境中展现出优异的任务执行效率和命令理解能力。未来计划以模块化方式扩展模型功能，提升其在复杂场景中的适应性，推动无人机技术向真实环境应用迈进。
原文摘要 · Abstract (English)
Benefiting from the rapid advancements in large language models (LLMs), human-drone interaction has reached unprecedented opportunities. In this paper, we propose a method that integrates a fine-tuned CodeT5 model with the Unreal Engine-based AirSim drone simulator to efficiently execute multi-task operations using natural language commands. This approach enables users to interact with simulated drones through prompts or command descriptions, allowing them to easily access and control the drone's status, significantly lowering the operational threshold. In the AirSim simulator, we can flexibly construct visually realistic dynamic environments to simulate drone applications in complex scenarios. By combining a large dataset of (natural language, program code) command-execution pairs generated by ChatGPT with developer-written drone code as training data, we fine-tune the CodeT5 to achieve automated translation from natural language to executable code for drone tasks. Experimental results demonstrate that the proposed method exhibits superior task execution efficiency and command understanding capabilities in simulated environments. In the future, we plan to extend the model functionality in a modular manner, enhancing its adaptability to complex scenarios and driving the application of drone technologies in real-world environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。