无人机与地面机器人协作，用自然语言完成未知环境下的任务。
Air-Ground Collaboration for Language-Specified Missions in Unknown Environments
- 用大模型解析语言指令，动态构建语义地图并共享信息。
- 在城乡环境中实现千米级导航，7种语言指令均成功执行。
- 适合多智能体协同、户外自主导航研究者参考。
随着自主机器人系统日益成熟，用户将更倾向于以意图而非低层细节来指定任务。语言是表达意图的直观方式。然而，实现语言引导的机器人团队面临重大技术挑战：需具备高级语义推理能力，异构机器人需在不同视角间有效协调并共享信息。此外，机器人间通信通常中断频繁，需利用通信机会制定鲁棒策略以维持协作并达成目标。本文提出首个此类系统，使无人飞行器（UAV）与无人地面车（UGV）能在未知环境中协作完成自然语言指定的任务，并实时响应指令变化。系统采用大语言模型（LLM）驱动的规划器，基于在线构建的语义-度量地图，由空地机器人间按需共享。实验涵盖城市与乡村区域的任务驱动导航，在七种自然语言指令下实现千米尺度路径规划，验证了系统在复杂场景中的有效性。
原文摘要 · Abstract (English)
As autonomous robotic systems become increasingly mature, users will want to specify missions at the level of intent rather than in low-level detail. Language is an expressive and intuitive medium for such mission specification. However, realizing language-guided robotic teams requires overcoming significant technical hurdles. Interpreting and realizing language-specified missions requires advanced semantic reasoning. Successful heterogeneous robots must effectively coordinate actions and share information across varying viewpoints. Additionally, communication between robots is typically intermittent, necessitating robust strategies that leverage communication opportunities to maintain coordination and achieve mission objectives. In this work, we present a first-of-its-kind system where an unmanned aerial vehicle (UAV) and an unmanned ground vehicle (UGV) are able to collaboratively accomplish missions specified in natural language while reacting to changes in specification on the fly. We leverage a Large Language Model (LLM)-enabled planner to reason over semantic-metric maps that are built online and opportunistically shared between an aerial and a ground robot. We consider task-driven navigation in urban and rural areas. Our system must infer mission-relevant semantics and actively acquire information via semantic mapping. In both ground and air-ground teaming experiments, we demonstrate our system on seven different natural-language specifications at up to kilometer-scale navigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。