arXiv:2510.26915cs.ROcs.AI2025-10被引 4

让异构机器人在未知环境中通过语言指令自主协作,成功率超87%。

Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence

  • 用大模型生成可执行的任务,并结合机器人能力动态验证与调整。
  • 仿真中成功率接近两倍于现有方法,真实场景达87%成功。
  • 适合需语言指令、在线反馈与多类型机器人协同的复杂任务。

在未结构化环境中,异构机器人团队常需完成复杂任务,依赖在线获取的信息进行协作与适应。由于环境缺乏先验地图,任务必须基于机器人能力与物理世界进行具身化定义。传统异构团队设计固定,而生成式智能使团队能理解自然语言任务并灵活应对。然而,现有基于大语言模型(LLM)的协作方法通常假设环境结构良好且已知,限制了在未结构化环境中的应用。本文提出SPINE-HT框架,通过三阶段流程将LLM推理能力与异构机器人团队上下文对齐:给定任务目标与团队能力的语言描述后,LLM生成具身子任务并验证可行性;子任务根据机器人能力(如移动性或感知)分配,并在在线运行中接收反馈进行优化。仿真实验中,闭环感知与控制下,该框架成功率接近现有方法的两倍;真实实验中,使用Clearpath Jackal、Husky、Boston Dynamics Spot及高空无人机,任务涉及能力推理与在线反馈修正,整体成功率高达87%。

原文摘要 · Abstract (English)

Heterogeneous robot teams operating in realistic settings often must accomplish complex missions requiring collaboration and adaptation to information acquired online. Because robot teams frequently operate in unstructured environments -- uncertain, open-world settings without prior maps -- subtasks must be grounded in robot capabilities and the physical world. While heterogeneous teams have typically been designed for fixed specifications, generative intelligence opens the possibility of teams that can accomplish a wide range of missions described in natural language. However, current large language model (LLM)-enabled teaming methods typically assume well-structured and known environments, limiting deployment in unstructured environments. We present SPINE-HT, a framework that addresses these limitations by grounding the reasoning abilities of LLMs in the context of a heterogeneous robot team through a three-stage process. Given language specifications describing mission goals and team capabilities, an LLM generates grounded subtasks which are validated for feasibility. Subtasks are then assigned to robots based on capabilities such as traversability or perception and refined given feedback collected during online operation. In simulation experiments with closed-loop perception and control, our framework achieves nearly twice the success rate compared to prior LLM-enabled heterogeneous teaming approaches. In real-world experiments with a Clearpath Jackal, a Clearpath Husky, a Boston Dynamics Spot, and a high-altitude UAV, our method achieves an 87\% success rate in missions requiring reasoning about robot capabilities and refining subtasks with online feedback. More information is provided at https://zacravichandran.github.io/SPINE-HT.

机器人协作生成智能异构系统语言指令

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。