用语义图结构让机器人团队更聪明地按指令找东西
Semantic Area Graph Reasoning for Multi-Robot Language-Guided Search

- 构建环境的语义区域图,把房间、连接关系等压缩成语言模型能理解的形式
- 在100个场景中,找特定物品效率提升最高达18.8%,尤其在大环境中表现更好
- 适合需要语义理解的多机器人搜索任务,如按房间类型找物品
协调多机器人系统(MRS)在未知环境中进行探索,对需要超越几何探索的语义推理任务尤为困难。传统策略依赖前沿覆盖或信息增益,无法融入高级任务意图,例如寻找与特定房间类型相关的物体。我们提出语义区域图推理(SAGR),一种分层框架,使大型语言模型(LLMs)通过环境的结构化语义拓扑抽象来协调多机器人探索与语义搜索。SAGR从语义占据图增量构建语义区域图,编码房间实例、连通性、前沿可用性和机器人状态,形成紧凑的任务相关表示供LLM推理。LLM基于空间结构和任务上下文进行高层次的语义房间分配,而确定性前沿规划与局部导航负责执行几何任务。在Habitat-Matterport3D数据集上100个场景的实验表明,SAGR在保持与最先进探索方法竞争力的同时,持续提升语义目标搜索效率,在大环境中最高提升18.8%。结果凸显了结构化语义抽象作为LLM推理与多机器人协调之间有效接口的价值。
原文摘要 · Abstract (English)
Coordinating multi-robot systems (MRS) to search in unknown environments is particularly challenging for tasks that require semantic reasoning beyond geometric exploration. Classical coordination strategies rely on frontier coverage or information gain and cannot incorporate high-level task intent, such as searching for objects associated with specific room types. We propose \textit{Semantic Area Graph Reasoning} (SAGR), a hierarchical framework that enables Large Language Models (LLMs) to coordinate multi-robot exploration and semantic search through a structured semantic-topological abstraction of the environment. SAGR incrementally constructs a semantic area graph from a semantic occupancy map, encoding room instances, connectivity, frontier availability, and robot states into a compact task-relevant representation for LLM reasoning. The LLM performs high-level semantic room assignment based on spatial structure and task context, while deterministic frontier planning and local navigation handle geometric execution within assigned rooms. Experiments on the Habitat-Matterport3D dataset across 100 scenarios show that SAGR remains competitive with state-of-the-art exploration methods while consistently improving semantic target search efficiency, with up to 18.8\% in large environments. These results highlight the value of structured semantic abstractions as an effective interface between LLM-based reasoning and multi-robot coordination in complex indoor environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。