arXiv:2602.12971cs.RO2026-02被引 3

让机器人持续理解复杂指令,支持带否定和多步空间约束的语义查询。

INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval

  • 分双流异步更新,用自然语言摘要构建可检索的知识库。
  • 在真实环境与新基准上实现最先进性能,尤其擅长处理否定和链式空间条件。
  • 结合大模型推理与场景图结构,通过视觉验证减少误检,结果可解释。

受基础模型进展推动,语义场景图已成为机器人导航中高层3D环境抽象的有前景范式。然而,现有框架难以有效处理复杂具身查询,同时保障连续语义图构建。为此,我们提出INHerit-SG,一种异步双流架构,将3D环境系统性地组织为支持RAG风格检索的知识库。该框架整合了全面的节点表征、事件触发的异步更新机制以及结构化检索方法。几何分割与语义推理解耦以保持建图效率,语义节点还存储自然语言摘要以支持文本检索。此外,我们提出一个可解释的检索流程,将多角色大语言模型的推理能力与场景图拓扑结构相结合,并通过视觉验证过程降低误报率。我们在新构建的复杂具身语义查询检索基准HM3DSem-SQR及真实环境上评估INHerit-SG,实验表明,该系统在复杂查询上达到最先进性能,尤其在涉及否定和链式空间约束时表现突出。

原文摘要 · Abstract (English)

Driven by recent advancements in foundation models, semantic scene graphs have emerged as a promising paradigm for high-level 3D environmental abstraction in robot navigation. However, existing frameworks struggle to successfully handle complex embodied queries while ensuring continuous semantic graph construction. To address these limitations, we present INHerit-SG, an asynchronous dual-stream architecture that systematically structures the 3D environment into a RAG-ready knowledge base. Specifically, our framework integrates comprehensive node representations, an event-triggered asynchronous update scheme, and a structured retrieval mechanism. While geometric segmentation is decoupled from semantic reasoning to maintain mapping efficiency, the semantic nodes also store natural language summaries to support text-based retrieval. Furthermore, we propose an interpretable retrieval pipeline that couples the reasoning capabilities of multi-role LLMs with the topological structure of the scene graph, followed by a visual verification process to mitigate false positives. We evaluate INHerit-SG on a newly constructed benchmark for complex embodied semantic query retrieval, HM3DSem-SQR, and in real-world environments. Experiments demonstrate that our system achieves state-of-the-art performance on complex queries, especially for those involving negations and chained spatial constraints. Project Page: https://fangyuktung.github.io/INHeritSG.github.io/

语义图机器人导航大模型检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。