arXiv:2603.05642cs.ROcs.AI2026-03

用3D场景图+关系推理,实现高效开放世界物品搜索

Relational Semantic Reasoning on 3D Scene Graphs for Open World Interactive Object Search

  • 基于3D场景图,用包含与共现等关系规则评分引导探索
  • 在仿真和真实环境均超越传统相似度方法,接近大模型性能
  • 轻量模型可部署于机器人,适合实际交互任务

家庭环境中开放世界交互式物品搜索需要理解物体间的语义关系及其上下文以高效引导探索。以往方法依赖视觉-语言嵌入相似性,难以捕捉任务相关的结构化关系;或使用大型语言模型(LLMs),但实时部署成本过高。本文提出SCOUT:一种基于场景图的探索框架,通过房间-物体包含、物体-物体共现等关系启发式规则为房间、前沿和物体分配效用分值。为兼顾开放词汇泛化与效率,设计离线过程蒸馏框架,将LLM中的结构化关系知识提取至轻量模型用于机器人端推理。同时构建SymSearch,一个可扩展的符号化基准,用于评估交互式物品搜索中的语义推理能力。大量实验表明,SCOUT在符号化与仿真环境中优于基于嵌入相似性的方法,并达到与LLM相当的性能,同时计算开销显著更低。真实世界测试验证了其在现实感知与导航约束下的有效迁移能力。

原文摘要 · Abstract (English)

Open-world interactive object search in household environments requires understanding semantic relationships between objects and their surrounding context to guide exploration efficiently. Prior methods either rely on vision-language embeddings similarity, which does not reliably capture task-relevant relational semantics, or large language models (LLMs), which are too slow and costly for real-time deployment. We introduce SCOUT: Scene Graph-Based Exploration with Learned Utility for Open-World Interactive Object Search, a novel method that searches directly over 3D scene graphs by assigning utility scores to rooms, frontiers, and objects using relational exploration heuristics such as room-object containment and object-object co-occurrence. To make this practical without sacrificing open-vocabulary generalization, we propose an offline procedural distillation framework that extracts structured relational knowledge from LLMs into lightweight models for on-robot inference. Furthermore, we present SymSearch, a scalable symbolic benchmark for evaluating semantic reasoning in interactive object search tasks. Extensive evaluations across symbolic and simulation environments show that SCOUT outperforms embedding similarity-based methods and matches LLM-level performance while remaining computationally efficient. Finally, real-world experiments demonstrate effective transfer to physical environments, enabling open-world interactive object search under realistic sensing and navigation constraints.

3D场景图关系推理机器人搜索轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。