用场景图引导大模型澄清模糊指令，提升机器人规划可靠性。
SG-CoT: An Ambiguity-Aware Robotic Planning Framework using Scene Graph Representations
- 大模型通过查询环境场景图，逐步识别并澄清任务中的歧义
- 单智能体下问答准确率提升至少10%，多智能体成功率达15%以上
- 适合需要高鲁棒性交互的复杂场景机器人系统
大语言模型在作为机器人规划器时面临歧义挑战。本文提出场景图-思维链（SG-CoT）框架，分两阶段运行：首先从观测构建包含物体、属性及关系的结构化场景图；其次，大模型调用检索函数查询与指令相关的场景图部分，使推理基于实际观察，增强规划可靠性。该方法可定位歧义来源，并向用户或另一机器人提出针对性澄清问题。大量实验表明，SG-CoT在单智能体环境中问答准确率最低提升10%，多智能体环境下成功率提升至少15%，验证了其在更通用机器人规划中的有效性。
原文摘要 · Abstract (English)
Ambiguity poses a major challenge to large language models (LLMs) used as robotic planners. In this letter, we present Scene Graph-Chain-of-Thought (SG-CoT), a two-stage framework where LLMs iteratively query a scene graph representation of the environment to detect and clarify ambiguities. First, a structured scene graph representation of the environment is constructed from input observations, capturing objects, their attributes, and relationships with other objects. Second, the LLM is equipped with retrieval functions to query portions of the scene graph that are relevant to the provided instruction. This grounds the reasoning process of the LLM in the observation, increasing the reliability of robotic planners under ambiguous situations. SG-CoT also allows the LLM to identify the source of ambiguity and pose a relevant disambiguation question to the user or another robot. Extensive experimentation demonstrates that SG-CoT consistently outperforms prior methods, with a minimum of 10% improvement in question accuracy and a minimum success rate increase of 4% in single-agent and 15% in multi-agent environments, validating its effectiveness for more generalizable robot planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。