用多智能体系统实现分步推理，提升场景图问答与规划性能
Schema-Guided Scene-Graph Reasoning based on Multi-Agent Large Language Model System
- 分两模块协作：推理者规划任务，检索者按指令提取信息
- 在多个模拟环境中，问答与规划任务表现优于现有方法
- 基于场景图模式引导，减少幻觉，避免输入完整图数据
场景图已成为大型语言模型（LLM）进行具身空间推理的结构化表示。本文提出基于多智能体大语言模型系统的迭代式场景图推理框架SG²。该框架将智能体分为两个模块：(1) 推理模块负责抽象任务规划和生成图信息查询；(2) 检索模块根据代码编写指令执行信息提取。两个模块协同迭代，实现顺序推理并自适应关注图信息。场景图模式被提示给两个模块，不仅简化推理与检索流程，还引导模块间协作。该设计无需向LLM提示完整图数据，降低因无关信息导致的幻觉风险。在多个仿真环境中的实验表明，该框架在数值问答与规划任务中超越现有基于LLM的方法及基线单智能体工具式‘边推理边检索’策略。
原文摘要 · Abstract (English)
Scene graphs have emerged as a structured and serializable environment representation for grounded spatial reasoning with Large Language Models (LLMs). In this work, we propose SG^2, an iterative Schema-Guided Scene-Graph reasoning framework based on multi-agent LLMs. The agents are grouped into two modules: a (1) Reasoner module for abstract task planning and graph information queries generation, and a (2) Retriever module for extracting corresponding graph information based on code-writing following the queries. Two modules collaborate iteratively, enabling sequential reasoning and adaptive attention to graph information. The scene graph schema, prompted to both modules, serves to not only streamline both reasoning and retrieval process, but also guide the cooperation between two modules. This eliminates the need to prompt LLMs with full graph data, reducing the chance of hallucination due to irrelevant information. Through experiments in multiple simulation environments, we show that our framework surpasses existing LLM-based approaches and baseline single-agent, tool-based Reason-while-Retrieve strategy in numerical Q\&A and planning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。