arXiv:2510.20310cs.AI2025-10被引 5

让智能体用工具分步推理,更准更快答出环境问题。

Multi-Step Reasoning for Embodied Question Answering via Tool Augmentation

  • 引入外部工具辅助分步思考,指导探索方向
  • 在新场景中成功率达87.6%,比基线高20.2%
  • 适合需要精准导航与决策的机器人研究者

具身问答(EQA)要求智能体在3D环境中探索并回答场景相关问题。现有方法依赖视觉语言模型直接探索和作答,缺乏显式推理与规划,导致探索效率低、回应无效。本文提出ToolEQA,通过整合外部工具与多步推理,使智能体能获取额外信息,优化下一步探索路径,从而以更短距离获得有效信息并提高回答准确率。为增强工具使用与多步推理能力,我们设计了自动化数据生成流程,构建了包含约1.8万任务的EQA-RT数据集,分为训练集EQA-RT-Train及两个测试集:与训练集重叠的EQA-RT-Seen和全新场景的EQA-RT-Unseen。在两个测试集上,ToolEQA相比顶尖基线成功率提升9.2~20.2%,且比零样本版本高出10%。同时在HM-EQA、OpenEQA和EXPRESS-Bench上也达到领先水平,验证了其通用性。

原文摘要 · Abstract (English)

Embodied Question Answering (EQA) requires agents to explore 3D environments to obtain observations and answer questions related to the scene. Existing methods leverage VLMs to directly explore the environment and answer questions without explicit thinking or planning, which limits their reasoning ability and results in excessive or inefficient exploration as well as ineffective responses. In this paper, we introduce ToolEQA, an agent that integrates external tools with multi-step reasoning, where external tools can provide more useful information for completing the task, helping the model derive better exploration directions in the next step of reasoning and thus obtaining additional effective information. This enables ToolEQA to generate more accurate responses with a shorter exploration distance. To enhance the model's ability for tool-usage and multi-step reasoning, we further design a novel EQA data generation pipeline that automatically constructs large-scale EQA tasks with reasoning trajectories and corresponding answers. Based on the pipeline, we collect the EQA-RT dataset that contains about 18K tasks, divided into a training set EQA-RT-Train, and two test sets EQA-RT-Seen (scenes overlapping with the training set) and EQA-RT-Unseen (novel scenes). Experiments on EQA-RT-Seen and EQA-RT-Unseen show that ToolEQA improves the success rate by 9.2~20.2% over state-of-the-art baselines, while outperforming the zero-shot ToolEQA by 10% in success rate. In addition, ToolEQA also achieves state-of-the-art performance on the HM-EQA, OpenEQA, and EXPRESS-Bench datasets, demonstrating its generality. Our homepage see https://tooleqa.github.io.

具身问答多步推理工具增强智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。