arXiv:2512.18571cs.AIcs.CV2025-12被引 1

让机器人智能权衡探索与问人成本,高效完成寻物任务。

ESearch-R1: Learning Cost-Aware MLLM Agents for Interactive Embodied Search via Reinforcement Learning

  • 将问话、回忆、导航统一为决策流程,主动控制成本。
  • 任务成功率提升,总操作成本降低约50%。
  • 适合需要高效交互的机器人应用,如家庭服务。

多模态大语言模型(MLLM)赋予了具身智能体强大的规划与推理能力。然而,在面对模糊指令(如在杂乱房间中‘拿工具’)时,现有智能体难以平衡物理探索的高成本与人际互动的认知成本。它们通常将澄清问题视为被动感知,缺乏战略性推理以最小化总执行成本。为此,我们提出ESearch-R1,一个统一交互对话(Ask)、情景记忆检索(GetMemory)和物理导航(Navigate)的代价感知具身推理框架。引入HC-GRPO(异质成本感知组相对策略优化),不依赖独立价值评判器,通过采样推理轨迹组,强化在信息增益与异质成本(如导航时间、人类注意力)间取得最优权衡的策略。在AI2-THOR中的大量实验表明,ESearch-R1显著优于标准的ReAct基线代理,任务成功率达更高,总操作成本降低约50%,验证了GRPO在对齐MLLM智能体与物理世界约束方面的有效性。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) have empowered embodied agents with remarkable capabilities in planning and reasoning. However, when facing ambiguous natural language instructions (e.g., "fetch the tool" in a cluttered room), current agents often fail to balance the high cost of physical exploration against the cognitive cost of human interaction. They typically treat disambiguation as a passive perception problem, lacking the strategic reasoning to minimize total task execution costs. To bridge this gap, we propose ESearch-R1, a cost-aware embodied reasoning framework that unifies interactive dialogue (Ask), episodic memory retrieval (GetMemory), and physical navigation (Navigate) into a single decision process. We introduce HC-GRPO (Heterogeneous Cost-Aware Group Relative Policy Optimization). Unlike traditional PPO which relies on a separate value critic, HC-GRPO optimizes the MLLM by sampling groups of reasoning trajectories and reinforcing those that achieve the optimal trade-off between information gain and heterogeneous costs (e.g., navigate time, and human attention). Extensive experiments in AI2-THOR demonstrate that ESearch-R1 significantly outperforms standard ReAct-based agents. It improves task success rates while reducing total operational costs by approximately 50\%, validating the effectiveness of GRPO in aligning MLLM agents with physical world constraints.

具身智能多模态模型强化学习成本优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。