将智能体能力拆解为可动态调用的工具,提升任务表现
Enabling Extensible Embodied Capabilities with Tools

- 把感知、推理、执行等能力做成独立工具,按需调用
- 在仿真和真实场景中平均性能提升31%~36%,认知类效果最好
- 当前模型仍难判断何时何地用什么工具,适合研究智能体工具使用
现有具身智能方法将感知、推理、规划与控制统一建模于单一策略中,但这些能力具有层次性与异质性,难以在单一模型中可靠学习与模块化。本文提出能力外化方法,将异构能力解耦为可独立优化的工具,在推理时动态调用。为此,我们设计具身工具协议(ETP),实现工具注册、发现、调用与执行的标准化,并构建包含100+经验证工具的工具库,覆盖感知、认知、推理与执行。基于此,我们建立EmbodiedToolBench,评估工具增强对具身性能的影响,以及模型在工具必要性识别、选择、执行与链式组合中的表现。跨仿真与真实平台实验表明,能力外化显著提升性能(EB-ALFRED平均提升31%,EB-Navigation提升36%),但执行类能力增益有限。分析揭示:所有模型均面临何时、何地、如何调用工具的挑战,凸显具身工具使用能力是未来研究关键方向。
原文摘要 · Abstract (English)
Most existing embodied intelligence methods formulate perception, reasoning, planning, and control within a unified parameterized policy. Yet these capabilities are inherently hierarchical and heterogeneous, making them difficult to reliably learn and modularize within a single model. We propose a capability externalization approach that decouples heterogeneous capabilities into independently optimized tools, dynamically invoked at inference time. To this end, we introduce Embodied Tool Protocol (ETP), a standardized protocol for embodied tool registration, discovery, invocation, and execution, and curate 100+ validated tools spanning perception, cognition, reasoning, and execution as the tool base. Building on this, we construct EmbodiedToolBench to evaluate both whether tool augmentation improves embodied performance and how well current models use tools across tool-necessity recognition, tool selection, tool execution, and tool-chain composition. Experiments across simulation and real-world platforms confirm that capability externalization consistently improves embodied performance (avg. gain 31% on EB-ALFRED and 36% on EB-Navigation), yet reveal a clear boundary: gains are substantial for cognition and perception but are limited for execution-type capabilities. Moreover, our analysis reveals that knowing when, which, and how to invoke tools remains a persistent challenge across all models, thereby highlighting embodied tool competence as a critical direction for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。