通用智能体无需导航策略,仅靠视觉与离散动作就能实现媲美工业级的导航能力。
Embodied Agents Take Control: Minimal-Interface Zero-Shot Agents Rival Industrial-Scale Policies in Vision-and-Language Navigation

- 让大模型直接控制每一步行动,实现感知-决策-纠错闭环
- 零样本导航成功率最高达78%,比现有方法更省时间与步骤
- 适合追求轻量级、通用性智能体的开发者与研究者
自主具身智能体需在多个步骤中持续完成感知、执行、验证与自我修正的决策循环。当前系统依赖任务特定流程或具身策略,灵活性差且错误时缺乏有效恢复机制。本文发现,通用型智能体可自行维持该循环,称之为‘代理式具身控制’:推理模型直接驱动每一步动作,确保推理与控制对齐。以零样本导航为测试场景,我们仅用单目摄像头和离散动作,为三个编码器-智能体框架配备通用模型。默认努力下,Opus-5平均成功率达70.7±3.5%,最大努力下Fable-5达到78%。当引入训练好的路径点工具作为辅助,混合式Fable-5在默认努力下达成76.7±0.6%成功率,环境步数减半,耗时不足四分之一。消融实验表明,模型选择主导性能差异,框架差异较小;强制路径点对弱模型有帮助,但可能拖累强模型。尽管长视野、延迟与上下文增长仍是持续自主的障碍,结果表明通用模型已能在无导航策略前提下实现竞争力的具身控制。
原文摘要 · Abstract (English)
Autonomous embodied agents must sustain a long decision-making loop that involves perceiving, acting, verifying, and self-correcting over many steps. Current systems sustain this loop through task-specific workflows or embodied policies. However, these fixed workflows and policies offer limited flexibility across environments and often lack effective recovery strategies when execution goes wrong. We find that a general-purpose agent can instead sustain the loop on its own. We term this organization agentic embodied control: the reasoning model directly steers every action, keeping reasoning and control aligned. Using zero-shot navigation as a controlled testbed, we equip three coding-agent harnesses with only a monocular RGB camera and discrete actions. At default effort, replicated opus-5 runs average $70.7\pm3.5$% success, while fable-5 reaches 78% at maximum effort. When a trained waypoint tool is offered alongside primitives, the hybrid fable-5 agent reaches $76.7\pm0.6$% at default effort, using half the environment steps and under a quarter of the wall time. Across the ablations, model choice dominates performance variation. Observed harness differences are modest, and forced waypoints help weaker models but can hinder stronger ones. Although longer horizons, latency, and context growth remain barriers to sustained autonomy, these results show that a general-purpose model can already achieve competitive embodied control without a navigation policy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。