arXiv:2608.30396cs.AIcs.RO2026-08

用视觉语言模型和导航模型协同,实现长时程物理世界探索的稳定导航。

Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation

论文配图:Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation
图 1 · 摘自论文原文
  • 将VLM与NFM通过三通道协作,实现目标推理与闭环导航的联动。
  • 在HM-EQA上比基线提升14.9个百分点,任务越长优势越明显。
  • 适用于需持续决策的机器人导航场景,尤其适合复杂长程任务。

长时程物理世界智能体需在远距离目标下进行推理,同时保持可靠的闭环行为。当前基础模型分拆了这些能力:视觉语言模型(VLM)可推断缺失信息并调整高层计划,但在重复导航定位上脆弱且低效;导航基础模型(NFM)能稳健执行语义目标,但仅限于短时程片段,缺乏持续的任务级推理。本文提出NavMCP,一种代理式框架,将VLM推理代理与NFM执行器耦合,实现长时程探索。VLM决定需收集什么证据、去何处搜索、何时停止,而NFM将每个语义子目标转化为闭环导航动作。三类通道构建协作:意图将证据需求转为导航调用,观察将轨迹回放转化为源相关证据,记忆累积发现、负证据及未解目标。该设计使孤立导航回放变为持续具身交互,无需重新训练任一模型。在具身问答任务中,NavMCP在HM-EQA、MT-HM3D和EXPRESS-Bench上达到最先进水平。使用相同主干模型时,其在HM-EQA上较片段时间接口提升14.9个百分点。在Unitree Go2机器人上,成功率达78.3%,随着任务时程增长,其相对于最强基线的差距从10点扩大至45点。结果表明,将互补基础模型组合为长时程物理世界代理具有巨大潜力。

原文摘要 · Abstract (English)

Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior. Today's foundation models split these capabilities: vision-language models (VLMs) infer missing information and adapt high-level plans but remain brittle and inefficient at repeated navigation grounding, while navigation foundation models (NFMs) robustly execute semantic goals but operate as bounded episodes without persistent task-level reasoning. We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration. The VLM decides what evidence to seek, where to search, and when to stop, while the NFM grounds each semantic sub-goal into closed-loop navigation. Three channels structure their collaboration: intent translates evidence needs into navigation calls, observation converts rollouts into source-grounded trajectory evidence, and memory accumulates findings, negative evidence, and unresolved goals across calls. This design turns isolated navigation rollouts into persistent embodied interaction without retraining either model. On Embodied Question Answering, NavMCP achieves state-of-the-art results on HM-EQA, MT-HM3D, and EXPRESS-Bench. Under matched agent and executor backbones, it outperforms an episodic interface by 14.9 percentage points on HM-EQA. On a Unitree Go2, NavMCP reaches 78.3% success, with its margin over the strongest baseline growing from 10 to 45 points as the task horizon increases. These results demonstrate the potential of scaffolding complementary foundation models into long-horizon physical-world agents.

长程导航多模态协同具身智能代理框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。