让智能体同时规划路径并预判未来,提升长距离导航成功率。
NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction
- 用统一模型分解任务、制定子目标并预测环境变化。
- 在R2R-CE和RxR-CE上达到领先性能,显著降低失败率。
- 适合研究具身智能、导航与多模态决策的学者参考。
面向复杂自然语言指令的长程具身导航仍是人工智能的重大挑战。现有智能体在未知环境中的长期规划能力不足,导致高失败率。为此,我们提出NavForesee——一种统一视觉-语言模型,将高层语言规划与生成式世界模型预测融合于同一框架。该模型基于完整指令与历史观测,既能分解任务、跟踪进展、生成子目标,又能预测短时环境动态与长程导航里程碑。结构化计划引导精准预测,想象的未来为行动提供丰富上下文,形成感知-规划/预测-行动的闭环反馈。在R2R-CE和RxR-CE基准上的大量实验表明,NavForesee在复杂场景中表现优异,凸显了显式语言规划与隐式时空预测融合的巨大潜力,推动更智能具身代理的发展。
原文摘要 · Abstract (English)
Embodied navigation for long-horizon tasks, guided by complex natural language instructions, remains a formidable challenge in artificial intelligence. Existing agents often struggle with robust long-term planning about unseen environments, leading to high failure rates. To address these limitations, we introduce NavForesee, a novel Vision-Language Model (VLM) that unifies high-level language planning and predictive world model imagination within a single, unified framework. Our approach empowers a single VLM to concurrently perform planning and predictive foresight. Conditioned on the full instruction and historical observations, the model is trained to understand the navigation instructions by decomposing the task, tracking its progress, and formulating the subsequent sub-goal. Simultaneously, it functions as a generative world model, providing crucial foresight by predicting short-term environmental dynamics and long-term navigation milestones. The VLM's structured plan guides its targeted prediction, while the imagined future provides rich context to inform the navigation actions, creating a powerful internal feedback loop of perception-planning/prediction-action. We demonstrate through extensive experiments on the R2R-CE and RxR-CE benchmark that NavForesee achieves highly competitive performance in complex scenarios. Our work highlights the immense potential of fusing explicit language planning with implicit spatiotemporal prediction, paving the way for more intelligent and capable embodied agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。