统一视觉导航世界模型，让智能体像人一样预判未来并规划行动。
Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight
- 用统一的自回归架构融合视觉预测与路径规划，动作选择基于想象结果。
- 在4个基准上成功率最高提升30%，零样本迁移至未见数据集表现优异。
- 适合做具身智能、机器人导航和需要长期推理的任务研究者。
让具身智能体具备想象未来状态的能力，是实现鲁棒且通用视觉导航的关键。现有先进系统多采用模块化设计，将导航规划与视觉世界建模分离，常导致状态-动作错配，在新场景或动态环境中适应性差。本文提出UniWM——一种统一的、记忆增强的世界模型，将自我中心视觉预见与规划整合到单一多模态自回归主干网络中。UniWM明确将动作选择锚定在视觉想象的结果上，使预测与控制紧密对齐。同时，分层记忆机制融合短期感知线索与长期轨迹上下文，支持长时程稳定连贯的推理。在四个挑战性基准（Go Stanford, ReCon, SCAND, HuRoN）及1X Humanoid Dataset上的大量实验表明，UniWM将导航成功率最高提升30%，显著降低轨迹误差，零样本泛化至未见的TartanDrive数据集，并自然扩展至高维人形机器人导航。这些结果标志着迈向统一、想象驱动的具身导航的重要一步。代码与模型已公开于https://github.com/UWMILab/UniWM。
原文摘要 · Abstract (English)
Enabling embodied agents to imagine future states is essential for robust and generalizable visual navigation. Yet, state-of-the-art systems typically rely on modular designs that decouple navigation planning from visual world modeling, which often induces state-action misalignment and weak adaptability in novel or dynamic scenarios. We propose UniWM, a unified, memory-augmented world model that integrates egocentric visual foresight and planning within a single multimodal autoregressive backbone. UniWM explicitly grounds action selection in visually imagined outcomes, tightly aligning prediction with control. Meanwhile, a hierarchical memory mechanism fuses short-term perceptual cues with longer-term trajectory context, supporting stable and coherent reasoning over extended horizons. Extensive experiments on four challenging benchmarks (Go Stanford, ReCon, SCAND, HuRoN) and the 1X Humanoid Dataset show that UniWM improves navigation success rates by up to 30%, substantially reduces trajectory errors against strong baselines, generalizes zero-shot to the unseen TartanDrive dataset, and scales naturally to high-dimensional humanoid navigation. These results position UniWM as a principled step toward unified, imagination-driven embodied navigation. The code and models are available at https://github.com/UWMILab/UniWM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。