让机器人在陌生环境自主导航,靠自己建图并预判视觉,不依赖人工地图。
SpatialAnt: Autonomous Zero-Shot Robot Navigation via Active Scene Reconstruction and Visual Anticipation
- 用自建的模糊地图做基础,通过物理定位恢复真实尺度。
- 在不完整地图上预演未来视野,提前排除错误路径。
- 适合真实机器人部署,实测成功率达52%以上。
视觉-语言导航(VLN)近期受益于多模态大语言模型(MLLM),实现了零样本导航。现有基于探索的零样本方法虽借助全局场景先验取得良好效果,但依赖高质量的人工构建场景重建,难以用于真实机器人部署。面对未知环境时,机器人应通过预探索自行建立先验。然而,自建重建不可避免地存在不完整与噪声问题,严重损害对高质量重建有依赖的方法。为此,我们提出SpatialAnt框架,旨在弥合不完美自建重建与鲁棒执行之间的差距。SpatialAnt引入物理定位策略,恢复单目重建的绝对度量尺度;此外,不将噪声点云视为绝对空间参考,而是提出一种新型视觉预见机制,利用噪声点云渲染未来观测,使智能体能进行反事实推理,剔除违背人类指令的路径。在模拟与真实环境中的大量实验表明,SpatialAnt显著优于现有零样本方法,在R2R-CE上达到66%成功率(SR),在RxR-CE上达50.8% SR。在Hello Robot上的真实部署进一步验证了框架的高效性与有效性,在挑战性真实场景中实现52% SR。
原文摘要 · Abstract (English)
Vision-and-Language Navigation (VLN) has recently benefited from Multimodal Large Language Models (MLLMs), enabling zero-shot navigation. While recent exploration-based zero-shot methods have shown promising results by leveraging global scene priors, they rely on high-quality human-crafted scene reconstructions, which are impractical for real-world robot deployment. When encountering an unseen environment, a robot should build its own priors through pre-exploration. However, these self-built reconstructions are inevitably incomplete and noisy, which severely degrade methods that depend on high-quality scene reconstructions. To address these issues, we propose SpatialAnt, a zero-shot navigation framework designed to bridge the gap between imperfect self-reconstructions and robust execution. SpatialAnt introduces a physical grounding strategy to recover the absolute metric scale for monocular-based reconstructions. Furthermore, rather than treating the noisy self-reconstructed scenes as absolute spatial references, we propose a novel visual anticipation mechanism. This mechanism leverages the noisy point clouds to render future observations, enabling the agent to perform counterfactual reasoning and prune paths that contradict human instructions. Extensive experiments in both simulated and real-world environments demonstrate that SpatialAnt significantly outperforms existing zero-shot methods. We achieve a 66% Success Rate (SR) on R2R-CE and 50.8% SR on RxR-CE benchmarks. Physical deployment on a Hello Robot further confirms the efficiency and efficacy of our framework, achieving a 52% SR in challenging real-world settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。