构建高保真3D导航数据与仿真平台,实现多任务统一建模。
GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation

- 基于3DGS构建可交互高保真仿真环境,支持碰撞感知导航。
- 提出GN-BAE模型,在GN-Bench上超越现有方法,提升长程导航能力。
- 适合研究具身智能、人机交互及机器人路径规划的开发者使用。
具身导航将智能体与物理世界连接,是通用机器人智能的基础。受限于导航数据的数量和质量,视觉语言导航(VLN)系统在泛化能力和长程任务上表现不足。为此,我们构建了多样化的3D场景,并开发了大规模导航数据的自动化生成流程,形成GN-Matrix数据集。基于3D高斯溅射(3DGS)引擎,我们设计了一个支持交互式漫游和碰撞感知导航的高保真仿真平台。进一步提出首个基于鸟瞰图(BEV)的基准评测框架GN-Bench,引入动态3DGS虚拟人用于人机交互评估。为充分利用仿真环境,我们开发了基于强化学习的导航基础模型——破与立(BAE)。在监督学习后,通过DAgger算法使模型接触回放生成的状态,打破专家分布局限,支持下游强化学习探索。该统一范式整合了基于地图与无地图任务,涵盖指令遵循、人类跟随与目标导航。GN-BAE将高保真3DGS渲染的鸟瞰图表示作为紧凑记忆,激发视觉语言模型的潜在空间推理能力。在GN-Bench和VLN-CE上的大量实验表明,GN0显著优于现有先进方法。总体而言,GN-Matrix提供了一个涵盖数据、仿真与学习的统一框架,推动了具身导航在科研与工业应用中的发展。
原文摘要 · Abstract (English)
Embodied navigation connects intelligent agents with the physical world and is fundamental for general robotic intelligence. Limited availability and quality of navigation data have constrained Vision-and-Language Navigation (VLN) systems' generalization and long-horizon capabilities. To address this, we curate diverse 3D scenes and develop an automated pipeline for large-scale navigation data, resulting in the GN-Matrix dataset. Building on a 3D Gaussian Splatting (3DGS) engine, we introduce a high-fidelity simulation platform supporting interactive roaming and collision-aware navigation. We further propose GN-Bench, the first BEV-based benchmark incorporating dynamic 3DGS avatars for human-robot interaction evaluation. To leverage the simulator, we develop an RL-driven navigation foundation model, Break and Establish (BAE). After supervised learning, DAgger exposes the model to rollout-induced states, breaking narrow expert-centric distributions and enabling downstream RL exploration. This unified VLN paradigm integrates map-based and map-free tasks, including instruction following, human following, and goal navigation. GN-BAE formalizes high-fidelity 3DGS-rendered Bird's Eye View representations as compact memory, unlocking latent spatial reasoning in VLMs. Extensive evaluations on GN-Bench and VLN-CE show that GN0 outperforms state-of-the-art VLN methods. Overall, GN-Matrix offers a unified framework spanning data, simulation, and learning, advancing embodied navigation in research and industrial applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。