arXiv:2606.03682cs.RO2026-06被引 5

构建高保真3D导航数据与仿真平台,实现多任务统一建模。

GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation

论文配图:GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation
图 1 · 摘自论文原文
  • 基于3DGS构建可交互高保真仿真环境,支持碰撞感知导航。
  • 提出GN-BAE模型,在GN-Bench上超越现有方法,提升长程导航能力。
  • 适合研究具身智能、人机交互及机器人路径规划的开发者使用。

具身导航将智能体与物理世界连接,是通用机器人智能的基础。受限于导航数据的数量和质量,视觉语言导航(VLN)系统在泛化能力和长程任务上表现不足。为此,我们构建了多样化的3D场景,并开发了大规模导航数据的自动化生成流程,形成GN-Matrix数据集。基于3D高斯溅射(3DGS)引擎,我们设计了一个支持交互式漫游和碰撞感知导航的高保真仿真平台。进一步提出首个基于鸟瞰图(BEV)的基准评测框架GN-Bench,引入动态3DGS虚拟人用于人机交互评估。为充分利用仿真环境,我们开发了基于强化学习的导航基础模型——破与立(BAE)。在监督学习后,通过DAgger算法使模型接触回放生成的状态,打破专家分布局限,支持下游强化学习探索。该统一范式整合了基于地图与无地图任务,涵盖指令遵循、人类跟随与目标导航。GN-BAE将高保真3DGS渲染的鸟瞰图表示作为紧凑记忆,激发视觉语言模型的潜在空间推理能力。在GN-Bench和VLN-CE上的大量实验表明,GN0显著优于现有先进方法。总体而言,GN-Matrix提供了一个涵盖数据、仿真与学习的统一框架,推动了具身导航在科研与工业应用中的发展。

原文摘要 · Abstract (English)

Embodied navigation connects intelligent agents with the physical world and is fundamental for general robotic intelligence. Limited availability and quality of navigation data have constrained Vision-and-Language Navigation (VLN) systems' generalization and long-horizon capabilities. To address this, we curate diverse 3D scenes and develop an automated pipeline for large-scale navigation data, resulting in the GN-Matrix dataset. Building on a 3D Gaussian Splatting (3DGS) engine, we introduce a high-fidelity simulation platform supporting interactive roaming and collision-aware navigation. We further propose GN-Bench, the first BEV-based benchmark incorporating dynamic 3DGS avatars for human-robot interaction evaluation. To leverage the simulator, we develop an RL-driven navigation foundation model, Break and Establish (BAE). After supervised learning, DAgger exposes the model to rollout-induced states, breaking narrow expert-centric distributions and enabling downstream RL exploration. This unified VLN paradigm integrates map-based and map-free tasks, including instruction following, human following, and goal navigation. GN-BAE formalizes high-fidelity 3DGS-rendered Bird's Eye View representations as compact memory, unlocking latent spatial reasoning in VLMs. Extensive evaluations on GN-Bench and VLN-CE show that GN0 outperforms state-of-the-art VLN methods. Overall, GN-Matrix offers a unified framework spanning data, simulation, and learning, advancing embodied navigation in research and industrial applications.

具身智能3DGS导航强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。