arXiv:2507.08831cs.CVcs.LG2025-07中稿 · RA-L 2026被引 7

让AI导航模型无视视角变化,提升在真实环境中的稳定性。

View Invariant Learning for Vision-Language Navigation in Continuous Environments

  • 用对比学习提取稀疏且视角不变的特征,增强导航鲁棒性。
  • 在R2R-CE和RxR-CE上成功率提升8%-15%,RxR-CE达顶尖水平。
  • 可作为即插即用模块,适用于仿真与真实机器人部署。

视觉语言导航在连续环境(VLNCE)中,要求智能体根据指令自由移动至目标位置,是具身AI的关键问题。然而现有方法对视角变化敏感,如摄像头高度与视角差异。本文提出更通用的场景V²-VLNCE(带多种视角的VLNCE),并设计视图不变后训练框架VIL(View Invariant Learning),使已有导航策略更抗视角变化。VIL采用对比学习机制学习稀疏、视角不变特征,并引入教师-学生框架优化路径预测模块,由依赖视角的教师模型向视角无关的学生模型知识迁移。通过端到端联合优化,实验表明该方法在两个标准基准数据集R2R-CE和RxR-CE上成功率提升8%-15%;在更具挑战性的RxR-CE上达到当前最优表现。同时,在标准VLNCE设置下性能未下降,说明VIL可作为即插即用的后处理模块。进一步在模拟真实机器人配置(如Stretch RE-1、LoCoBot)及配备全景相机+激光雷达的真实环境进行验证,均实现一致性能提升。代码已开源。

原文摘要 · Abstract (English)

Vision-Language Navigation in Continuous Environments (VLNCE), where an agent follows instructions and moves freely to reach a destination, is a key research problem in embodied AI. However, most existing approaches are sensitive to viewpoint changes, i.e. variations in camera height and viewing angle. Here we introduce a more general scenario, V$^2$-VLNCE (VLNCE with Varied Viewpoints) and propose a view-invariant post-training framework, called VIL (View Invariant Learning), that makes existing navigation policies more robust to changes in camera viewpoint. VIL employs a contrastive learning framework to learn sparse and view-invariant features. We also introduce a teacher-student framework for the Waypoint Predictor Module, a standard part of VLNCE baselines, where a view-dependent teacher model distills knowledge into a view-invariant student model. We employ an end-to-end training paradigm to jointly optimize these components. Empirical results show that our method outperforms state-of-the-art approaches on V$^2$-VLNCE by 8-15\% measured on Success Rate for two standard benchmark datasets R2R-CE and RxR-CE. Evaluation of VIL in standard VLNCE settings shows that despite being trained for varied viewpoints, VIL often still improves performance. On the harder RxR-CE dataset, our method also achieved state-of-the-art performance across all metrics. This suggests that adding VIL does not diminish the standard viewpoint performance and can serve as a plug-and-play post-training method. We further evaluate VIL for simulated camera placements derived from real robot configurations (e.g. Stretch RE-1, LoCoBot), showing consistent improvements of performance. Finally, we present a proof-of-concept real-robot evaluation in two physical environments using a panoramic RGB sensor combined with LiDAR. The code is available at https://github.com/realjoshqsun/V2-VLNCE.

视觉导航视角不变具身AI机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。