用真实视频训练能避障的导航模型，无需人工标注轨迹。
VEGA: Learning Navigation VLAs from In-the-Wild Egocentric Video with Geometric Trajectory Supervision
- 从单目视频重建场景几何，生成带障碍物感知的导航路径。
- 在基准上减少33%碰撞，避障成功率提升150%以上。
- 适合做视觉导航的开发者，尤其关注真实场景泛化能力。
我们提出 VEGA，一种从无标注第一人称导航视频中训练视觉语言动作（VLA）模型的方法。互联网规模的第一人称视频提供了大量包含复杂场景、近距离障碍物和自然人类运动的真实空间数据，但这些视频无法直接用于策略学习，因其未提供以机器人坐标系为参考的、面向明确导航目标的障碍物感知轨迹。VEGA 通过从单目视频重建局部场景几何，采样导航目标（文本、图像或空间路点），并利用构建的几何信息生成障碍物感知轨迹，从而训练流匹配型 VLA 导航策略。该方法仅在训练阶段使用几何信息，将障碍物感知规划直接融入视觉策略中。此外，我们构建了 VEGA-Bench 基准，包含 250 万场景与约 500 万导航目标及对应场景几何，用于评估目标达成率、碰撞规避与障碍物清除能力。实验表明，VEGA 在 VEGA-Bench 上实现与最强基线相当的目标达成率，碰撞减少 33.0%，障碍物清除提升 17.9%；在真实世界测试中，成功率达基线至少 150.0%，碰撞减少至少 66.7%，障碍物清除提升至少 60.0%。结果证明，视频衍生的几何监督是训练障碍物感知导航 VLA 的可扩展有效信号。代码与基准将在发表时公开。
原文摘要 · Abstract (English)
We introduce VEGA, an approach for training navigation VisionLanguage-Action (VLA) models from unlabeled egocentric navigation videos. Internet-scale egocentric videos provide a scalable source of navigation-relevant visual observations, capturing cluttered scenes, close-range obstacles, and natural human motion through real-world spaces. However, these videos are not directly usable for policy learning because they do not provide obstacle-aware trajectories conditioned on explicit navigation goals in the robot's coordinate frame. VEGA addresses this gap by reconstructing local scene geometry from monocular video, sampling navigation goals (represented as text, image, or spatial waypoints) and generating obstacle-aware trajectories using the constructed geometry. The resulting trajectory distribution is then used to train a flow-matching VLA navigation policy. By using geometry exclusively during training, VEGA distills obstacle-aware planning directly into a vision-based policy. Furthermore, we introduce VEGA-Bench, a benchmark containing 250k scenes and approximately 5 million navigation goals paired with scene geometry, designed to evaluate goal progress, collision avoidance, and obstacle clearance of VLAs. Our evaluation shows that VEGA achieves competitive goal progress while reducing collisions by 33.0% and improving obstacle clearance by 17.9% over the strongest baseline on VEGABench, while improving success by at least 150.0%, reducing collisions by at least 66.7%, and improving obstacle clearance by at least 60.0% in real-world trials. Ultimately, we demonstrate that video-derived geometric supervision provides a scalable and effective signal for training obstacle-aware navigation VLAs. The code and benchmark will be released at the time of publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。