仅用单目摄像头实现跨机器人通用导航,性能超越现有方法。
Robostral Navigate

- 用单个RGB摄像头输入,直接在图像空间预测目标点,无需深度或多相机。
- 在R2R-CE上达77.4%成功率,比最强单目方法高10.5点,比多传感器系统高5.3点。
- 采用前缀缓存训练和树状注意力掩码,训练时间从月级缩至天级。
大规模部署导航系统需兼顾传感器依赖少、适应多种机器人形态、训练高效。现有最佳系统依赖深度传感器、多相机阵列或预建地图,限制硬件支持并推高成本。本文提出Robostral Navigate,一个80亿参数的视觉语言模型,仅以单目RGB图像流为输入,通过在当前相机视图中指向目标位置来预测航点。其纯图像空间操作方式天然抵抗相机内参与场景尺度变化,使该策略可直接应用于轮式、足式及飞行机器人,无需重新校准。我们生成了240万条轨迹,覆盖35万模拟场景,大幅降低对真实数据的依赖并实现高效扩展。引入前缀缓存训练方案,将整个任务序列打包为单一训练序列,使训练令牌减少22倍,训练时间由数月缩短至数日。树状注意力掩码阻止模型依赖先前真值动作,促进基于视觉的动作预测;结合强化学习进一步提升探索与恢复能力。在Room-to-Room和Room-Across-Room连续环境(R2R-CE与RxR-CE)基准测试中,Robostral Navigate达到新最优表现:在R2R-CE上成功率达77.4%,较最优单目方法提升10.5个百分点,较最强深度或多相机系统高出5.3个百分点;在RxR-CE上达到75.1%成功率,优于所有单目基线。
原文摘要 · Abstract (English)
Deploying navigation systems at scale requires a recipe that minimizes sensor assumptions, generalizes across robot embodiments, and trains efficiently. Yet, today's best systems depend on depth sensors, multi-camera rigs, or pre-built maps, limiting the hardware they support and increasing deployment cost. We introduce Robostral Navigate, an 8B vision-language model built around this scalability objective. The model consumes only a stream of monocular RGB images - the most ubiquitous sensor across robotic platforms and predicts waypoints by pointing to the next target location in the current camera view. Operating purely in image space, rather than robot-specific coordinates, makes the policy naturally robust to changes in camera intrinsics and scene scale, enabling deployment across wheeled, legged, and aerial robots without recalibration. We generate 2.4 million trajectories across 350k simulated scenes to reduce the reliance on real-world data collection and scale easily. We further introduce a prefix-caching training recipe that packs entire episodes into single training sequences, reducing training tokens by 22x and cutting training time from months to days. A tree-based attention mask prevents conditioning on previous ground-truth actions, encouraging visually grounded action prediction, and reinforcement learning is used to further improve exploration and recovery capabilities. On the Room-to-Room and Room-Across-Room in Continuous Environments (R2R-CE and RxR-CE) benchmarks, Robostral Navigate sets a new state of the art. On R2R-CE, it achieves a 77.4% success rate, surpassing the best monocular method by 10.5 points and the strongest depth- or multi-camera system by 5.3 points despite using only a single RGB camera. On RxR-CE, it reaches 75.1% success rate, outperforming all monocular baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。