arXiv:2606.20458cs.RO2026-06

用视觉语言模型提升机器人导航选路能力,解决复杂场景下选错路径的问题。

Slow Brain, Fast Planner: Latency-Resilient VLM-Augmented Urban Navigation

论文配图:Slow Brain, Fast Planner: Latency-Resilient VLM-Augmented Urban Navigation
图 1 · 摘自论文原文
  • 用VLM从规划器候选轨迹中选出最优索引,再融合进原始输出
  • 在真实场景中使轨迹误差降低30%,复杂路口成功率显著提升
  • 无需训练,能容忍长达5秒延迟,适合实际部署的移动机器人

基于学习的路边导航规划器可实时生成多种候选轨迹,但其评分函数在复杂场景中常无法选出最优路径,导致机器人驶上草坪、朝向行人或走错方向,即使更优路径存在于同一集合中。我们称此为轨迹评分差距:在真实路边导航中,锚点规划器首选与最佳候选之间的差距显著,可能源于规划器对场景理解能力有限。不同于用端到端视觉-语言-动作模型替代规划器,本文提出一种VLM-规划器接口,利用VLM从规划器的候选集中选择最优轨迹索引,并将其与规划器初始输出融合。然而,VLM每次查询需1–3秒,无法直接用于5–20Hz控制循环。为此,我们设计了一种免训练、抗延迟的轨迹级融合层,通过几何相似性加权衰减,将滞后的VLM选择转化为实时评分。在约2,000个挑战性真实场景(如路口、行人相遇)中,VLM选择相比规划器最佳选择实现30%的ADE降低;在模拟中,得分融合在高达5秒延迟下仍保持超过80%的成功率。我们在移动机器人上验证了整套系统,在校园复杂人行道上实现了多网络延迟下的稳定导航。

原文摘要 · Abstract (English)

Learning-based planners for sidewalk navigation can generate diverse candidate trajectories in real time, yet their scoring functions often fail to select the best trajectory in challenging situations, outputting trajectories that make the mobile robot drive onto grass, toward pedestrians, or in the wrong direction, even when better candidates exist in the same set. We call this the trajectory scoring gap: in real-world sidewalk navigation, the gap between an anchor-based planner's top choice and the best possible candidate is substantial, likely due to limited high-level scene understanding capability of the planner. Rather than replacing the planner with an end-to-end Vision-Language-Action model, we propose a VLM-Planner interface that uses a VLM to select a candidate index from the planner's proposal set and then fuse it with the planner's initial output. However, VLMs take 1--3s per query and so cannot directly drive a 5--20Hz control loop. We contribute a training-free, latency-resilient trajectory-level fusion layer that turns a stale VLM selection into real-time planner scoring via geometric similarity with exponential decay. On $\sim$2,000 challenging real-world scenarios (e.g., junctions, pedestrian encounters), VLM selection achieves 30% ADE reduction versus the planner's best selection, while the planner remains competitive in routine situations. In simulation, Score Fusion maintains >80% success rate with delays up to 5s. We demonstrate the full system on a mobile robot navigating challenging campus sidewalks with varied network latency.

导航规划视觉语言模型延迟容忍机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。