用轻量拓扑图+视觉停顿机制,让导航更抗环境变化。
HaltNav: Reactive Visual Halting over Lightweight Topological Priors for Robust Vision-Language Navigation
- 基于文本的轻量拓扑图osmAG实现全局规划
- 引入视觉停顿检测局部障碍并自动重规划
- 适合长期导航任务中应对真实环境动态变化
视觉-语言导航(VLN)正从固定步骤指令执行转向开放词汇、目标导向的自主导航。为实现这一转变而无需大量路径提示,需依赖结构先验。现有方法多依赖计算复杂的2D/3D度量地图,我们则采用一种轻量级、基于文本的osmAG(OpenStreetMap Area Graph),即楼层级拓扑表示,易于获取与维护。然而仅靠全局地图规划在真实部署中仍脆弱,因局部连通性可能变化(如门关闭或通道拥堵),导致执行失败。为此,我们提出分层导航框架HaltNav,将osmAG的鲁棒全局规划与VLN的局部探索及指令对齐能力结合。其核心为基于MLLM的脑模块,能进行高层任务对齐与障碍感知;基于osmAG生成局部执行片段,并提供前导接地、目标聚焦的子指令。同时,通过称为“反应式视觉停顿”(RVH)的机制检测局部异常,中断控制循环,通过失效对应拓扑更新osmAG,并触发重规划以制定可行绕行路径。为高效训练该停顿能力,我们设计数据合成流水线,利用生成模型向本可通行场景注入真实障碍,大幅丰富困难负样本。大量实验表明,该分层框架在无繁琐语言指令下优于多个基线方法,显著提升长时程视觉-语言导航在环境变化下的鲁棒性。
原文摘要 · Abstract (English)
Vision-and-Language Navigation (VLN) is shifting from rigid, step-by-step instruction following toward open-vocabulary, goal-oriented autonomy. Achieving this transition without exhaustive routing prompts requires agents to leverage structural priors. While prior work often assumes computationally heavy 2D/3D metric maps, we instead exploit a lightweight, text-based osmAG (OpenStreetMap Area Graph), a floorplan-level topological representation that is easy to obtain and maintain. However, global planning over a prior map alone is brittle in real-world deployments, where local connectivity can change (e.g., closed doors or crowded passages), leading to execution-time failures. To address this gap, we propose a hierarchical navigation framework HaltNav that couples the robust global planning of osmAG with the local exploration and instruction-grounding capability of VLN. Our approach features an MLLM-based brain module, which is capable of high-level task grounding and obstruction awareness. Conditioned on osmAG, the brain converts the global route into a sequence of localized execution snippets, providing the VLN executor with prior-grounded, goal-centric sub-instructions. Meanwhile, it detects local anomalies via a mechanism we term Reactive Visual Halting (RVH), which interrupts the local control loop, updates osmAG by invalidating the corresponding topology, and triggers replanning to orchestrate a viable detour. To train this halting capability efficiently, we introduce a data synthesis pipeline that leverages generative models to inject realistic obstacles into otherwise navigable scenes, substantially enriching hard negative samples. Extensive experiments demonstrate that our hierarchical framework outperforms several baseline methods without tedious language instructions, and significantly improves robustness for long-horizon vision-language navigation under environmental changes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。