arXiv:2508.09444cs.ROcs.CV2025-08被引 6

用扩散模型统一导航规划与路径点生成,提升复杂环境下的指令跟随能力。

DAgger Diffusion Navigation: DAgger Boosted Diffusion Policy for Vision-Language Navigation

  • 将路径点生成与导航规划合并为一个扩散策略,端到端优化。
  • 在Matterport3D上成功率提升至72.1%,超越以往两阶段方法。
  • 结合DAgger在线纠错,增强长程任务中的容错与空间推理能力。

视觉语言导航在连续环境(VLN-CE)中要求智能体根据自然语言指令在自由形态的3D空间中移动。现有方法通常采用两阶段方式:先由高层路径点预测器生成可导航路径点,再由导航规划器在高层动作空间中提出中间目标。然而该框架存在两大问题:(1) 各阶段使用代理目标导致全局次优;(2) 性能严重依赖第一阶段路径点质量。为此,本文提出DAgger Diffusion Navigation(DifNav),一种端到端优化的VLN-CE策略,将传统两阶段统一为单一扩散策略。DifNav采用条件扩散模型直接建模未来动作在连续导航空间中的多模态分布,无需路径点预测器,同时支持捕捉多种符合指令的行为。为缓解模仿学习中的误差累积并增强长程任务的空间推理能力,引入DAgger进行在线策略训练与专家轨迹增强,并利用聚合数据进一步微调策略。实验表明,在不使用路径点预测器的前提下,该方法在基准数据集上显著优于以往最先进两阶段模型。代码已开源:https://github.com/Tokishx/DifNav。

原文摘要 · Abstract (English)

Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to follow natural language instructions through free-form 3D spaces. Existing VLN-CE approaches typically use a two-stage waypoint planning framework, where a high-level waypoint predictor generates the navigable waypoints, and then a navigation planner suggests the intermediate goals in the high-level action space. However, this two-stage decomposition framework suffers from: (1) global sub-optimization due to the proxy objective in each stage, and (2) a performance bottleneck caused by the strong reliance on the quality of the first-stage predicted waypoints. To address these limitations, we propose DAgger Diffusion Navigation (DifNav), an end-to-end optimized VLN-CE policy that unifies the traditional two stages, i.e. waypoint generation and planning, into a single diffusion policy. Notably, DifNav employs a conditional diffusion policy to directly model multi-modal action distributions over future actions in continuous navigation space, eliminating the need for a waypoint predictor while enabling the agent to capture multiple possible instruction-following behaviors. To address the issues of compounding error in imitation learning and enhance spatial reasoning in long-horizon navigation tasks, we employ DAgger for online policy training and expert trajectory augmentation, and use the aggregated data to further fine-tune the policy. This approach significantly improves the policy's robustness and its ability to recover from error states. Extensive experiments on benchmark datasets demonstrate that, even without a waypoint predictor, the proposed method substantially outperforms previous state-of-the-art two-stage waypoint-based models in terms of navigation performance. Our code is available at: https://github.com/Tokishx/DifNav.

视觉语言导航扩散模型端到端DAgger

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。