用双重验证提升大模型导航的可靠性,减少错误累积。
DV-VLN: Dual Verification for Reliable LLM-Based Vision-and-Language Navigation
- 先生成思维链,再通过真假和掩码实体双通道验证候选动作。
- 在R2R、RxR、REVERIE上均超越单次决策基线,性能领先语言类模型。
- 适合关注可解释性与鲁棒性的视觉语言导航研究者。
视觉-语言导航(VLN)要求具身智能体根据自然语言指令在复杂3D环境中导航。近期大语言模型(LLMs)的发展使语言驱动导航具备更强可解释性。然而,多数基于LLM的智能体仍依赖单次动作决策,需从噪声化、多视角的文本观测中选择单一选项。由于局部不匹配和中间推理不完善,此类决策易偏离正确路径,导致误差累积,在未见环境中可靠性下降。本文提出DV-VLN框架,采用生成-验证范式:首先对开源LLaMA-2主干进行参数高效域内适配,生成结构化导航思维链;随后通过两个互补通道验证候选动作:真假验证(TFV)与掩码实体验证(MEV)。通过聚合多样本验证成功次数,生成可解释评分以重排序动作。在R2R、RxR(英文子集)和REVERIE上的实验表明,DV-VLN持续优于直接预测与仅采样基线,在语言类模型中表现具有竞争力,并展现出与部分跨模态系统相当的前景。代码已公开于https://github.com/PlumJun/DV-VLN。
原文摘要 · Abstract (English)
Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in a complex 3D environment according to natural language instructions. Recent progress in large language models (LLMs) has enabled language-driven navigation with improved interpretability. However, most LLM-based agents still rely on single-shot action decisions, where the model must choose one option from noisy, textualized multi-perspective observations. Due to local mismatches and imperfect intermediate reasoning, such decisions can easily deviate from the correct path, leading to error accumulation and reduced reliability in unseen environments. In this paper, we propose DV-VLN, a new VLN framework that follows a generate-then-verify paradigm. DV-VLN first performs parameter-efficient in-domain adaptation of an open-source LLaMA-2 backbone to produce a structured navigational chain-of-thought, and then verifies candidate actions with two complementary channels: True-False Verification (TFV) and Masked-Entity Verification (MEV). DV-VLN selects actions by aggregating verification successes across multiple samples, yielding interpretable scores for reranking. Experiments on R2R, RxR (English subset), and REVERIE show that DV-VLN consistently improves over direct prediction and sampling-only baselines, achieving competitive performance among language-only VLN agents and promising results compared with several cross-modal systems.Code is available at https://github.com/PlumJun/DV-VLN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。