提出新基准,测试视觉语言导航中条件分支的决策能力。
If, Then, Otherwise: Diagnosing Conditional Branching in Vision-Language Navigation

- 构建基于场景图的条件指令生成框架,支持精确控制分支逻辑。
- 实测四款先进模型在条件分支上失败率超40%,暴露隐藏缺陷。
- 适合研究具身智能、视觉推理与导航融合的学者使用。
视觉语言导航代理通常在固定目标路径指令下评估,但真实导航常依赖环境状态:若条件成立则走某路径,否则换另一条。这要求代理能评估场景证据、选择正确逻辑分支并执行对应行为。现有评估难以区分失败是源于感知、语义对齐、导航还是逻辑判断。本文提出 CondVLN,一个基于场景图的条件分支诊断基准。该基准可程序化生成以可验证3D场景图谓词为依据的条件指令,可控调节分支深度、依赖链长度、空间组合、证据可观测性及指令时序跨度。涵盖 AI2-THOR、Matterport3D、Gibson 与 ReplicaCAD 共超过11,500条生成指令,采用标准 VLN 指标与分支特异性诊断(分支选择准确率、条件成功率)评估。测试四种前沿 VLN 代理(VLN-Zero、NaVid、NaVILA、Open-Nav)发现,条件分支暴露了标准成功率与路径长度无法捕捉的失败:代理可能路径合理却选错分支。我们还提出轻量级神经符号分支选择模型,将条件判断与导航执行分离,性能提升2倍。CondVLN 提供可复用的测试平台,用于检验具身代理是否能在正确条件下执行正确的指令。
原文摘要 · Abstract (English)
Vision-language navigation agents are often evaluated on their ability to follow route-like instructions toward a fixed goal. Yet, real navigation instructions often depend on observed states of the environment: if a condition holds, then follow one path, otherwise take another. Such instructions require an agent to evaluate scene evidence, select the correct logical branch, and execute the corresponding navigation behavior. Existing evaluations provide limited control over conditional branch execution, making it difficult to determine whether agents fail because of perception, grounding, navigation, or logical decision-making. We introduce CondVLN, a scene-graph-grounded benchmark for diagnosing conditional branching in vision-language navigation. CondVLN programmatically generates instructions whose branch conditions are grounded in verifiable 3D scene-graph predicates, with controlled variation in branch depth, dependency chain length, spatial composition, evidence observability, and instruction horizon. CondVLN contains over 11,500 generated conditional instructions across AI2-THOR, Matterport3D, Gibson, and ReplicaCAD, and evaluates agents using standard VLN metrics and branch-specific diagnostics: Branch Selection Accuracy and Conditional Success Rate. Evaluating four state-of-the-art VLN agents (VLN-Zero, NaVid, NaVILA, and Open-Nav) shows that conditional branching exposes failures that are not captured by standard success rate or path length alone: agents can navigate plausibly while committing to a branch inconsistent with the observed scene condition. We also present a lightweight neurosymbolic branch-selection model that separates condition grounding from navigation execution, improving performance by 2x. CondVLN provides a reusable testbed for measuring whether embodied agents can not only follow instructions, but follow the right instruction under the right condition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。