测试视觉语言模型走复杂路径的能力，发现交叉点是主要难点。
TraversalBench: Challenging Paths to Follow for Vision Language Models
- 设计可控的连续折线路径任务，要求模型按顺序识别顶点。
- 自相交是主要难点，首次交叉后性能骤降，暴露模型决策瓶颈。
- 揭示模型存在从左到右的阅读偏好，适合研究视觉推理与导航的学者。
视觉语言模型（VLMs）在多模态基准上表现优异，但其对复杂视觉路径的跟随能力仍缺乏系统评估。本文提出TraversalBench，一个用于精确视觉路径追踪的受控基准。每个样本包含一条带有唯一起点标记和标注顶点的连续折线；模型需恢复从起点到终点的有序顶点序列。该基准平衡了自相交数量、路径弯曲度、顶点数及邻近干扰线，同时限制对OCR、常识或开放式规划的依赖。实验发现，自相交是主要困难来源。首次交叉分析表明，模型在首次交叉前表现稳定，交叉后性能急剧下降，失败集中在交叉点判断。邻近干扰物有较弱但累积的影响，辅助阅读顺序任务揭示出一致的从左到右偏倚。这些结果刻画了VLMs对视觉路径的感知与失效模式。最后，我们定位TraversalBench为持续且精确视觉定位基准的新贡献。代码、数据与可视化示例已公开于https://github.com/clarapetrova/traversalbench。
原文摘要 · Abstract (English)
Vision-language models (VLMs) perform strongly on multimodal benchmarks, but their ability to follow complex visual paths remains under-tested. We introduce TraversalBench, a controlled benchmark for exact visual path traversal. Each instance contains a continuous polyline with a unique start marker and labeled vertices; models must recover the ordered sequence encountered from start to finish. The benchmark balances self-intersection count, tortuosity, vertex count, and nearby confounding lines while limiting reliance on OCR, world knowledge, or open-ended planning. We find that self-intersections are the dominant source of difficulty. A first-crossing analysis localizes failures to crossing points: performance is stable before the first crossing, then drops sharply when the model must resolve the correct continuation. Nearby confounders have weaker but compounding effects, and an auxiliary reading-order benchmark reveals a consistent left-to-right bias. Together, these results characterize how VLMs perceive and fail on visual paths. Finally, we position TraversalBench as a new contribution to the growing line of sustained and precise visual grounding benchmarks for VLMs. Code, benchmark data, and rendered examples are available at https://github.com/clarapetrova/traversalbench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。