评测视觉语言模型在地图上做可验证的出行决策能力。
MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps ?
- 构建基于开放街图的可验证地图决策基准，支持结构化输出与图谱校验。
- 7个VLM模型在设施定位任务中表现接近随机，且普遍存在缩放不一致问题。
- 适合研究地图理解、导航系统与多模态推理的学者使用。
视觉语言模型（VLMs）被广泛用于物流、配送和无障碍导航中的地图读取，其输出需符合道路网络约束。然而现有地图评测多依赖自由文本或选择题，无法验证结果是否符合底层路网。本文提出MapReason-OSM，一个针对自渲染OpenStreetMap面板的可图谱验证的出行决策基准与评估工具。我们在10个美国市中心以两个对齐缩放级别生成固定风格的地图，采用统一标记语法，并为每张图配以隐藏的道路图谱与精确答案，共生成6,000个实例（跨两个缩放级别共12,000张图），涵盖12项任务，包括路径规划、设施选址与视觉歧义消解。模型输出结构化决策，我们将其投影回图谱，评估有效性、合法性、最优性及约束满足度，并检测跨缩放一致性。在7个VLM中，模型虽能简单读图与规划路径，但在图谱成本推理上表现不佳（单设施定位准确率接近随机），且频繁出现缩放不一致。我们开源了基准、评估工具与确定性生成器。代码与数据：https://github.com/Vi-Sri/mapreason-osm
原文摘要 · Abstract (English)
Vision-language models (VLMs) are increasingly used to read maps for logistics, delivery, and accessible navigation, where the output is an actionable decision (a route, a pin, a parking choice) that must respect the road network. Yet most map benchmarks grade free text or multiple-choice answers that cannot be verified against the underlying graph. We present MapReason-OSM, a benchmark and evaluation harness for graph-verifiable mobility decisions on self-rendered OpenStreetMap panels. We render fixed-style maps for ten U.S. downtowns at two aligned zoom scales, overlay a consistent marker grammar, and pair each panel with a hidden street graph and exact oracles, yielding 6,000 instances (12,000 panels across the two zooms) over 12 routing, facility-location, and visual disambiguation tasks. Models return structured decisions that we snap back to the graph and score for validity, legality, optimality, and constraint satisfaction, plus cross-zoom consistency. Across seven VLMs, models read maps and route simply but fail at graph cost reasoning (single-facility pin placement is near chance even for frontier reasoning models), and are frequently scale-inconsistent. We release the benchmark, harness, and deterministic generator. Code and data: https://github.com/Vi-Sri/mapreason-osm
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。