构建细粒度交通标志数据集，揭示视觉语言模型的识别短板。
Mapillary Vistas Validation for Fine-Grained Traffic Signs: A Benchmark Revealing Vision-Language Model Limitations
- 将复合交通标志拆分为细粒度语义类别，提供像素级实例掩码。
- DINOv2 在交通标志识别与常见物体上均优于所有视觉语言模型。
- 适合自动驾驶感知系统研究者，推动高精度语义理解发展。
获取高质量的细粒度交通标志标注对自动驾驶中准确安全决策至关重要。现有广泛使用的数据集如Mapillary通常仅提供粗粒度标签，无法区分停驶、限速等语义关键类型。为此，我们基于Mapillary数据集构建了新的交通标志验证集——Mapillary Vistas Validation for Traffic Signs（MVV），将复合交通标志分解为细粒度、语义明确的类别。该数据集包含像素级实例掩码，由专家人工标注以确保标签可靠性。我们在此数据集上对多种前沿视觉语言模型（VLMs）与自监督的DINOv2模型进行基准测试，结果表明DINOv2在交通标志识别及车辆、行人等高频类别上均持续优于所有VLM基线。分析揭示当前VLM在细粒度视觉理解方面存在显著局限，并确立DINOv2作为自动驾驶场景下密集语义匹配的强基准。该数据集与评估框架为更可靠、可解释、可扩展的感知系统铺平道路。代码与数据见：https://github.com/nec-labs-ma/relabeling
原文摘要 · Abstract (English)
Obtaining high-quality fine-grained annotations for traffic signs is critical for accurate and safe decision-making in autonomous driving. Widely used datasets, such as Mapillary, often provide only coarse-grained labels - without distinguishing semantically important types such as stop signs or speed limit signs. To this end, we present a new validation set for traffic signs derived from the Mapillary dataset called Mapillary Vistas Validation for Traffic Signs (MVV), where we decompose composite traffic signs into granular, semantically meaningful categories. The dataset includes pixel-level instance masks and has been manually annotated by expert annotators to ensure label fidelity. Further, we benchmark several state-of-the-art VLMs against the self-supervised DINOv2 model on this dataset and show that DINOv2 consistently outperforms all VLM baselines-not only on traffic sign recognition, but also on heavily represented categories like vehicles and humans. Our analysis reveals significant limitations in current vision-language models for fine-grained visual understanding and establishes DINOv2 as a strong baseline for dense semantic matching in autonomous driving scenarios. This dataset and evaluation framework pave the way for more reliable, interpretable, and scalable perception systems. Code and data are available at: https://github.com/nec-labs-ma/relabeling
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。