用自然语言描述自动驾驶数据集图像差异，提升可解释性。
Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language
- 基于目标检测提取图像区域，分析两组图像间的语义差异。
- 在真实场景中验证方法，低频差异下仍能准确识别变化特征。
- 开源基准和代码，适合研究数据分布与模型鲁棒性的团队使用。
理解大规模自动驾驶数据集的组成对于保障系统安全、鲁棒性和跨域可靠性至关重要。例如，不同地理位置间的数据分布偏移可能导致运行环境与训练数据不匹配，引发性能下降甚至危险。现有数据分析方法多依赖元数据、预定义标签或人工检查，缺乏语义深度且难以扩展。本文研究集合差异描述任务：给定两组图像子集，生成自然语言描述其差异。基于两阶段框架，我们通过目标检测提取的物体中心区域简化聚合，并实现差异归因至具体物体实例或类别。为评估该设定的域内表现，我们引入新基准 AD-Diff Bench。低浓度实验检验了方法在稀疏真实差异下的适用性。所有实验仅使用开源权重模型以确保可复现性和部署便捷性。所提基准与分析为自动驾驶数据集的实用化、可解释性审视提供了新路径。实现与数据集已公开于 https://github.com/KIT-MRT/AD-Diff。
原文摘要 · Abstract (English)
Understanding the composition of large-scale autonomous driving datasets is essential for safety, robustness, and reliable operation across domains. For example, domain shift between locations could lead to the operating environment being misaligned with the training data, resulting in potentially dangerous performance degradation. Yet, existing data analysis pipelines largely rely on metadata, predefined labels, or manual inspection, which provide limited semantic insight or do not scale. This paper studies set difference captioning: given two subsets of images, the goal is to produce a natural-language hypothesis describing differences between the target and reference set. Building on a two-stage formulation, we adapt the method to autonomous driving by focusing on object-centric patches derived from object detection, which simplifies aggregation and enables attribution of differences to specific object instances or categories. To evaluate this setting in-domain, we introduce a new benchmark, AD-Diff Bench. Low-concentration experiments assess the suitability of set-difference-captioning approaches to sparse, real-world differences. We restrict our experiments to open-weight models to support reproducibility and ease of deployment. The proposed benchmark and analysis provide a step towards practical, human-interpretable dataset introspection for autonomous driving datasets. Our implementation and benchmark dataset are available at https://github.com/KIT-MRT/AD-Diff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。