提出新评估指标,更好区分在线地图生成模型优劣
Beyond Chamfer Distance: Granular Order-aware Evaluation Metric For Online Mapping

- 引入SOSPA度量点序关系,精细评估几何形状
- 提出PLD软匹配机制,同时衡量检测与精度
- 在nuScenes上发现检测能力是当前主要瓶颈
在线地图估计是自动驾驶系统的关键,减少对高成本高清地图的依赖。现有方法将地图元素表示为点序列构成折线和多边形,评估主要基于阈值化切比雪夫距离(CD)的平均精度(mAP)。该框架对点序不敏感,几何质量评估粒度不足,难以区分性能差异。本文从两方面改进:针对单个实例,提出顺序最优子模式分配(SOSPA),一种考虑点序的度量,可精细评估几何形状且满足所有度量公理;针对多实例评估,提出折线定位与检测(PLD),一种软度量,联合捕捉检测质量和几何精度,取代mAP的硬阈值。在nuScenes数据集上的实验表明,PLD能有效排序SOTA方法(MapTRv2、StreamMapNet、MapTracker),并提供分解误差分析,揭示检测能力是当前方法的主要瓶颈,这一趋势mAP无法捕捉。评估代码将公开。
原文摘要 · Abstract (English)
Online map estimation is a crucial component of autonomous driving systems that reduces the reliance on costly high-definition maps. State-of-the-art (SOTA) methods commonly predict map elements as ordered sequences of points that form polylines and polygons. The evaluation of these methods relies predominantly on mean average precision (mAP) based on thresholded Chamfer distance (CD). This framework lacks sensitivity to point ordering and provides limited granularity in assessing geometric quality, making it difficult to distinguish which methods truly excel over others. In this work, we address these limitations on two fronts. For the single-instance similarity measure, we introduce sequence optimal sub-pattern assignment (SOSPA), an order-aware metric that enables fine-grained evaluation of individual geometries while satisfying all metric axioms. For the multi-instance evaluation framework, we propose polyline localisation and detection (PLD), a soft metric that jointly captures detection quality and geometric accuracy, replacing the hard thresholding of mAP with a principled soft assignment. Through evaluations on nuScenes, we demonstrate that PLD effectively ranks SOTA online mapping methods (MapTRv2, StreamMapNet, MapTracker) while providing a decomposed error analysis. This analysis identifies detection capability as the dominant bottleneck in current methods, revealing a performance trend that mAP fails to capture. Code for evaluation using our metrics will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。