用离散向量量化生成鸟瞰图地图,提升复杂环境下的语义一致性。
VQ-Map: Bird's-Eye-View Map Layout Estimation in Tokenized Discrete Space via Vector Quantization
- 通过VQ-VAE思想在离散空间学习鸟瞰图语义先验
- 在nuScenes上达62.2/47.6平均IoU,Argoverse上73.4 IoU
- 适合自动驾驶地图建模与多视角融合研究者
鸟瞰图(BEV)地图布局估计需要对自车周围环境元素有准确完整的语义理解,以确保结果连贯且真实。由于遮挡、成像条件差和低分辨率等问题,近期研究关注如何从透视图(PV)的损坏或无效区域生成对应的BEV语义地图。核心挑战在于如何对齐PV特征与生成模型。本文提出利用类似向量量化变分自编码器(VQ-VAE)的生成模型,在离散令牌空间中获取高层BEV语义先验。借助来自真实地图的BEV令牌及其对应的码本嵌入,我们设计专用令牌解码模块,直接将稀疏图像特征与生成的BEV令牌对齐,最终以码本嵌入为桥梁生成高质量BEV地图。在nuScenes和Argoverse基准上评估,VQ-Map在环绕视图/单目评测中分别取得62.2/47.6平均交并比,在Argoverse单目评测中达到73.4平均交并比,刷新该任务记录。
原文摘要 · Abstract (English)
Bird's-eye-view (BEV) map layout estimation requires an accurate and full understanding of the semantics for the environmental elements around the ego car to make the results coherent and realistic. Due to the challenges posed by occlusion, unfavourable imaging conditions and low resolution, \emph{generating} the BEV semantic maps corresponding to corrupted or invalid areas in the perspective view (PV) is appealing very recently. \emph{The question is how to align the PV features with the generative models to facilitate the map estimation}. In this paper, we propose to utilize a generative model similar to the Vector Quantized-Variational AutoEncoder (VQ-VAE) to acquire prior knowledge for the high-level BEV semantics in the tokenized discrete space. Thanks to the obtained BEV tokens accompanied with a codebook embedding encapsulating the semantics for different BEV elements in the groundtruth maps, we are able to directly align the sparse backbone image features with the obtained BEV tokens from the discrete representation learning based on a specialized token decoder module, and finally generate high-quality BEV maps with the BEV codebook embedding serving as a bridge between PV and BEV. We evaluate the BEV map layout estimation performance of our model, termed VQ-Map, on both the nuScenes and Argoverse benchmarks, achieving 62.2/47.6 mean IoU for surround-view/monocular evaluation on nuScenes, as well as 73.4 IoU for monocular evaluation on Argoverse, which all set a new record for this map layout estimation task. The code and models are available on \url{https://github.com/Z1zyw/VQ-Map}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。