用跨模态蒸馏让小模型学会大模型的地图知识,提速又提准。
MapKD: Unlocking Prior Knowledge with Cross-Modal Distillation for Efficient Online HD Map Construction
- 用教师-教练-学生框架,把多模态先验知识传给纯视觉小模型。
- 在nuScenes上提升6.68点mIoU、10.94点mAP,推理速度更快。
- 适合追求轻量高效在线高精地图构建的自动驾驶系统使用。
在线高精地图构建是自动驾驶系统的核心任务,旨在基于实时传感器输入获取车辆周边地图元素的语义信息。现有方法虽利用离线先验(如SD/HD地图)或融合多模态数据取得进展,但依赖过时的离线地图和多传感器配置,导致推理计算开销过高。为此,本文提出MapKD,一种新型多层级跨模态知识蒸馏框架,采用教师-教练-学生(TCS)范式:以带地图先验的相机-LiDAR融合模型为教师;引入含先验与模拟激光雷达的视觉中心教练模型,弥合跨模态知识传递鸿沟;最终训练一个轻量级纯视觉学生模型。设计了两种针对性蒸馏策略:用于鸟瞰图特征对齐的Token-Guided 2D Patch Distillation(TGPD),以及用于语义学习引导的Masked Semantic Response Distillation(MSRD)。在挑战性nuScenes数据集上的大量实验表明,MapKD使学生模型在保持高效的同时,实现+6.68 mIoU和+10.94 mAP的性能提升。代码已开源。
原文摘要 · Abstract (English)
Online HD map construction is a fundamental task in autonomous driving systems, aiming to acquire semantic information of map elements around the ego vehicle based on real-time sensor inputs. Recently, several approaches have achieved promising results by incorporating offline priors such as SD maps and HD maps or by fusing multi-modal data. However, these methods depend on stale offline maps and multi-modal sensor suites, resulting in avoidable computational overhead at inference. To address these limitations, we employ a knowledge distillation strategy to transfer knowledge from multimodal models with prior knowledge to an efficient, low-cost, and vision-centric student model. Specifically, we propose MapKD, a novel multi-level cross-modal knowledge distillation framework with an innovative Teacher-Coach-Student (TCS) paradigm. This framework consists of: (1) a camera-LiDAR fusion model with SD/HD map priors serving as the teacher; (2) a vision-centric coach model with prior knowledge and simulated LiDAR to bridge the cross-modal knowledge transfer gap; and (3) a lightweight vision-based student model. Additionally, we introduce two targeted knowledge distillation strategies: Token-Guided 2D Patch Distillation (TGPD) for bird's eye view feature alignment and Masked Semantic Response Distillation (MSRD) for semantic learning guidance. Extensive experiments on the challenging nuScenes dataset demonstrate that MapKD improves the student model by +6.68 mIoU and +10.94 mAP while simultaneously accelerating inference speed. The code is available at:https://github.com/2004yan/MapKD2026.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。