用知识蒸馏让Mamba模型学会Transformer的长程建模,提升3D点云检测速度与精度。
Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation
- 跨模型蒸馏,将Transformer的全局建模能力传给Mamba学生模型。
- 在Waymo和nuScenes上实现FLOPs减半、内存降低75%,精度提升1-2%。
- 适合追求实时高效3D感知的自动驾驶研发者使用。
面向自动驾驶的实时3D感知,高效准确的激光雷达检测模型至关重要。现有模型多依赖复杂特征变换,导致实时性差、资源消耗高。本文提出FASD框架,通过自适应对齐稀疏体素,实现异构模型间知识蒸馏,将Transformer的序列建模能力注入Mamba模型,显著提升性能。教师模型采用尺度自适应注意力机制强化多尺度融合;学生模型则通过空间对齐适配器,以潜在空间特征和跨度头输出分布为监督信号,实现高效且精准的检测。在Waymo和nuScenes数据集上,该方法实现高达2倍的浮点运算量(FLOPs)减少和4倍的内存消耗降低,同时相较基线提升1-2个百分点的检测精度,并保持高部署效率。
原文摘要 · Abstract (English)
The LiDAR 3D object detector that balances accuracy and speed is crucial for achieving real-time perception in autonomous driving. However, many existing LiDAR detection models rely on complex feature transformations, leading to poor real-time performance and high resource consumption, which limits their practical effectiveness. In this work, we propose a Faster LiDAR 3D object detection framework that Adaptively aligns Sparse voxels to enable efficient heterogeneous knowledge Distillation, called FASD. We aim to distill the Transformer's sequence modeling capability into Mamba models, significantly boosting accuracy through knowledge transfer. Specifically, we first design a cross-model knowledge distillation architecture to convey the global contextual understanding capabilities of the Transformer to Mamba. The Transformer-based teacher model employs a scale-adaptive attention mechanism to enhance multi-scale fusion. In contrast, the Mamba-based student model leverages feature alignment through spatial alignment adapters, supervised with latent-space features and span-head logit distributions, leading to improved performance and efficiency. We evaluated FASD on the Waymo and nuScenes datasets, achieving up to a 2x reduction in FLOPs and a 4x reduction in memory consumption, while improving baseline performance by 1-2 percentage points and maintaining high deployment efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。